2024-04-16T05:10:34.714 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-16T05:10:34.715 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_mchangir@teuthology --name mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --verbose --archive /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900 -- /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/orig.config.yaml 2024-04-16T05:10:34.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T05:10:34.870 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900 branch: qa-debug-resource-temporarily-unavailable-issue description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: mchangir@redhat.com first_in_suite: false job_id: '7657900' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: qa-debug-resource-temporarily-unavailable-issue ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a87fcff6340baeedd43af126e793f3241b6400ea subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: a87fcff6340baeedd43af126e793f3241b6400ea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: qa-debug-resource-temporarily-unavailable-issue sha1: a87fcff6340baeedd43af126e793f3241b6400ea owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2991 sha1: a87fcff6340baeedd43af126e793f3241b6400ea sleep_before_teardown: 0 suite: fs suite_branch: qa-debug-resource-temporarily-unavailable-issue suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a87fcff6340baeedd43af126e793f3241b6400ea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOUw9861A2kAugXtARfjhVPkJuj32IRx3km2XaBKw+HCrIj1EzImIZKfMFJPwyrDHls5zdNClsKAgEEZtZSxE84= smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDlv4FQ2yvg1V1SYekmUQEmzpivEfefYfw5pWw7UWEcNFS/JhgGGDT5BI0ZGTbX94+miYPd7Ex76z8iOmnMvk/Y= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/CC4ob0jVLDKDhUZ6Lm6BL6F+8cQ5JQP35kE2KKVTnpj+yO2ef8tzSCEWSuSMU1EfE/szdGmXlqW4Db7haK5s= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-16_05:02:59 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-16T05:10:34.871 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa; will attempt to use it 2024-04-16T05:10:34.872 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks 2024-04-16T05:10:34.872 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-16T05:10:34.873 INFO:teuthology.task.internal:Checking packages... 2024-04-16T05:10:34.897 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a87fcff6340baeedd43af126e793f3241b6400ea' 2024-04-16T05:10:34.897 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-16T05:10:34.897 INFO:teuthology.packaging:ref: None 2024-04-16T05:10:34.897 INFO:teuthology.packaging:tag: None 2024-04-16T05:10:34.897 INFO:teuthology.packaging:branch: qa-debug-resource-temporarily-unavailable-issue 2024-04-16T05:10:34.897 INFO:teuthology.packaging:sha1: a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:10:34.897 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=qa-debug-resource-temporarily-unavailable-issue 2024-04-16T05:10:35.035 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3101.ga87fcff6 2024-04-16T05:10:35.038 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-16T05:10:35.044 INFO:teuthology.task.internal:no buildpackages task found 2024-04-16T05:10:35.044 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-16T05:10:35.070 INFO:teuthology.task.internal:Saving configuration 2024-04-16T05:10:35.091 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-16T05:10:35.104 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T05:10:35.129 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:47.818500', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOUw9861A2kAugXtARfjhVPkJuj32IRx3km2XaBKw+HCrIj1EzImIZKfMFJPwyrDHls5zdNClsKAgEEZtZSxE84='} 2024-04-16T05:10:35.151 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:47.817236', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDlv4FQ2yvg1V1SYekmUQEmzpivEfefYfw5pWw7UWEcNFS/JhgGGDT5BI0ZGTbX94+miYPd7Ex76z8iOmnMvk/Y='} 2024-04-16T05:10:35.173 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:47.815343', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/CC4ob0jVLDKDhUZ6Lm6BL6F+8cQ5JQP35kE2KKVTnpj+yO2ef8tzSCEWSuSMU1EfE/szdGmXlqW4Db7haK5s='} 2024-04-16T05:10:35.173 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-16T05:10:35.180 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-16T05:10:35.180 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-16T05:10:35.180 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-16T05:10:35.180 INFO:teuthology.run_tasks:Running task console_log... 2024-04-16T05:10:35.339 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3d20f160d0>, signals=[15]) 2024-04-16T05:10:35.340 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-16T05:10:35.346 INFO:teuthology.task.internal:Opening connections... 2024-04-16T05:10:35.346 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:10:35.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:10:35.453 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-04-16T05:10:35.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:10:35.543 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-04-16T05:10:35.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:10:35.629 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-16T05:10:35.636 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-04-16T05:10:35.655 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-04-16T05:10:35.655 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-04-16T05:10:35.712 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Stream" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="9" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="9" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el9" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:LOGO="fedora-logo-icon" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2024-04-16T05:10:35.713 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T05:10:35.714 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T05:10:35.714 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T05:10:35.714 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-04-16T05:10:35.738 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-04-16T05:10:35.756 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-04-16T05:10:35.756 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-04-16T05:10:35.813 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-04-16T05:10:35.813 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-04-16T05:10:35.813 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-04-16T05:10:35.813 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T05:10:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T05:10:35.815 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-04-16T05:10:35.837 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-04-16T05:10:35.855 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-04-16T05:10:35.855 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-04-16T05:10:35.911 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T05:10:35.912 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T05:10:35.913 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T05:10:35.913 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-04-16T05:10:35.938 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-16T05:10:35.948 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-16T05:10:35.972 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-16T05:10:35.972 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-04-16T05:10:35.974 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-04-16T05:10:35.977 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-04-16T05:10:35.992 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-16T05:10:35.997 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-16T05:10:35.997 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-04-16T05:10:36.031 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-04-16T05:10:36.033 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-04-16T05:10:36.059 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-16T05:10:36.134 INFO:teuthology.run_tasks:Running task kernel... 2024-04-16T05:10:36.153 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-16T05:10:36.154 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-16T05:10:36.154 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-16T05:10:36.154 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T05:10:36.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:10:36.171 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-16T05:10:36.228 INFO:teuthology.orchestra.run.smithi032.stdout:5.14.0-361.el9.x86_64 2024-04-16T05:10:36.229 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.14.0-361.el9.x86_64 2024-04-16T05:10:36.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum install -y kernel 2024-04-16T05:10:39.897 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS-9 - AppStream 3.4 MB/s | 6.7 MB 00:01 2024-04-16T05:10:42.089 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS-9 - BaseOS 1.9 MB/s | 2.0 MB 00:01 2024-04-16T05:10:43.329 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-04-16T05:10:51.032 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 19 MB 00:06 2024-04-16T05:11:03.038 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 754 kB/s | 6.2 MB 00:08 2024-04-16T05:11:04.605 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-16T05:11:26.175 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 994 kB/s | 21 MB 00:21 2024-04-16T05:11:31.372 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-04-16T05:11:33.803 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-16T05:11:33.804 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-16T05:11:33.901 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-04-16T05:11:33.909 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-16T05:11:33.909 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-04-16T05:11:33.909 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-16T05:11:33.909 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-04-16T05:11:33.909 INFO:teuthology.orchestra.run.smithi032.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-16T05:11:33.909 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout:Install 4 Packages 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 77 M 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 123 M 2024-04-16T05:11:33.910 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-04-16T05:11:34.065 INFO:teuthology.orchestra.run.smithi032.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.8 MB/s | 277 kB 00:00 2024-04-16T05:11:34.673 INFO:teuthology.orchestra.run.smithi032.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 20 MB/s | 15 MB 00:00 2024-04-16T05:11:35.098 INFO:teuthology.orchestra.run.smithi032.stdout:(3/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 28 MB/s | 34 MB 00:01 2024-04-16T05:11:35.416 INFO:teuthology.orchestra.run.smithi032.stdout:(4/4): kernel-modules-core-5.14.0-438.el9.x86_6 21 MB/s | 28 MB 00:01 2024-04-16T05:11:35.418 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:11:35.419 INFO:teuthology.orchestra.run.smithi032.stdout:Total 51 MB/s | 77 MB 00:01 2024-04-16T05:11:35.419 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-04-16T05:11:35.855 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-04-16T05:11:35.855 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-04-16T05:11:36.718 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-04-16T05:11:36.718 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-04-16T05:11:39.135 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-04-16T05:11:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-16T05:11:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T05:11:42.746 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T05:11:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T05:11:47.953 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T05:11:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:11:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:24.636 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:24.636 INFO:teuthology.orchestra.run.smithi032.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-16T05:12:24.636 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:12:24.680 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-16T05:12:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T05:12:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T05:12:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T05:12:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:12:25.479 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-04-16T05:12:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-16T05:12:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-16T05:12:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-16T05:12:25.480 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-16T05:12:25.480 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:12:25.480 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-04-16T05:12:25.820 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T05:12:25.927 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T05:12:25.927 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T05:12:25.927 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-16T05:12:25.927 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T05:12:25.928 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T05:12:25.928 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-16T05:12:25.928 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T05:12:25.928 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T05:12:26.011 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T05:12:26.011 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T05:12:26.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -qi grub2-tools 2024-04-16T05:12:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:Name : grub2-tools 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Epoch : 1 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Version : 2.06 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Release : 68.el9 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: x86_64 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Group : Unspecified 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Size : 8264254 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:License : GPLv3+ 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-16T05:12:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:Packager : builder@centos.org 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:Vendor : CentOS 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:Summary : Support tools for GRUB. 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:Description : 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:hardware devices. 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:12:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:This subpackage provides tools for support of all platforms. 2024-04-16T05:12:26.057 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-16T05:12:26.057 INFO:teuthology.task.kernel:Updating grub on smithi032 to boot 5.14.0-438.el9.x86_64 2024-04-16T05:12:26.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-16T05:12:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-04-16T05:12:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-16T05:12:28.114 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-04-16T05:12:28.119 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-16T05:12:28.135 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.1lDlQmE1hb 2024-04-16T05:12:28.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1lDlQmE1hb 2024-04-16T05:12:28.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.1lDlQmE1hb 2024-04-16T05:12:28.347 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.1lDlQmE1hb is 6KB 2024-04-16T05:12:28.396 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.1lDlQmE1hb 2024-04-16T05:12:28.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo /bin/ls /boot/loader/entries || true 2024-04-16T05:12:28.478 INFO:teuthology.orchestra.run.smithi032.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-16T05:12:28.478 INFO:teuthology.orchestra.run.smithi032.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-16T05:12:28.478 INFO:teuthology.orchestra.run.smithi032.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-16T05:12:28.478 INFO:teuthology.orchestra.run.smithi032.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-16T05:12:28.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-16T05:12:28.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2024-04-16T05:12:58.634 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-16T05:12:58.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-16T05:12:58.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:13:17.090 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-16T05:13:26.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-16T05:13:26.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:13:44.482 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-16T05:13:56.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-16T05:13:56.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:13:56.857 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-16T05:13:57.336 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-16T05:13:57.336 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-438.el9.x86_64"... 2024-04-16T05:13:57.337 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-16T05:13:57.351 INFO:teuthology.orchestra.run.smithi032.stdout:5.14.0-438.el9.x86_64 2024-04-16T05:13:57.351 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-16T05:13:57.351 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T05:13:57.351 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-16T05:13:58.352 DEBUG:teuthology.parallel:result is None 2024-04-16T05:13:58.353 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-16T05:13:58.360 INFO:teuthology.task.internal:Creating test directory... 2024-04-16T05:13:58.361 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T05:13:58.364 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T05:13:58.367 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T05:13:58.385 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-16T05:13:58.443 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-16T05:13:58.489 INFO:teuthology.task.internal:Creating archive directory... 2024-04-16T05:13:58.489 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T05:13:58.492 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T05:13:58.494 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T05:13:58.601 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-16T05:13:58.609 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-16T05:13:58.609 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T05:13:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:13:58.626 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T05:13:58.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:13:58.643 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T05:13:58.646 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T05:13:58.669 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T05:13:58.705 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:13:58.727 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:13:58.730 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:13:58.752 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:13:58.776 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:13:58.793 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T05:13:58.795 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-16T05:13:58.802 INFO:teuthology.task.internal:Configuring sudo... 2024-04-16T05:13:58.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T05:13:58.806 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T05:13:58.808 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T05:13:58.839 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-16T05:13:58.848 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-16T05:13:58.848 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T05:13:58.879 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T05:13:58.881 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T05:13:58.897 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:13:58.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:13:59.032 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:13:59.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:13:59.168 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:13:59.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T05:13:59.255 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:13:59.298 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:13:59.353 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:13:59.436 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:13:59.483 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:13:59.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T05:13:59.552 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:13:59.595 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T05:13:59.650 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:13:59.735 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T05:13:59.780 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:13:59.780 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T05:13:59.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-04-16T05:13:59.851 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-04-16T05:13:59.853 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-04-16T05:13:59.890 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:13:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:13:59.939 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:14:00.410 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-16T05:14:00.418 INFO:teuthology.task.internal:Starting timer... 2024-04-16T05:14:00.418 INFO:teuthology.run_tasks:Running task pcp... 2024-04-16T05:14:00.467 INFO:teuthology.run_tasks:Running task selinux... 2024-04-16T05:14:00.516 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-16T05:14:00.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2024-04-16T05:14:00.567 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2024-04-16T05:14:00.569 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-04-16T05:14:00.639 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-04-16T05:14:00.640 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-04-16T05:14:00.717 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-04-16T05:14:00.719 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-16T05:14:00.719 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2024-04-16T05:14:00.740 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2024-04-16T05:14:00.740 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-04-16T05:14:00.763 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-04-16T05:14:00.763 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-04-16T05:14:00.796 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-04-16T05:14:00.797 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-04-16T05:14:00.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:14:00.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:14:00.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:14:00.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:14:00.855 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:14:00.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:14:00.883 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-16T05:14:00.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2024-04-16T05:14:00.934 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-04-16T05:14:00.973 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-04-16T05:14:01.007 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-16T05:14:01.021 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-16T05:14:01.064 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-16T05:14:01.065 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi082.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-16T05:19:29.480 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-04-16T05:19:29.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-16T05:19:29.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:19:29.578 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-16T05:19:29.653 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-16T05:19:29.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-16T05:19:29.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:19:29.743 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-16T05:19:29.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-16T05:19:29.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-16T05:19:29.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T05:19:29.897 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-04-16T05:19:29.969 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-16T05:19:29.969 INFO:teuthology.run_tasks:Running task clock... 2024-04-16T05:19:29.980 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-16T05:19:29.981 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T05:19:29.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:19:29.983 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T05:19:29.983 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:19:29.986 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T05:19:29.986 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:19:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T05:19:30.014 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T05:19:30.031 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T05:19:30.032 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T05:19:30.041 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T05:19:30.058 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T05:19:30.060 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-04-16T05:19:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2024-04-16T05:19:30.074 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-04-16T05:19:30.077 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2024-04-16T05:19:30.088 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-04-16T05:19:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T05:19:30.095 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T05:19:30.102 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-04-16T05:19:30.106 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T05:19:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T05:19:30.118 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T05:19:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T05:19:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:19:30.152 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:19:30.153 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-04-16T05:19:30.153 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.153 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.153 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.153 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:19:30.169 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:19:30.169 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-04-16T05:19:30.169 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.169 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.169 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.170 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.178 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:19:30.181 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:19:30.181 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-16T05:19:30.182 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.182 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.182 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.182 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:19:30.182 INFO:teuthology.run_tasks:Running task install... 2024-04-16T05:19:30.190 DEBUG:teuthology.task.install:project ceph 2024-04-16T05:19:30.191 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea'}} 2024-04-16T05:19:30.191 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea'} 2024-04-16T05:19:30.191 INFO:teuthology.task.install:Using flavor: default 2024-04-16T05:19:30.198 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-16T05:19:30.199 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-16T05:19:30.199 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'tag': None, 'wait_for_package': False} 2024-04-16T05:19:30.199 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:19:30.202 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'tag': None, 'wait_for_package': False} 2024-04-16T05:19:30.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:19:30.203 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'tag': None, 'wait_for_package': False} 2024-04-16T05:19:30.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:19:30.342 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/ 2024-04-16T05:19:30.342 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.ga87fcff6 2024-04-16T05:19:30.351 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/ 2024-04-16T05:19:30.351 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.ga87fcff6 2024-04-16T05:19:30.453 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T05:19:30.453 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:19:30.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T05:19:30.461 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T05:19:30.461 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:19:30.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T05:19:30.482 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T05:19:30.483 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a87fcff6340baeedd43af126e793f3241b6400ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T05:19:30.488 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T05:19:30.488 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a87fcff6340baeedd43af126e793f3241b6400ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T05:19:30.527 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/ 2024-04-16T05:19:30.527 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.ga87fcff6 2024-04-16T05:19:30.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T05:19:30.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T05:19:30.633 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T05:19:30.635 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T05:19:30.701 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-04-16T05:19:30.702 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2024-04-16T05:19:30.703 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-04-16T05:19:30.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2024-04-16T05:19:30.722 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T05:19:30.722 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:19:30.722 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T05:19:30.753 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T05:19:30.753 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a87fcff6340baeedd43af126e793f3241b6400ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T05:19:30.819 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T05:19:30.898 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T05:19:30.936 INFO:teuthology.orchestra.run.smithi082.stdout:74 files removed 2024-04-16T05:19:30.951 INFO:teuthology.orchestra.run.smithi032.stdout:74 files removed 2024-04-16T05:19:30.963 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T05:19:30.964 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-04-16T05:19:30.966 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-04-16T05:19:30.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T05:19:31.198 INFO:teuthology.orchestra.run.smithi134.stdout:74 files removed 2024-04-16T05:19:31.225 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T05:19:31.566 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 268 kB/s | 83 kB 00:00 2024-04-16T05:19:31.808 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 161 kB/s | 83 kB 00:00 2024-04-16T05:19:31.891 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-04-16T05:19:32.008 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 169 kB/s | 83 kB 00:00 2024-04-16T05:19:32.209 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-16T05:19:32.250 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-04-16T05:19:32.492 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-16T05:19:32.593 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-04-16T05:19:32.711 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-16T05:19:32.809 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 6.6 kB/s | 1.9 kB 00:00 2024-04-16T05:19:33.145 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-16T05:19:33.253 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-04-16T05:19:34.277 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-04-16T05:19:34.612 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-04-16T05:19:34.637 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-04-16T05:19:35.620 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-16T05:19:35.911 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-04-16T05:19:37.026 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-04-16T05:19:37.662 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-16T05:19:39.079 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-16T05:19:42.332 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.2 MB 00:01 2024-04-16T05:19:43.312 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 19 MB 00:06 2024-04-16T05:19:43.998 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-16T05:19:47.755 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-04-16T05:19:49.440 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-16T05:19:50.742 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 780 kB/s | 6.2 MB 00:08 2024-04-16T05:19:53.093 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-16T05:19:54.020 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-04-16T05:19:55.365 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 21 MB 00:05 2024-04-16T05:19:58.969 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-04-16T05:20:00.698 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-04-16T05:20:01.623 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:20:01.624 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:20:01.637 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T05:20:01.638 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T05:20:01.639 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T05:20:01.640 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T05:20:01.642 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T05:20:01.645 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T05:20:01.646 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T05:20:01.725 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-04-16T05:20:01.742 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-04-16T05:20:01.742 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-04-16T05:20:01.742 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-04-16T05:20:01.742 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-04-16T05:20:01.742 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T05:20:01.742 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 6.6 k 2024-04-16T05:20:01.742 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 890 k 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 145 k 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 1.5 M 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 142 k 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 3.5 M 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 7.4 M 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 49 k 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 11 M 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 49 M 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 273 k 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 762 k 2024-04-16T05:20:01.743 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 224 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 27 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 18 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 32 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 746 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 127 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T05:20:01.744 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 161 k 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 321 k 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 298 k 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 100 k 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 85 k 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 170 k 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-04-16T05:20:01.745 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T05:20:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.4 M 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-04-16T05:20:01.747 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 21 M 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 34 k 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 2.4 M 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 249 k 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 4.7 M 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 17 M 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 16 k 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 25 k 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T05:20:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 163 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T05:20:01.749 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 501 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T05:20:01.750 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 45 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 135 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T05:20:01.751 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T05:20:01.752 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T05:20:01.753 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T05:20:01.754 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T05:20:01.755 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T05:20:01.756 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-04-16T05:20:01.757 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T05:20:01.757 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:20:01.757 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-04-16T05:20:01.757 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-04-16T05:20:01.757 INFO:teuthology.orchestra.run.smithi082.stdout:Install 150 Packages 2024-04-16T05:20:01.757 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 24 Packages 2024-04-16T05:20:01.757 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:20:01.758 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 229 M 2024-04-16T05:20:01.758 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-04-16T05:20:03.145 INFO:teuthology.orchestra.run.smithi082.stdout:(1/174): ceph-19.0.0-3101.ga87fcff6.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-16T05:20:03.471 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:20:03.472 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:20:03.487 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T05:20:03.488 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T05:20:03.490 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T05:20:03.491 INFO:teuthology.orchestra.run.smithi032.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T05:20:03.493 INFO:teuthology.orchestra.run.smithi032.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T05:20:03.496 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T05:20:03.497 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T05:20:03.582 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-04-16T05:20:03.598 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================ 2024-04-16T05:20:03.598 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-04-16T05:20:03.598 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================ 2024-04-16T05:20:03.598 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-04-16T05:20:03.598 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 6.6 k 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 890 k 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 145 k 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 1.5 M 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 142 k 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 3.5 M 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 7.4 M 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 49 k 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 11 M 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 49 M 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 273 k 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 762 k 2024-04-16T05:20:03.599 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 224 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 27 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 18 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 32 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 746 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 127 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T05:20:03.600 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 161 k 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 321 k 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 298 k 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 100 k 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 85 k 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 170 k 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2024-04-16T05:20:03.601 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.4 M 2024-04-16T05:20:03.602 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T05:20:03.603 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 21 M 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 34 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 2.4 M 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 249 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 4.7 M 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 17 M 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 16 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 25 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T05:20:03.604 INFO:teuthology.orchestra.run.smithi032.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 163 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 501 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T05:20:03.605 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T05:20:03.606 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 45 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 135 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T05:20:03.607 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T05:20:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T05:20:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T05:20:03.610 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout:Installing weak dependencies: 2024-04-16T05:20:03.611 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T05:20:03.612 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:20:03.612 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-04-16T05:20:03.612 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================ 2024-04-16T05:20:03.612 INFO:teuthology.orchestra.run.smithi032.stdout:Install 150 Packages 2024-04-16T05:20:03.612 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 24 Packages 2024-04-16T05:20:03.612 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:20:03.612 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 229 M 2024-04-16T05:20:03.613 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-04-16T05:20:03.708 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-16T05:20:04.632 INFO:teuthology.orchestra.run.smithi032.stdout:(1/174): ceph-19.0.0-3101.ga87fcff6.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-16T05:20:05.333 INFO:teuthology.orchestra.run.smithi032.stdout:(2/174): ceph-fuse-19.0.0-3101.ga87fcff6.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-16T05:20:05.450 INFO:teuthology.orchestra.run.smithi032.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 1.2 MB/s | 145 kB 00:00 2024-04-16T05:20:05.837 INFO:teuthology.orchestra.run.smithi082.stdout:(2/174): ceph-base-19.0.0-3101.ga87fcff6.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-04-16T05:20:05.995 INFO:teuthology.orchestra.run.smithi082.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 918 kB/s | 145 kB 00:00 2024-04-16T05:20:07.001 INFO:teuthology.orchestra.run.smithi032.stdout:(4/174): ceph-mds-19.0.0-3101.ga87fcff6.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-04-16T05:20:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:(4/174): ceph-mds-19.0.0-3101.ga87fcff6.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-04-16T05:20:07.503 INFO:teuthology.orchestra.run.smithi082.stdout:(5/174): ceph-fuse-19.0.0-3101.ga87fcff6.el9.x8 204 kB/s | 890 kB 00:04 2024-04-16T05:20:07.850 INFO:teuthology.orchestra.run.smithi032.stdout:(5/174): ceph-base-19.0.0-3101.ga87fcff6.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-16T05:20:08.142 INFO:teuthology.orchestra.run.smithi032.stdout:(6/174): ceph-mgr-19.0.0-3101.ga87fcff6.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-04-16T05:20:08.821 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-16T05:20:09.934 INFO:teuthology.orchestra.run.smithi032.stdout:(7/174): ceph-mon-19.0.0-3101.ga87fcff6.el9.x86 2.3 MB/s | 4.7 MB 00:02 2024-04-16T05:20:10.962 INFO:teuthology.orchestra.run.smithi082.stdout:(6/174): ceph-mon-19.0.0-3101.ga87fcff6.el9.x86 1.4 MB/s | 4.7 MB 00:03 2024-04-16T05:20:11.477 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:20:11.478 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T05:20:11.492 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T05:20:11.493 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T05:20:11.494 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T05:20:11.495 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T05:20:11.497 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T05:20:11.500 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T05:20:11.501 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T05:20:11.512 INFO:teuthology.orchestra.run.smithi082.stdout:(7/174): ceph-mgr-19.0.0-3101.ga87fcff6.el9.x86 380 kB/s | 1.5 MB 00:04 2024-04-16T05:20:11.583 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-16T05:20:11.605 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-04-16T05:20:11.605 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-04-16T05:20:11.605 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-04-16T05:20:11.605 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-04-16T05:20:11.605 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T05:20:11.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 6.6 k 2024-04-16T05:20:11.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:20:11.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 890 k 2024-04-16T05:20:11.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 145 k 2024-04-16T05:20:11.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 1.5 M 2024-04-16T05:20:11.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 142 k 2024-04-16T05:20:11.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 3.5 M 2024-04-16T05:20:11.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 7.4 M 2024-04-16T05:20:11.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 49 k 2024-04-16T05:20:11.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 11 M 2024-04-16T05:20:11.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 49 M 2024-04-16T05:20:11.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 273 k 2024-04-16T05:20:11.608 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 762 k 2024-04-16T05:20:11.608 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 224 k 2024-04-16T05:20:11.608 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 27 k 2024-04-16T05:20:11.608 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 18 k 2024-04-16T05:20:11.608 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T05:20:11.608 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T05:20:11.609 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T05:20:11.609 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 32 k 2024-04-16T05:20:11.609 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 746 k 2024-04-16T05:20:11.609 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 127 k 2024-04-16T05:20:11.609 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T05:20:11.610 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T05:20:11.610 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T05:20:11.610 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T05:20:11.610 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T05:20:11.610 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T05:20:11.610 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 161 k 2024-04-16T05:20:11.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 321 k 2024-04-16T05:20:11.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 298 k 2024-04-16T05:20:11.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 100 k 2024-04-16T05:20:11.611 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 85 k 2024-04-16T05:20:11.611 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:20:11.611 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 170 k 2024-04-16T05:20:11.612 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T05:20:11.612 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-04-16T05:20:11.612 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T05:20:11.613 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T05:20:11.613 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T05:20:11.613 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T05:20:11.613 INFO:teuthology.orchestra.run.smithi134.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T05:20:11.613 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.4 M 2024-04-16T05:20:11.614 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 3.1 M 2024-04-16T05:20:11.615 INFO:teuthology.orchestra.run.smithi134.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T05:20:11.615 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T05:20:11.615 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T05:20:11.615 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T05:20:11.615 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T05:20:11.615 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 21 M 2024-04-16T05:20:11.616 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 34 k 2024-04-16T05:20:11.617 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 2.4 M 2024-04-16T05:20:11.617 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 249 k 2024-04-16T05:20:11.617 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 4.7 M 2024-04-16T05:20:11.617 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 17 M 2024-04-16T05:20:11.617 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.ga87fcff6.el9 ceph-noarch 16 k 2024-04-16T05:20:11.617 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 25 k 2024-04-16T05:20:11.617 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T05:20:11.617 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T05:20:11.618 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T05:20:11.618 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T05:20:11.618 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T05:20:11.618 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T05:20:11.618 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T05:20:11.618 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T05:20:11.618 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T05:20:11.618 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 163 k 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 501 k 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 5.4 M 2024-04-16T05:20:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T05:20:11.620 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T05:20:11.620 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T05:20:11.620 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T05:20:11.620 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T05:20:11.620 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T05:20:11.620 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T05:20:11.620 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T05:20:11.620 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 45 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.0.0-3101.ga87fcff6.el9 ceph 135 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T05:20:11.621 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T05:20:11.622 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T05:20:11.623 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T05:20:11.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T05:20:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout:Install 150 Packages 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 24 Packages 2024-04-16T05:20:11.626 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:20:11.627 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 229 M 2024-04-16T05:20:11.627 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-04-16T05:20:12.631 INFO:teuthology.orchestra.run.smithi134.stdout:(1/174): ceph-19.0.0-3101.ga87fcff6.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-16T05:20:13.514 INFO:teuthology.orchestra.run.smithi134.stdout:(2/174): ceph-fuse-19.0.0-3101.ga87fcff6.el9.x8 1.0 MB/s | 890 kB 00:00 2024-04-16T05:20:13.639 INFO:teuthology.orchestra.run.smithi134.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 1.1 MB/s | 145 kB 00:00 2024-04-16T05:20:14.573 INFO:teuthology.orchestra.run.smithi134.stdout:(4/174): ceph-mds-19.0.0-3101.ga87fcff6.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-04-16T05:20:15.465 INFO:teuthology.orchestra.run.smithi134.stdout:(5/174): ceph-mgr-19.0.0-3101.ga87fcff6.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-16T05:20:17.642 INFO:teuthology.orchestra.run.smithi032.stdout:(8/174): ceph-common-19.0.0-3101.ga87fcff6.el9. 1.6 MB/s | 21 MB 00:13 2024-04-16T05:20:17.734 INFO:teuthology.orchestra.run.smithi032.stdout:(9/174): ceph-selinux-19.0.0-3101.ga87fcff6.el9 276 kB/s | 25 kB 00:00 2024-04-16T05:20:19.048 INFO:teuthology.orchestra.run.smithi134.stdout:(6/174): ceph-mon-19.0.0-3101.ga87fcff6.el9.x86 1.3 MB/s | 4.7 MB 00:03 2024-04-16T05:20:21.394 INFO:teuthology.orchestra.run.smithi082.stdout:(8/174): ceph-common-19.0.0-3101.ga87fcff6.el9. 1.1 MB/s | 21 MB 00:18 2024-04-16T05:20:21.561 INFO:teuthology.orchestra.run.smithi082.stdout:(9/174): ceph-selinux-19.0.0-3101.ga87fcff6.el9 151 kB/s | 25 kB 00:00 2024-04-16T05:20:21.606 INFO:teuthology.orchestra.run.smithi134.stdout:(7/174): ceph-base-19.0.0-3101.ga87fcff6.el9.x8 583 kB/s | 5.4 MB 00:09 2024-04-16T05:20:23.401 INFO:teuthology.orchestra.run.smithi032.stdout:(10/174): ceph-radosgw-19.0.0-3101.ga87fcff6.el 806 kB/s | 11 MB 00:13 2024-04-16T05:20:24.134 INFO:teuthology.orchestra.run.smithi032.stdout:(11/174): cephfs-mirror-19.0.0-3101.ga87fcff6.e 306 kB/s | 224 kB 00:00 2024-04-16T05:20:24.251 INFO:teuthology.orchestra.run.smithi032.stdout:(12/174): cephfs-shell-19.0.0-3101.ga87fcff6.el 232 kB/s | 27 kB 00:00 2024-04-16T05:20:24.376 INFO:teuthology.orchestra.run.smithi032.stdout:(13/174): libcephfs-devel-19.0.0-3101.ga87fcff6 254 kB/s | 32 kB 00:00 2024-04-16T05:20:27.135 INFO:teuthology.orchestra.run.smithi032.stdout:(14/174): libcephfs2-19.0.0-3101.ga87fcff6.el9. 271 kB/s | 746 kB 00:02 2024-04-16T05:20:27.602 INFO:teuthology.orchestra.run.smithi032.stdout:(15/174): libcephsqlite-19.0.0-3101.ga87fcff6.e 350 kB/s | 163 kB 00:00 2024-04-16T05:20:28.002 INFO:teuthology.orchestra.run.smithi032.stdout:(16/174): librados-devel-19.0.0-3101.ga87fcff6. 317 kB/s | 127 kB 00:00 2024-04-16T05:20:28.244 INFO:teuthology.orchestra.run.smithi032.stdout:(17/174): ceph-osd-19.0.0-3101.ga87fcff6.el9.x8 857 kB/s | 17 MB 00:20 2024-04-16T05:20:29.356 INFO:teuthology.orchestra.run.smithi134.stdout:(8/174): ceph-common-19.0.0-3101.ga87fcff6.el9. 1.2 MB/s | 21 MB 00:17 2024-04-16T05:20:29.544 INFO:teuthology.orchestra.run.smithi032.stdout:(18/174): libradosstriper1-19.0.0-3101.ga87fcff 325 kB/s | 501 kB 00:01 2024-04-16T05:20:29.745 INFO:teuthology.orchestra.run.smithi032.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.ga8 226 kB/s | 45 kB 00:00 2024-04-16T05:20:29.756 INFO:teuthology.orchestra.run.smithi134.stdout:(9/174): ceph-selinux-19.0.0-3101.ga87fcff6.el9 63 kB/s | 25 kB 00:00 2024-04-16T05:20:30.161 INFO:teuthology.orchestra.run.smithi032.stdout:(20/174): python3-ceph-common-19.0.0-3101.ga87f 323 kB/s | 135 kB 00:00 2024-04-16T05:20:30.703 INFO:teuthology.orchestra.run.smithi032.stdout:(21/174): python3-cephfs-19.0.0-3101.ga87fcff6. 298 kB/s | 161 kB 00:00 2024-04-16T05:20:32.187 INFO:teuthology.orchestra.run.smithi032.stdout:(22/174): python3-rados-19.0.0-3101.ga87fcff6.e 216 kB/s | 321 kB 00:01 2024-04-16T05:20:32.929 INFO:teuthology.orchestra.run.smithi032.stdout:(23/174): python3-rbd-19.0.0-3101.ga87fcff6.el9 402 kB/s | 298 kB 00:00 2024-04-16T05:20:33.220 INFO:teuthology.orchestra.run.smithi032.stdout:(24/174): python3-rgw-19.0.0-3101.ga87fcff6.el9 342 kB/s | 100 kB 00:00 2024-04-16T05:20:33.621 INFO:teuthology.orchestra.run.smithi032.stdout:(25/174): rbd-fuse-19.0.0-3101.ga87fcff6.el9.x8 213 kB/s | 85 kB 00:00 2024-04-16T05:20:33.959 INFO:teuthology.orchestra.run.smithi082.stdout:(10/174): ceph-radosgw-19.0.0-3101.ga87fcff6.el 483 kB/s | 11 MB 00:22 2024-04-16T05:20:34.451 INFO:teuthology.orchestra.run.smithi082.stdout:(11/174): cephfs-mirror-19.0.0-3101.ga87fcff6.e 457 kB/s | 224 kB 00:00 2024-04-16T05:20:34.551 INFO:teuthology.orchestra.run.smithi082.stdout:(12/174): cephfs-shell-19.0.0-3101.ga87fcff6.el 270 kB/s | 27 kB 00:00 2024-04-16T05:20:34.651 INFO:teuthology.orchestra.run.smithi082.stdout:(13/174): libcephfs-devel-19.0.0-3101.ga87fcff6 318 kB/s | 32 kB 00:00 2024-04-16T05:20:36.060 INFO:teuthology.orchestra.run.smithi082.stdout:(14/174): libcephfs2-19.0.0-3101.ga87fcff6.el9. 530 kB/s | 746 kB 00:01 2024-04-16T05:20:36.148 INFO:teuthology.orchestra.run.smithi134.stdout:(10/174): ceph-radosgw-19.0.0-3101.ga87fcff6.el 746 kB/s | 11 MB 00:14 2024-04-16T05:20:36.460 INFO:teuthology.orchestra.run.smithi082.stdout:(15/174): libcephsqlite-19.0.0-3101.ga87fcff6.e 408 kB/s | 163 kB 00:00 2024-04-16T05:20:36.481 INFO:teuthology.orchestra.run.smithi134.stdout:(11/174): cephfs-mirror-19.0.0-3101.ga87fcff6.e 674 kB/s | 224 kB 00:00 2024-04-16T05:20:36.581 INFO:teuthology.orchestra.run.smithi134.stdout:(12/174): cephfs-shell-19.0.0-3101.ga87fcff6.el 270 kB/s | 27 kB 00:00 2024-04-16T05:20:36.682 INFO:teuthology.orchestra.run.smithi134.stdout:(13/174): libcephfs-devel-19.0.0-3101.ga87fcff6 318 kB/s | 32 kB 00:00 2024-04-16T05:20:36.751 INFO:teuthology.orchestra.run.smithi082.stdout:(16/174): librados-devel-19.0.0-3101.ga87fcff6. 434 kB/s | 127 kB 00:00 2024-04-16T05:20:37.544 INFO:teuthology.orchestra.run.smithi082.stdout:(17/174): ceph-osd-19.0.0-3101.ga87fcff6.el9.x8 648 kB/s | 17 MB 00:26 2024-04-16T05:20:37.660 INFO:teuthology.orchestra.run.smithi082.stdout:(18/174): libradosstriper1-19.0.0-3101.ga87fcff 551 kB/s | 501 kB 00:00 2024-04-16T05:20:37.748 INFO:teuthology.orchestra.run.smithi134.stdout:(14/174): libcephfs2-19.0.0-3101.ga87fcff6.el9. 700 kB/s | 746 kB 00:01 2024-04-16T05:20:37.769 INFO:teuthology.orchestra.run.smithi082.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.ga8 416 kB/s | 45 kB 00:00 2024-04-16T05:20:37.982 INFO:teuthology.orchestra.run.smithi134.stdout:(15/174): libcephsqlite-19.0.0-3101.ga87fcff6.e 700 kB/s | 163 kB 00:00 2024-04-16T05:20:38.119 INFO:teuthology.orchestra.run.smithi082.stdout:(20/174): python3-ceph-common-19.0.0-3101.ga87f 385 kB/s | 135 kB 00:00 2024-04-16T05:20:38.199 INFO:teuthology.orchestra.run.smithi134.stdout:(16/174): librados-devel-19.0.0-3101.ga87fcff6. 585 kB/s | 127 kB 00:00 2024-04-16T05:20:38.411 INFO:teuthology.orchestra.run.smithi082.stdout:(21/174): python3-cephfs-19.0.0-3101.ga87fcff6. 554 kB/s | 161 kB 00:00 2024-04-16T05:20:38.891 INFO:teuthology.orchestra.run.smithi134.stdout:(17/174): libradosstriper1-19.0.0-3101.ga87fcff 724 kB/s | 501 kB 00:00 2024-04-16T05:20:39.086 INFO:teuthology.orchestra.run.smithi082.stdout:(22/174): python3-rados-19.0.0-3101.ga87fcff6.e 475 kB/s | 321 kB 00:00 2024-04-16T05:20:39.736 INFO:teuthology.orchestra.run.smithi082.stdout:(23/174): python3-rbd-19.0.0-3101.ga87fcff6.el9 458 kB/s | 298 kB 00:00 2024-04-16T05:20:39.945 INFO:teuthology.orchestra.run.smithi082.stdout:(24/174): python3-rgw-19.0.0-3101.ga87fcff6.el9 479 kB/s | 100 kB 00:00 2024-04-16T05:20:40.153 INFO:teuthology.orchestra.run.smithi082.stdout:(25/174): rbd-fuse-19.0.0-3101.ga87fcff6.el9.x8 409 kB/s | 85 kB 00:00 2024-04-16T05:20:40.454 INFO:teuthology.orchestra.run.smithi032.stdout:(26/174): librgw2-19.0.0-3101.ga87fcff6.el9.x86 451 kB/s | 5.4 MB 00:12 2024-04-16T05:20:40.862 INFO:teuthology.orchestra.run.smithi032.stdout:(27/174): rbd-nbd-19.0.0-3101.ga87fcff6.el9.x86 417 kB/s | 170 kB 00:00 2024-04-16T05:20:40.979 INFO:teuthology.orchestra.run.smithi032.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 293 kB/s | 34 kB 00:00 2024-04-16T05:20:41.304 INFO:teuthology.orchestra.run.smithi032.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.ga87fcff 438 kB/s | 142 kB 00:00 2024-04-16T05:20:42.538 INFO:teuthology.orchestra.run.smithi032.stdout:(30/174): rbd-mirror-19.0.0-3101.ga87fcff6.el9. 355 kB/s | 3.1 MB 00:08 2024-04-16T05:20:43.386 INFO:teuthology.orchestra.run.smithi082.stdout:(26/174): librgw2-19.0.0-3101.ga87fcff6.el9.x86 942 kB/s | 5.4 MB 00:05 2024-04-16T05:20:43.712 INFO:teuthology.orchestra.run.smithi082.stdout:(27/174): rbd-nbd-19.0.0-3101.ga87fcff6.el9.x86 525 kB/s | 170 kB 00:00 2024-04-16T05:20:43.812 INFO:teuthology.orchestra.run.smithi082.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 343 kB/s | 34 kB 00:00 2024-04-16T05:20:44.070 INFO:teuthology.orchestra.run.smithi082.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.ga87fcff 550 kB/s | 142 kB 00:00 2024-04-16T05:20:45.853 INFO:teuthology.orchestra.run.smithi082.stdout:(30/174): rbd-mirror-19.0.0-3101.ga87fcff6.el9. 555 kB/s | 3.1 MB 00:05 2024-04-16T05:20:46.416 INFO:teuthology.orchestra.run.smithi134.stdout:(18/174): librgw2-19.0.0-3101.ga87fcff6.el9.x86 731 kB/s | 5.4 MB 00:07 2024-04-16T05:20:46.591 INFO:teuthology.orchestra.run.smithi134.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.ga8 258 kB/s | 45 kB 00:00 2024-04-16T05:20:46.991 INFO:teuthology.orchestra.run.smithi134.stdout:(20/174): python3-ceph-common-19.0.0-3101.ga87f 337 kB/s | 135 kB 00:00 2024-04-16T05:20:47.433 INFO:teuthology.orchestra.run.smithi134.stdout:(21/174): python3-cephfs-19.0.0-3101.ga87fcff6. 366 kB/s | 161 kB 00:00 2024-04-16T05:20:48.150 INFO:teuthology.orchestra.run.smithi134.stdout:(22/174): python3-rados-19.0.0-3101.ga87fcff6.e 448 kB/s | 321 kB 00:00 2024-04-16T05:20:48.800 INFO:teuthology.orchestra.run.smithi134.stdout:(23/174): python3-rbd-19.0.0-3101.ga87fcff6.el9 458 kB/s | 298 kB 00:00 2024-04-16T05:20:49.059 INFO:teuthology.orchestra.run.smithi134.stdout:(24/174): python3-rgw-19.0.0-3101.ga87fcff6.el9 386 kB/s | 100 kB 00:00 2024-04-16T05:20:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:(25/174): rbd-fuse-19.0.0-3101.ga87fcff6.el9.x8 465 kB/s | 85 kB 00:00 2024-04-16T05:20:49.428 INFO:teuthology.orchestra.run.smithi082.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.ga87fc 676 kB/s | 3.5 MB 00:05 2024-04-16T05:20:49.787 INFO:teuthology.orchestra.run.smithi082.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.ga8 695 kB/s | 249 kB 00:00 2024-04-16T05:20:49.887 INFO:teuthology.orchestra.run.smithi082.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.ga87fcff6.e 493 kB/s | 49 kB 00:00 2024-04-16T05:20:49.979 INFO:teuthology.orchestra.run.smithi082.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.ga 180 kB/s | 16 kB 00:00 2024-04-16T05:20:50.421 INFO:teuthology.orchestra.run.smithi082.stdout:(35/174): ceph-volume-19.0.0-3101.ga87fcff6.el9 618 kB/s | 273 kB 00:00 2024-04-16T05:20:50.904 INFO:teuthology.orchestra.run.smithi032.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.ga87fc 377 kB/s | 3.5 MB 00:09 2024-04-16T05:20:51.321 INFO:teuthology.orchestra.run.smithi082.stdout:(36/174): cephadm-19.0.0-3101.ga87fcff6.el9.noa 847 kB/s | 762 kB 00:00 2024-04-16T05:20:51.421 INFO:teuthology.orchestra.run.smithi082.stdout:(37/174): cephfs-top-19.0.0-3101.ga87fcff6.el9. 183 kB/s | 18 kB 00:00 2024-04-16T05:20:51.554 INFO:teuthology.orchestra.run.smithi082.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-04-16T05:20:51.630 INFO:teuthology.orchestra.run.smithi082.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 920 kB 00:00 2024-04-16T05:20:51.663 INFO:teuthology.orchestra.run.smithi082.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-16T05:20:51.706 INFO:teuthology.orchestra.run.smithi082.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-04-16T05:20:51.730 INFO:teuthology.orchestra.run.smithi082.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2024-04-16T05:20:51.830 INFO:teuthology.orchestra.run.smithi032.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.ga8 269 kB/s | 249 kB 00:00 2024-04-16T05:20:51.831 INFO:teuthology.orchestra.run.smithi082.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-16T05:20:51.864 INFO:teuthology.orchestra.run.smithi082.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 515 kB/s | 17 kB 00:00 2024-04-16T05:20:51.898 INFO:teuthology.orchestra.run.smithi082.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 300 kB/s | 10 kB 00:00 2024-04-16T05:20:51.948 INFO:teuthology.orchestra.run.smithi082.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-16T05:20:52.024 INFO:teuthology.orchestra.run.smithi082.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 590 kB/s | 44 kB 00:00 2024-04-16T05:20:52.071 INFO:teuthology.orchestra.run.smithi032.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.ga87fcff6.e 204 kB/s | 49 kB 00:00 2024-04-16T05:20:52.099 INFO:teuthology.orchestra.run.smithi082.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-04-16T05:20:52.166 INFO:teuthology.orchestra.run.smithi082.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-04-16T05:20:52.213 INFO:teuthology.orchestra.run.smithi032.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.ga 116 kB/s | 16 kB 00:00 2024-04-16T05:20:52.251 INFO:teuthology.orchestra.run.smithi082.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 549 kB/s | 46 kB 00:00 2024-04-16T05:20:52.301 INFO:teuthology.orchestra.run.smithi082.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-16T05:20:52.360 INFO:teuthology.orchestra.run.smithi082.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-04-16T05:20:52.376 INFO:teuthology.orchestra.run.smithi134.stdout:(26/174): ceph-osd-19.0.0-3101.ga87fcff6.el9.x8 517 kB/s | 17 MB 00:33 2024-04-16T05:20:52.418 INFO:teuthology.orchestra.run.smithi082.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-04-16T05:20:52.452 INFO:teuthology.orchestra.run.smithi082.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-04-16T05:20:52.611 INFO:teuthology.orchestra.run.smithi082.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-16T05:20:52.678 INFO:teuthology.orchestra.run.smithi082.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2024-04-16T05:20:52.828 INFO:teuthology.orchestra.run.smithi082.stdout:(57/174): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 817 kB 00:00 2024-04-16T05:20:52.859 INFO:teuthology.orchestra.run.smithi134.stdout:(27/174): rbd-nbd-19.0.0-3101.ga87fcff6.el9.x86 353 kB/s | 170 kB 00:00 2024-04-16T05:20:52.861 INFO:teuthology.orchestra.run.smithi082.stdout:(58/174): postgresql-private-libs-13.14-1.el9.x 3.9 MB/s | 132 kB 00:00 2024-04-16T05:20:52.976 INFO:teuthology.orchestra.run.smithi134.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 293 kB/s | 34 kB 00:00 2024-04-16T05:20:53.020 INFO:teuthology.orchestra.run.smithi082.stdout:(59/174): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-04-16T05:20:53.071 INFO:teuthology.orchestra.run.smithi082.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-04-16T05:20:53.088 INFO:teuthology.orchestra.run.smithi032.stdout:(35/174): ceph-volume-19.0.0-3101.ga87fcff6.el9 312 kB/s | 273 kB 00:00 2024-04-16T05:20:53.104 INFO:teuthology.orchestra.run.smithi082.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-04-16T05:20:53.246 INFO:teuthology.orchestra.run.smithi082.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-04-16T05:20:53.279 INFO:teuthology.orchestra.run.smithi082.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 735 kB/s | 24 kB 00:00 2024-04-16T05:20:53.313 INFO:teuthology.orchestra.run.smithi082.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 542 kB/s | 18 kB 00:00 2024-04-16T05:20:53.338 INFO:teuthology.orchestra.run.smithi082.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 5.7 MB/s | 144 kB 00:00 2024-04-16T05:20:53.368 INFO:teuthology.orchestra.run.smithi134.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.ga87fcff 363 kB/s | 142 kB 00:00 2024-04-16T05:20:53.372 INFO:teuthology.orchestra.run.smithi082.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 944 kB/s | 32 kB 00:00 2024-04-16T05:20:53.497 INFO:teuthology.orchestra.run.smithi082.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-04-16T05:20:53.531 INFO:teuthology.orchestra.run.smithi082.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 6.4 MB/s | 217 kB 00:00 2024-04-16T05:20:53.564 INFO:teuthology.orchestra.run.smithi082.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 5.1 MB/s | 174 kB 00:00 2024-04-16T05:20:53.598 INFO:teuthology.orchestra.run.smithi082.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-04-16T05:20:53.640 INFO:teuthology.orchestra.run.smithi082.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 984 kB/s | 41 kB 00:00 2024-04-16T05:20:53.674 INFO:teuthology.orchestra.run.smithi082.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 7.0 MB/s | 238 kB 00:00 2024-04-16T05:20:53.707 INFO:teuthology.orchestra.run.smithi082.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.9 MB/s | 132 kB 00:00 2024-04-16T05:20:53.740 INFO:teuthology.orchestra.run.smithi082.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 6.2 MB/s | 211 kB 00:00 2024-04-16T05:20:53.766 INFO:teuthology.orchestra.run.smithi082.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.8 MB/s | 47 kB 00:00 2024-04-16T05:20:53.791 INFO:teuthology.orchestra.run.smithi082.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 1.7 MB/s | 43 kB 00:00 2024-04-16T05:20:54.342 INFO:teuthology.orchestra.run.smithi082.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-16T05:20:54.408 INFO:teuthology.orchestra.run.smithi082.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 581 kB/s | 39 kB 00:00 2024-04-16T05:20:54.476 INFO:teuthology.orchestra.run.smithi082.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 613 kB/s | 41 kB 00:00 2024-04-16T05:20:54.534 INFO:teuthology.orchestra.run.smithi082.stdout:(80/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-04-16T05:20:54.593 INFO:teuthology.orchestra.run.smithi082.stdout:(81/174): qatlib-service-24.02.0-1.el9.x86_64.r 597 kB/s | 35 kB 00:00 2024-04-16T05:20:54.643 INFO:teuthology.orchestra.run.smithi082.stdout:(82/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 925 kB/s | 46 kB 00:00 2024-04-16T05:20:54.685 INFO:teuthology.orchestra.run.smithi082.stdout:(83/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-04-16T05:20:54.718 INFO:teuthology.orchestra.run.smithi082.stdout:(84/174): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2024-04-16T05:20:54.752 INFO:teuthology.orchestra.run.smithi082.stdout:(85/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-16T05:20:54.819 INFO:teuthology.orchestra.run.smithi082.stdout:(86/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 662 kB/s | 44 kB 00:00 2024-04-16T05:20:54.830 INFO:teuthology.orchestra.run.smithi032.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 618 kB/s | 7.4 MB 00:12 2024-04-16T05:20:54.870 INFO:teuthology.orchestra.run.smithi082.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-16T05:20:54.920 INFO:teuthology.orchestra.run.smithi082.stdout:(88/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-16T05:20:54.956 INFO:teuthology.orchestra.run.smithi032.stdout:(37/174): cephfs-top-19.0.0-3101.ga87fcff6.el9. 146 kB/s | 18 kB 00:00 2024-04-16T05:20:54.978 INFO:teuthology.orchestra.run.smithi082.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-04-16T05:20:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:(38/174): cephadm-19.0.0-3101.ga87fcff6.el9.noa 403 kB/s | 762 kB 00:01 2024-04-16T05:20:55.012 INFO:teuthology.orchestra.run.smithi082.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-04-16T05:20:55.045 INFO:teuthology.orchestra.run.smithi082.stdout:(91/174): python3-pycparser-2.20-6.el9.noarch.r 3.6 MB/s | 124 kB 00:00 2024-04-16T05:20:55.079 INFO:teuthology.orchestra.run.smithi082.stdout:(92/174): python3-urllib3-1.26.5-5.el9.noarch.r 5.5 MB/s | 187 kB 00:00 2024-04-16T05:20:55.156 INFO:teuthology.orchestra.run.smithi032.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 5.1 MB/s | 920 kB 00:00 2024-04-16T05:20:55.181 INFO:teuthology.orchestra.run.smithi032.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 662 kB 00:00 2024-04-16T05:20:55.207 INFO:teuthology.orchestra.run.smithi032.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-04-16T05:20:55.229 INFO:teuthology.orchestra.run.smithi082.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-04-16T05:20:55.232 INFO:teuthology.orchestra.run.smithi032.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-16T05:20:55.257 INFO:teuthology.orchestra.run.smithi032.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-04-16T05:20:55.341 INFO:teuthology.orchestra.run.smithi032.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-16T05:20:55.366 INFO:teuthology.orchestra.run.smithi032.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 159 kB/s | 17 kB 00:00 2024-04-16T05:20:55.391 INFO:teuthology.orchestra.run.smithi032.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-04-16T05:20:55.416 INFO:teuthology.orchestra.run.smithi032.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-16T05:20:55.418 INFO:teuthology.orchestra.run.smithi134.stdout:(30/174): rbd-mirror-19.0.0-3101.ga87fcff6.el9. 512 kB/s | 3.1 MB 00:06 2024-04-16T05:20:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-04-16T05:20:55.484 INFO:teuthology.orchestra.run.smithi032.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-04-16T05:20:55.518 INFO:teuthology.orchestra.run.smithi032.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-04-16T05:20:55.543 INFO:teuthology.orchestra.run.smithi032.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 806 kB/s | 46 kB 00:00 2024-04-16T05:20:55.568 INFO:teuthology.orchestra.run.smithi032.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-16T05:20:55.585 INFO:teuthology.orchestra.run.smithi032.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-04-16T05:20:55.619 INFO:teuthology.orchestra.run.smithi032.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-04-16T05:20:55.644 INFO:teuthology.orchestra.run.smithi032.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 585 kB/s | 34 kB 00:00 2024-04-16T05:20:55.696 INFO:teuthology.orchestra.run.smithi082.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-04-16T05:20:55.752 INFO:teuthology.orchestra.run.smithi032.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-04-16T05:20:55.788 INFO:teuthology.orchestra.run.smithi082.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-16T05:20:55.794 INFO:teuthology.orchestra.run.smithi032.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-04-16T05:20:55.828 INFO:teuthology.orchestra.run.smithi032.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 817 kB 00:00 2024-04-16T05:20:55.862 INFO:teuthology.orchestra.run.smithi032.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 132 kB 00:00 2024-04-16T05:20:55.914 INFO:teuthology.orchestra.run.smithi082.stdout:(96/174): python3-requests-2.25.1-8.el9.noarch. 998 kB/s | 125 kB 00:00 2024-04-16T05:20:55.962 INFO:teuthology.orchestra.run.smithi032.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-16T05:20:56.029 INFO:teuthology.orchestra.run.smithi032.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-04-16T05:20:56.054 INFO:teuthology.orchestra.run.smithi032.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 850 kB/s | 78 kB 00:00 2024-04-16T05:20:56.305 INFO:teuthology.orchestra.run.smithi032.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-04-16T05:20:56.321 INFO:teuthology.orchestra.run.smithi032.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 91 kB/s | 24 kB 00:00 2024-04-16T05:20:56.355 INFO:teuthology.orchestra.run.smithi032.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 546 kB/s | 18 kB 00:00 2024-04-16T05:20:56.380 INFO:teuthology.orchestra.run.smithi032.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-04-16T05:20:56.397 INFO:teuthology.orchestra.run.smithi032.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 757 kB/s | 32 kB 00:00 2024-04-16T05:20:56.548 INFO:teuthology.orchestra.run.smithi032.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-04-16T05:20:56.573 INFO:teuthology.orchestra.run.smithi032.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-04-16T05:20:56.590 INFO:teuthology.orchestra.run.smithi032.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.1 MB/s | 174 kB 00:00 2024-04-16T05:20:56.605 INFO:teuthology.orchestra.run.smithi082.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 31 kB/s | 21 kB 00:00 2024-04-16T05:20:56.615 INFO:teuthology.orchestra.run.smithi032.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-04-16T05:20:56.640 INFO:teuthology.orchestra.run.smithi032.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-04-16T05:20:56.666 INFO:teuthology.orchestra.run.smithi032.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-16T05:20:56.691 INFO:teuthology.orchestra.run.smithi032.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-16T05:20:56.716 INFO:teuthology.orchestra.run.smithi032.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-16T05:20:56.741 INFO:teuthology.orchestra.run.smithi032.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-04-16T05:20:56.767 INFO:teuthology.orchestra.run.smithi032.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 852 kB/s | 43 kB 00:00 2024-04-16T05:20:57.000 INFO:teuthology.orchestra.run.smithi032.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 166 kB/s | 39 kB 00:00 2024-04-16T05:20:57.159 INFO:teuthology.orchestra.run.smithi032.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-16T05:20:57.184 INFO:teuthology.orchestra.run.smithi032.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 226 kB/s | 41 kB 00:00 2024-04-16T05:20:57.201 INFO:teuthology.orchestra.run.smithi032.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2024-04-16T05:20:57.226 INFO:teuthology.orchestra.run.smithi032.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 837 kB/s | 35 kB 00:00 2024-04-16T05:20:57.252 INFO:teuthology.orchestra.run.smithi032.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 918 kB/s | 46 kB 00:00 2024-04-16T05:20:57.255 INFO:teuthology.orchestra.run.smithi082.stdout:(98/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 39 kB/s | 25 kB 00:00 2024-04-16T05:20:57.286 INFO:teuthology.orchestra.run.smithi032.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-04-16T05:20:57.311 INFO:teuthology.orchestra.run.smithi032.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 963 kB/s | 56 kB 00:00 2024-04-16T05:20:57.336 INFO:teuthology.orchestra.run.smithi032.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-16T05:20:57.370 INFO:teuthology.orchestra.run.smithi032.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 752 kB/s | 44 kB 00:00 2024-04-16T05:20:57.403 INFO:teuthology.orchestra.run.smithi032.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-04-16T05:20:57.445 INFO:teuthology.orchestra.run.smithi032.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-16T05:20:57.554 INFO:teuthology.orchestra.run.smithi032.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 7.6 MB/s | 1.1 MB 00:00 2024-04-16T05:20:57.571 INFO:teuthology.orchestra.run.smithi032.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 821 kB/s | 103 kB 00:00 2024-04-16T05:20:57.604 INFO:teuthology.orchestra.run.smithi032.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 3.6 MB/s | 124 kB 00:00 2024-04-16T05:20:57.630 INFO:teuthology.orchestra.run.smithi032.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-04-16T05:20:58.663 INFO:teuthology.orchestra.run.smithi032.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 177 kB/s | 187 kB 00:01 2024-04-16T05:20:59.472 INFO:teuthology.orchestra.run.smithi032.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 436 kB/s | 803 kB 00:01 2024-04-16T05:20:59.497 INFO:teuthology.orchestra.run.smithi032.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 234 kB/s | 195 kB 00:00 2024-04-16T05:20:59.764 INFO:teuthology.orchestra.run.smithi032.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 81 kB/s | 21 kB 00:00 2024-04-16T05:20:59.864 INFO:teuthology.orchestra.run.smithi032.stdout:(98/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-16T05:21:00.248 INFO:teuthology.orchestra.run.smithi032.stdout:(99/174): python3-requests-2.25.1-8.el9.noarch. 161 kB/s | 125 kB 00:00 2024-04-16T05:21:00.559 INFO:teuthology.orchestra.run.smithi134.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.ga87fc 504 kB/s | 3.5 MB 00:07 2024-04-16T05:21:00.690 INFO:teuthology.orchestra.run.smithi032.stdout:(100/174): openssl-devel-3.0.7-27.el9.x86_64.rp 5.0 MB/s | 4.1 MB 00:00 2024-04-16T05:21:00.798 INFO:teuthology.orchestra.run.smithi032.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.2 MB/s | 248 kB 00:00 2024-04-16T05:21:00.832 INFO:teuthology.orchestra.run.smithi032.stdout:(102/174): python3-devel-3.9.17-2.el9.x86_64.rp 420 kB/s | 245 kB 00:00 2024-04-16T05:21:00.974 INFO:teuthology.orchestra.run.smithi032.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 534 kB/s | 94 kB 00:00 2024-04-16T05:21:00.999 INFO:teuthology.orchestra.run.smithi032.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-04-16T05:21:01.124 INFO:teuthology.orchestra.run.smithi032.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-16T05:21:01.126 INFO:teuthology.orchestra.run.smithi134.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.ga8 439 kB/s | 249 kB 00:00 2024-04-16T05:21:01.318 INFO:teuthology.orchestra.run.smithi134.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.ga87fcff6.e 257 kB/s | 49 kB 00:00 2024-04-16T05:21:01.435 INFO:teuthology.orchestra.run.smithi134.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.ga 141 kB/s | 16 kB 00:00 2024-04-16T05:21:01.500 INFO:teuthology.orchestra.run.smithi032.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-16T05:21:01.592 INFO:teuthology.orchestra.run.smithi032.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-16T05:21:01.617 INFO:teuthology.orchestra.run.smithi032.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 627 kB/s | 309 kB 00:00 2024-04-16T05:21:01.709 INFO:teuthology.orchestra.run.smithi032.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-16T05:21:01.801 INFO:teuthology.orchestra.run.smithi032.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-04-16T05:21:01.934 INFO:teuthology.orchestra.run.smithi032.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-16T05:21:01.960 INFO:teuthology.orchestra.run.smithi032.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-04-16T05:21:01.993 INFO:teuthology.orchestra.run.smithi134.stdout:(35/174): ceph-volume-19.0.0-3101.ga87fcff6.el9 489 kB/s | 273 kB 00:00 2024-04-16T05:21:02.043 INFO:teuthology.orchestra.run.smithi032.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-16T05:21:02.169 INFO:teuthology.orchestra.run.smithi032.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-16T05:21:02.194 INFO:teuthology.orchestra.run.smithi032.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-16T05:21:02.261 INFO:teuthology.orchestra.run.smithi032.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-04-16T05:21:02.286 INFO:teuthology.orchestra.run.smithi032.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-04-16T05:21:02.353 INFO:teuthology.orchestra.run.smithi032.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-16T05:21:02.403 INFO:teuthology.orchestra.run.smithi032.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-16T05:21:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-16T05:21:02.537 INFO:teuthology.orchestra.run.smithi032.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-16T05:21:02.596 INFO:teuthology.orchestra.run.smithi032.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-16T05:21:02.629 INFO:teuthology.orchestra.run.smithi032.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.1 MB/s | 203 kB 00:00 2024-04-16T05:21:02.730 INFO:teuthology.orchestra.run.smithi032.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-16T05:21:02.797 INFO:teuthology.orchestra.run.smithi032.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-16T05:21:02.822 INFO:teuthology.orchestra.run.smithi032.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-16T05:21:02.906 INFO:teuthology.orchestra.run.smithi032.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-16T05:21:02.931 INFO:teuthology.orchestra.run.smithi032.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-16T05:21:03.006 INFO:teuthology.orchestra.run.smithi032.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-16T05:21:03.032 INFO:teuthology.orchestra.run.smithi032.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-16T05:21:03.132 INFO:teuthology.orchestra.run.smithi032.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-16T05:21:03.157 INFO:teuthology.orchestra.run.smithi032.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-04-16T05:21:03.249 INFO:teuthology.orchestra.run.smithi032.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 675 kB/s | 79 kB 00:00 2024-04-16T05:21:03.274 INFO:teuthology.orchestra.run.smithi032.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 494 kB/s | 58 kB 00:00 2024-04-16T05:21:03.375 INFO:teuthology.orchestra.run.smithi032.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-16T05:21:03.400 INFO:teuthology.orchestra.run.smithi032.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-16T05:21:03.483 INFO:teuthology.orchestra.run.smithi032.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 830 kB/s | 90 kB 00:00 2024-04-16T05:21:03.485 INFO:teuthology.orchestra.run.smithi134.stdout:(36/174): cephadm-19.0.0-3101.ga87fcff6.el9.noa 511 kB/s | 762 kB 00:01 2024-04-16T05:21:03.517 INFO:teuthology.orchestra.run.smithi032.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-04-16T05:21:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 674 kB/s | 4.1 MB 00:06 2024-04-16T05:21:03.576 INFO:teuthology.orchestra.run.smithi032.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-16T05:21:03.602 INFO:teuthology.orchestra.run.smithi134.stdout:(37/174): cephfs-top-19.0.0-3101.ga87fcff6.el9. 156 kB/s | 18 kB 00:00 2024-04-16T05:21:03.609 INFO:teuthology.orchestra.run.smithi032.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-16T05:21:03.668 INFO:teuthology.orchestra.run.smithi032.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-04-16T05:21:03.726 INFO:teuthology.orchestra.run.smithi032.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-04-16T05:21:03.768 INFO:teuthology.orchestra.run.smithi032.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-04-16T05:21:03.794 INFO:teuthology.orchestra.run.smithi134.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-04-16T05:21:03.827 INFO:teuthology.orchestra.run.smithi032.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-16T05:21:03.861 INFO:teuthology.orchestra.run.smithi032.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 975 kB/s | 90 kB 00:00 2024-04-16T05:21:03.861 INFO:teuthology.orchestra.run.smithi134.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 920 kB 00:00 2024-04-16T05:21:03.895 INFO:teuthology.orchestra.run.smithi134.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-16T05:21:03.927 INFO:teuthology.orchestra.run.smithi032.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-16T05:21:03.953 INFO:teuthology.orchestra.run.smithi134.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-04-16T05:21:03.978 INFO:teuthology.orchestra.run.smithi032.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-16T05:21:03.987 INFO:teuthology.orchestra.run.smithi134.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 944 kB/s | 32 kB 00:00 2024-04-16T05:21:04.020 INFO:teuthology.orchestra.run.smithi032.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-16T05:21:04.079 INFO:teuthology.orchestra.run.smithi134.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-16T05:21:04.105 INFO:teuthology.orchestra.run.smithi134.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 691 kB/s | 17 kB 00:00 2024-04-16T05:21:04.138 INFO:teuthology.orchestra.run.smithi134.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 303 kB/s | 10 kB 00:00 2024-04-16T05:21:04.162 INFO:teuthology.orchestra.run.smithi032.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-16T05:21:04.172 INFO:teuthology.orchestra.run.smithi134.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-04-16T05:21:04.205 INFO:teuthology.orchestra.run.smithi134.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-16T05:21:04.221 INFO:teuthology.orchestra.run.smithi082.stdout:(100/174): ceph-mgr-diskprediction-local-19.0.0 413 kB/s | 7.4 MB 00:18 2024-04-16T05:21:04.256 INFO:teuthology.orchestra.run.smithi134.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-16T05:21:04.298 INFO:teuthology.orchestra.run.smithi134.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-04-16T05:21:04.361 INFO:teuthology.orchestra.run.smithi134.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 739 kB/s | 46 kB 00:00 2024-04-16T05:21:04.399 INFO:teuthology.orchestra.run.smithi134.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 7.4 MB/s | 282 kB 00:00 2024-04-16T05:21:04.442 INFO:teuthology.orchestra.run.smithi134.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.2 MB/s | 187 kB 00:00 2024-04-16T05:21:04.485 INFO:teuthology.orchestra.run.smithi134.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-04-16T05:21:04.497 INFO:teuthology.orchestra.run.smithi082.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 254 kB/s | 245 kB 00:00 2024-04-16T05:21:04.509 INFO:teuthology.orchestra.run.smithi134.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-04-16T05:21:04.660 INFO:teuthology.orchestra.run.smithi134.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-16T05:21:04.727 INFO:teuthology.orchestra.run.smithi134.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2024-04-16T05:21:04.794 INFO:teuthology.orchestra.run.smithi134.stdout:(57/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-04-16T05:21:04.844 INFO:teuthology.orchestra.run.smithi134.stdout:(58/174): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 132 kB 00:00 2024-04-16T05:21:04.986 INFO:teuthology.orchestra.run.smithi134.stdout:(59/174): postgresql-server-13.14-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-04-16T05:21:05.037 INFO:teuthology.orchestra.run.smithi134.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-04-16T05:21:05.071 INFO:teuthology.orchestra.run.smithi134.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-04-16T05:21:05.188 INFO:teuthology.orchestra.run.smithi082.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 256 kB/s | 248 kB 00:00 2024-04-16T05:21:05.212 INFO:teuthology.orchestra.run.smithi134.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-04-16T05:21:05.246 INFO:teuthology.orchestra.run.smithi134.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 734 kB/s | 24 kB 00:00 2024-04-16T05:21:05.280 INFO:teuthology.orchestra.run.smithi134.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 541 kB/s | 18 kB 00:00 2024-04-16T05:21:05.280 INFO:teuthology.orchestra.run.smithi082.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 120 kB/s | 94 kB 00:00 2024-04-16T05:21:05.305 INFO:teuthology.orchestra.run.smithi082.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-04-16T05:21:05.363 INFO:teuthology.orchestra.run.smithi134.stdout:(65/174): ceph-mgr-diskprediction-local-19.0.0- 763 kB/s | 7.4 MB 00:09 2024-04-16T05:21:05.389 INFO:teuthology.orchestra.run.smithi134.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-04-16T05:21:05.406 INFO:teuthology.orchestra.run.smithi082.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-04-16T05:21:05.414 INFO:teuthology.orchestra.run.smithi134.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-04-16T05:21:05.439 INFO:teuthology.orchestra.run.smithi082.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-04-16T05:21:05.464 INFO:teuthology.orchestra.run.smithi082.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-04-16T05:21:05.506 INFO:teuthology.orchestra.run.smithi082.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-04-16T05:21:05.522 INFO:teuthology.orchestra.run.smithi134.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-04-16T05:21:05.548 INFO:teuthology.orchestra.run.smithi134.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-04-16T05:21:05.573 INFO:teuthology.orchestra.run.smithi134.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-16T05:21:05.598 INFO:teuthology.orchestra.run.smithi134.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-16T05:21:05.599 INFO:teuthology.orchestra.run.smithi082.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-04-16T05:21:05.615 INFO:teuthology.orchestra.run.smithi134.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 983 kB/s | 41 kB 00:00 2024-04-16T05:21:05.615 INFO:teuthology.orchestra.run.smithi082.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 238 kB/s | 26 kB 00:00 2024-04-16T05:21:05.632 INFO:teuthology.orchestra.run.smithi082.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-04-16T05:21:05.649 INFO:teuthology.orchestra.run.smithi134.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-16T05:21:05.666 INFO:teuthology.orchestra.run.smithi082.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-16T05:21:05.674 INFO:teuthology.orchestra.run.smithi134.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-04-16T05:21:05.691 INFO:teuthology.orchestra.run.smithi082.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-16T05:21:05.703 INFO:teuthology.orchestra.run.smithi134.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-16T05:21:05.725 INFO:teuthology.orchestra.run.smithi134.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 948 kB/s | 47 kB 00:00 2024-04-16T05:21:05.733 INFO:teuthology.orchestra.run.smithi082.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-04-16T05:21:05.750 INFO:teuthology.orchestra.run.smithi134.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-04-16T05:21:05.775 INFO:teuthology.orchestra.run.smithi082.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-04-16T05:21:05.800 INFO:teuthology.orchestra.run.smithi082.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 649 kB/s | 43 kB 00:00 2024-04-16T05:21:05.825 INFO:teuthology.orchestra.run.smithi082.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-04-16T05:21:05.851 INFO:teuthology.orchestra.run.smithi082.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-16T05:21:05.876 INFO:teuthology.orchestra.run.smithi082.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-16T05:21:05.901 INFO:teuthology.orchestra.run.smithi082.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-16T05:21:05.952 INFO:teuthology.orchestra.run.smithi082.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-16T05:21:05.977 INFO:teuthology.orchestra.run.smithi082.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-04-16T05:21:06.002 INFO:teuthology.orchestra.run.smithi082.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-04-16T05:21:06.036 INFO:teuthology.orchestra.run.smithi082.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-04-16T05:21:06.078 INFO:teuthology.orchestra.run.smithi082.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-04-16T05:21:06.084 INFO:teuthology.orchestra.run.smithi134.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-04-16T05:21:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-16T05:21:06.111 INFO:teuthology.orchestra.run.smithi134.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 108 kB/s | 39 kB 00:00 2024-04-16T05:21:06.128 INFO:teuthology.orchestra.run.smithi082.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-16T05:21:06.134 INFO:teuthology.orchestra.run.smithi134.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 817 kB/s | 41 kB 00:00 2024-04-16T05:21:06.154 INFO:teuthology.orchestra.run.smithi082.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-16T05:21:06.160 INFO:teuthology.orchestra.run.smithi134.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-04-16T05:21:06.179 INFO:teuthology.orchestra.run.smithi082.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-16T05:21:06.185 INFO:teuthology.orchestra.run.smithi134.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 700 kB/s | 35 kB 00:00 2024-04-16T05:21:06.204 INFO:teuthology.orchestra.run.smithi082.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-16T05:21:06.210 INFO:teuthology.orchestra.run.smithi134.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 921 kB/s | 46 kB 00:00 2024-04-16T05:21:06.244 INFO:teuthology.orchestra.run.smithi134.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-04-16T05:21:06.254 INFO:teuthology.orchestra.run.smithi082.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-16T05:21:06.280 INFO:teuthology.orchestra.run.smithi082.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-04-16T05:21:06.286 INFO:teuthology.orchestra.run.smithi134.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 756 kB/s | 56 kB 00:00 2024-04-16T05:21:06.305 INFO:teuthology.orchestra.run.smithi082.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-16T05:21:06.314 INFO:teuthology.orchestra.run.smithi134.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 935 kB/s | 63 kB 00:00 2024-04-16T05:21:06.330 INFO:teuthology.orchestra.run.smithi082.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-16T05:21:06.345 INFO:teuthology.orchestra.run.smithi134.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-04-16T05:21:06.355 INFO:teuthology.orchestra.run.smithi082.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-16T05:21:06.370 INFO:teuthology.orchestra.run.smithi134.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-04-16T05:21:06.380 INFO:teuthology.orchestra.run.smithi082.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-04-16T05:21:06.395 INFO:teuthology.orchestra.run.smithi134.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-16T05:21:06.406 INFO:teuthology.orchestra.run.smithi082.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-16T05:21:06.429 INFO:teuthology.orchestra.run.smithi134.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-04-16T05:21:06.431 INFO:teuthology.orchestra.run.smithi082.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-16T05:21:06.454 INFO:teuthology.orchestra.run.smithi134.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-16T05:21:06.459 INFO:teuthology.orchestra.run.smithi082.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 586 kB/s | 31 kB 00:00 2024-04-16T05:21:06.480 INFO:teuthology.orchestra.run.smithi134.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-04-16T05:21:06.484 INFO:teuthology.orchestra.run.smithi082.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.5 MB/s | 188 kB 00:00 2024-04-16T05:21:06.505 INFO:teuthology.orchestra.run.smithi134.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-16T05:21:06.509 INFO:teuthology.orchestra.run.smithi082.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-16T05:21:06.551 INFO:teuthology.orchestra.run.smithi082.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 536 kB/s | 36 kB 00:00 2024-04-16T05:21:06.576 INFO:teuthology.orchestra.run.smithi082.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 762 kB/s | 51 kB 00:00 2024-04-16T05:21:06.602 INFO:teuthology.orchestra.run.smithi082.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-16T05:21:06.627 INFO:teuthology.orchestra.run.smithi082.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-16T05:21:06.652 INFO:teuthology.orchestra.run.smithi082.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-04-16T05:21:06.669 INFO:teuthology.orchestra.run.smithi082.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-04-16T05:21:06.694 INFO:teuthology.orchestra.run.smithi082.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-04-16T05:21:06.737 INFO:teuthology.orchestra.run.smithi082.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-16T05:21:06.763 INFO:teuthology.orchestra.run.smithi134.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 661 kB/s | 187 kB 00:00 2024-04-16T05:21:06.806 INFO:teuthology.orchestra.run.smithi134.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-16T05:21:06.864 INFO:teuthology.orchestra.run.smithi134.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-16T05:21:06.889 INFO:teuthology.orchestra.run.smithi134.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-16T05:21:06.948 INFO:teuthology.orchestra.run.smithi134.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-04-16T05:21:06.973 INFO:teuthology.orchestra.run.smithi134.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-04-16T05:21:07.090 INFO:teuthology.orchestra.run.smithi134.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.1 MB/s | 245 kB 00:00 2024-04-16T05:21:07.207 INFO:teuthology.orchestra.run.smithi134.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.1 MB/s | 248 kB 00:00 2024-04-16T05:21:07.258 INFO:teuthology.orchestra.run.smithi134.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-16T05:21:07.383 INFO:teuthology.orchestra.run.smithi134.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-04-16T05:21:07.522 INFO:teuthology.orchestra.run.smithi134.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.1 MB/s | 863 kB 00:00 2024-04-16T05:21:08.030 INFO:teuthology.orchestra.run.smithi134.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.8 MB/s | 4.1 MB 00:01 2024-04-16T05:21:08.055 INFO:teuthology.orchestra.run.smithi134.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-16T05:21:08.158 INFO:teuthology.orchestra.run.smithi134.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-04-16T05:21:08.640 INFO:teuthology.orchestra.run.smithi134.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-16T05:21:08.665 INFO:teuthology.orchestra.run.smithi134.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 487 kB/s | 309 kB 00:00 2024-04-16T05:21:08.757 INFO:teuthology.orchestra.run.smithi134.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-04-16T05:21:08.782 INFO:teuthology.orchestra.run.smithi134.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-04-16T05:21:08.849 INFO:teuthology.orchestra.run.smithi134.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-04-16T05:21:08.957 INFO:teuthology.orchestra.run.smithi134.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-04-16T05:21:08.982 INFO:teuthology.orchestra.run.smithi134.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 753 kB/s | 151 kB 00:00 2024-04-16T05:21:09.066 INFO:teuthology.orchestra.run.smithi134.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-16T05:21:09.078 INFO:teuthology.orchestra.run.smithi032.stdout:(150/174): librados2-19.0.0-3101.ga87fcff6.el9. 687 kB/s | 3.4 MB 00:05 2024-04-16T05:21:09.091 INFO:teuthology.orchestra.run.smithi134.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-16T05:21:09.158 INFO:teuthology.orchestra.run.smithi134.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-04-16T05:21:09.162 INFO:teuthology.orchestra.run.smithi032.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 28 MB/s | 2.3 MB 00:00 2024-04-16T05:21:09.184 INFO:teuthology.orchestra.run.smithi134.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-16T05:21:09.229 INFO:teuthology.orchestra.run.smithi032.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-04-16T05:21:09.259 INFO:teuthology.orchestra.run.smithi134.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-16T05:21:09.262 INFO:teuthology.orchestra.run.smithi032.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-16T05:21:09.304 INFO:teuthology.orchestra.run.smithi032.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-16T05:21:09.346 INFO:teuthology.orchestra.run.smithi032.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-16T05:21:09.375 INFO:teuthology.orchestra.run.smithi134.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-16T05:21:09.467 INFO:teuthology.orchestra.run.smithi134.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-04-16T05:21:09.493 INFO:teuthology.orchestra.run.smithi134.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2024-04-16T05:21:09.568 INFO:teuthology.orchestra.run.smithi134.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-16T05:21:09.668 INFO:teuthology.orchestra.run.smithi134.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-16T05:21:09.752 INFO:teuthology.orchestra.run.smithi134.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-16T05:21:09.820 INFO:teuthology.orchestra.run.smithi082.stdout:(150/174): librbd1-19.0.0-3101.ga87fcff6.el9.x8 1.0 MB/s | 3.1 MB 00:03 2024-04-16T05:21:09.844 INFO:teuthology.orchestra.run.smithi134.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-16T05:21:09.928 INFO:teuthology.orchestra.run.smithi134.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-04-16T05:21:09.935 INFO:teuthology.orchestra.run.smithi082.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 20 MB/s | 2.3 MB 00:00 2024-04-16T05:21:10.012 INFO:teuthology.orchestra.run.smithi082.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-16T05:21:10.013 INFO:teuthology.orchestra.run.smithi032.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 107 kB/s | 71 kB 00:00 2024-04-16T05:21:10.020 INFO:teuthology.orchestra.run.smithi134.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-16T05:21:10.045 INFO:teuthology.orchestra.run.smithi082.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-16T05:21:10.087 INFO:teuthology.orchestra.run.smithi082.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-16T05:21:10.111 INFO:teuthology.orchestra.run.smithi134.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-04-16T05:21:10.129 INFO:teuthology.orchestra.run.smithi082.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-16T05:21:10.228 INFO:teuthology.orchestra.run.smithi134.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-04-16T05:21:10.229 INFO:teuthology.orchestra.run.smithi082.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 714 kB/s | 71 kB 00:00 2024-04-16T05:21:10.280 INFO:teuthology.orchestra.run.smithi082.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 728 kB/s | 36 kB 00:00 2024-04-16T05:21:10.329 INFO:teuthology.orchestra.run.smithi134.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-16T05:21:10.389 INFO:teuthology.orchestra.run.smithi082.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-04-16T05:21:10.421 INFO:teuthology.orchestra.run.smithi134.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 861 kB/s | 79 kB 00:00 2024-04-16T05:21:10.472 INFO:teuthology.orchestra.run.smithi082.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-04-16T05:21:10.512 INFO:teuthology.orchestra.run.smithi134.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 630 kB/s | 58 kB 00:00 2024-04-16T05:21:10.522 INFO:teuthology.orchestra.run.smithi082.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 461 kB/s | 23 kB 00:00 2024-04-16T05:21:10.581 INFO:teuthology.orchestra.run.smithi082.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-16T05:21:10.613 INFO:teuthology.orchestra.run.smithi134.stdout:(134/174): python3-grpcio-1.46.7-10.el9.x86_64. 1.8 MB/s | 2.0 MB 00:01 2024-04-16T05:21:10.638 INFO:teuthology.orchestra.run.smithi134.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-16T05:21:10.648 INFO:teuthology.orchestra.run.smithi082.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-04-16T05:21:10.680 INFO:teuthology.orchestra.run.smithi032.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 55 kB/s | 36 kB 00:00 2024-04-16T05:21:10.705 INFO:teuthology.orchestra.run.smithi134.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-16T05:21:10.707 INFO:teuthology.orchestra.run.smithi082.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-04-16T05:21:10.731 INFO:teuthology.orchestra.run.smithi032.stdout:(158/174): librbd1-19.0.0-3101.ga87fcff6.el9.x8 490 kB/s | 3.1 MB 00:06 2024-04-16T05:21:10.739 INFO:teuthology.orchestra.run.smithi134.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-04-16T05:21:10.757 INFO:teuthology.orchestra.run.smithi082.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-04-16T05:21:10.797 INFO:teuthology.orchestra.run.smithi134.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-16T05:21:10.807 INFO:teuthology.orchestra.run.smithi082.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 574 kB/s | 29 kB 00:00 2024-04-16T05:21:10.831 INFO:teuthology.orchestra.run.smithi134.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-04-16T05:21:10.898 INFO:teuthology.orchestra.run.smithi134.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-16T05:21:10.923 INFO:teuthology.orchestra.run.smithi134.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-04-16T05:21:10.932 INFO:teuthology.orchestra.run.smithi082.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-04-16T05:21:10.990 INFO:teuthology.orchestra.run.smithi134.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-04-16T05:21:11.015 INFO:teuthology.orchestra.run.smithi134.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-16T05:21:11.107 INFO:teuthology.orchestra.run.smithi134.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 979 kB/s | 90 kB 00:00 2024-04-16T05:21:11.157 INFO:teuthology.orchestra.run.smithi134.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-16T05:21:11.216 INFO:teuthology.orchestra.run.smithi134.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-16T05:21:11.249 INFO:teuthology.orchestra.run.smithi134.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-16T05:21:11.316 INFO:teuthology.orchestra.run.smithi134.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-16T05:21:11.324 INFO:teuthology.orchestra.run.smithi082.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.5 MB/s | 2.2 MB 00:00 2024-04-16T05:21:11.542 INFO:teuthology.orchestra.run.smithi082.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-04-16T05:21:11.547 INFO:teuthology.orchestra.run.smithi032.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 222 kB/s | 192 kB 00:00 2024-04-16T05:21:11.600 INFO:teuthology.orchestra.run.smithi082.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.8 MB/s | 465 kB 00:00 2024-04-16T05:21:11.635 INFO:teuthology.orchestra.run.smithi032.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 283 kB/s | 256 kB 00:00 2024-04-16T05:21:11.650 INFO:teuthology.orchestra.run.smithi082.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 623 kB/s | 31 kB 00:00 2024-04-16T05:21:11.783 INFO:teuthology.orchestra.run.smithi134.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-04-16T05:21:11.793 INFO:teuthology.orchestra.run.smithi082.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-04-16T05:21:12.211 INFO:teuthology.orchestra.run.smithi032.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 35 kB/s | 23 kB 00:00 2024-04-16T05:21:12.360 INFO:teuthology.orchestra.run.smithi082.stdout:(172/174): librados2-19.0.0-3101.ga87fcff6.el9. 614 kB/s | 3.4 MB 00:05 2024-04-16T05:21:12.461 INFO:teuthology.orchestra.run.smithi032.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 131 kB/s | 108 kB 00:00 2024-04-16T05:21:12.518 INFO:teuthology.orchestra.run.smithi082.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 102 kB/s | 74 kB 00:00 2024-04-16T05:21:13.036 INFO:teuthology.orchestra.run.smithi032.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 188 kB/s | 155 kB 00:00 2024-04-16T05:21:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 175 kB/s | 136 kB 00:00 2024-04-16T05:21:13.712 INFO:teuthology.orchestra.run.smithi032.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 95 kB/s | 64 kB 00:00 2024-04-16T05:21:13.871 INFO:teuthology.orchestra.run.smithi032.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 49 kB/s | 29 kB 00:00 2024-04-16T05:21:15.308 INFO:teuthology.orchestra.run.smithi134.stdout:(150/174): librbd1-19.0.0-3101.ga87fcff6.el9.x8 913 kB/s | 3.1 MB 00:03 2024-04-16T05:21:15.475 INFO:teuthology.orchestra.run.smithi134.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-04-16T05:21:15.550 INFO:teuthology.orchestra.run.smithi134.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-16T05:21:15.585 INFO:teuthology.orchestra.run.smithi134.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-16T05:21:15.618 INFO:teuthology.orchestra.run.smithi134.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 7.3 MB/s | 248 kB 00:00 2024-04-16T05:21:15.659 INFO:teuthology.orchestra.run.smithi134.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-16T05:21:15.844 INFO:teuthology.orchestra.run.smithi134.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 388 kB/s | 71 kB 00:00 2024-04-16T05:21:15.902 INFO:teuthology.orchestra.run.smithi134.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 626 kB/s | 36 kB 00:00 2024-04-16T05:21:15.971 INFO:teuthology.orchestra.run.smithi032.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 549 kB/s | 1.2 MB 00:02 2024-04-16T05:21:15.977 INFO:teuthology.orchestra.run.smithi134.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-04-16T05:21:16.044 INFO:teuthology.orchestra.run.smithi134.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 256 kB 00:00 2024-04-16T05:21:16.095 INFO:teuthology.orchestra.run.smithi134.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 461 kB/s | 23 kB 00:00 2024-04-16T05:21:16.153 INFO:teuthology.orchestra.run.smithi134.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-16T05:21:16.212 INFO:teuthology.orchestra.run.smithi134.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-04-16T05:21:16.287 INFO:teuthology.orchestra.run.smithi134.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-04-16T05:21:16.354 INFO:teuthology.orchestra.run.smithi134.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 961 kB/s | 64 kB 00:00 2024-04-16T05:21:16.404 INFO:teuthology.orchestra.run.smithi134.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-04-16T05:21:16.521 INFO:teuthology.orchestra.run.smithi134.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-04-16T05:21:16.655 INFO:teuthology.orchestra.run.smithi134.stdout:(167/174): librados2-19.0.0-3101.ga87fcff6.el9. 651 kB/s | 3.4 MB 00:05 2024-04-16T05:21:16.713 INFO:teuthology.orchestra.run.smithi134.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-04-16T05:21:16.863 INFO:teuthology.orchestra.run.smithi134.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-04-16T05:21:16.922 INFO:teuthology.orchestra.run.smithi134.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 535 kB/s | 31 kB 00:00 2024-04-16T05:21:16.981 INFO:teuthology.orchestra.run.smithi134.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 1.7 MB/s | 465 kB 00:00 2024-04-16T05:21:17.039 INFO:teuthology.orchestra.run.smithi134.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.2 MB/s | 1.1 MB 00:00 2024-04-16T05:21:17.089 INFO:teuthology.orchestra.run.smithi134.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 686 kB/s | 74 kB 00:00 2024-04-16T05:21:17.379 INFO:teuthology.orchestra.run.smithi032.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 629 kB/s | 2.2 MB 00:03 2024-04-16T05:21:18.713 INFO:teuthology.orchestra.run.smithi032.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 349 kB/s | 465 kB 00:01 2024-04-16T05:21:19.405 INFO:teuthology.orchestra.run.smithi032.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 45 kB/s | 31 kB 00:00 2024-04-16T05:21:19.772 INFO:teuthology.orchestra.run.smithi032.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 615 kB/s | 2.3 MB 00:03 2024-04-16T05:21:20.297 INFO:teuthology.orchestra.run.smithi032.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 142 kB/s | 74 kB 00:00 2024-04-16T05:21:21.597 INFO:teuthology.orchestra.run.smithi032.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 501 kB/s | 1.1 MB 00:02 2024-04-16T05:21:30.388 INFO:teuthology.orchestra.run.smithi032.stdout:(174/174): ceph-test-19.0.0-3101.ga87fcff6.el9. 692 kB/s | 49 MB 01:12 2024-04-16T05:21:30.393 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:21:30.393 INFO:teuthology.orchestra.run.smithi032.stdout:Total 2.6 MB/s | 229 MB 01:26 2024-04-16T05:21:30.754 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-04-16T05:21:31.048 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-04-16T05:21:31.048 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-04-16T05:21:33.153 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-04-16T05:21:33.153 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-04-16T05:21:34.804 INFO:teuthology.orchestra.run.smithi134.stdout:(174/174): ceph-test-19.0.0-3101.ga87fcff6.el9. 773 kB/s | 49 MB 01:05 2024-04-16T05:21:34.809 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:21:34.810 INFO:teuthology.orchestra.run.smithi134.stdout:Total 2.8 MB/s | 229 MB 01:23 2024-04-16T05:21:35.153 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-04-16T05:21:35.420 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-04-16T05:21:35.420 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-04-16T05:21:36.680 INFO:teuthology.orchestra.run.smithi082.stdout:(174/174): ceph-test-19.0.0-3101.ga87fcff6.el9. 669 kB/s | 49 MB 01:15 2024-04-16T05:21:36.684 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:21:36.684 INFO:teuthology.orchestra.run.smithi082.stdout:Total 2.4 MB/s | 229 MB 01:34 2024-04-16T05:21:37.030 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-04-16T05:21:37.219 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-04-16T05:21:37.221 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-04-16T05:21:37.298 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-04-16T05:21:37.298 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-04-16T05:21:39.105 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-04-16T05:21:39.105 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-04-16T05:21:40.304 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-04-16T05:21:40.392 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T05:21:40.419 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T05:21:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:21:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:21:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T05:21:40.559 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T05:21:40.594 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T05:21:40.619 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T05:21:40.665 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T05:21:41.017 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T05:21:41.070 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:41.194 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:41.462 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:21:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T05:21:41.704 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T05:21:41.758 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T05:21:41.785 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:21:41.985 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:21:42.024 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T05:21:42.047 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:21:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:21:42.258 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:21:42.287 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:21:42.318 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T05:21:42.488 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T05:21:42.612 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:21:42.689 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:21:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:43.055 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:43.166 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T05:21:43.260 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T05:21:43.395 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T05:21:43.430 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T05:21:43.474 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T05:21:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T05:21:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T05:21:43.641 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T05:21:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T05:21:43.748 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T05:21:43.792 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T05:21:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T05:21:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T05:21:43.923 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T05:21:43.957 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 39/198 2024-04-16T05:21:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 40/198 2024-04-16T05:21:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 41/198 2024-04-16T05:21:44.100 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T05:21:44.118 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-04-16T05:21:44.160 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T05:21:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T05:21:44.283 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T05:21:44.316 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T05:21:44.347 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:21:44.357 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:21:44.383 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T05:21:44.405 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T05:21:44.440 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T05:21:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T05:21:44.492 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T05:21:44.518 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T05:21:44.576 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T05:21:44.904 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T05:21:44.969 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T05:21:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T05:21:45.089 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:45.120 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:45.169 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T05:21:45.210 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:45.220 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:21:45.230 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:21:45.240 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:21:45.265 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T05:21:45.278 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T05:21:45.319 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T05:21:45.356 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 53/198 2024-04-16T05:21:45.387 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T05:21:45.455 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T05:21:45.505 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T05:21:45.533 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:45.636 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:45.659 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:21:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T05:21:45.722 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:21:45.762 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T05:21:45.785 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:21:45.800 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:21:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T05:21:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:21:45.963 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T05:21:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:21:45.997 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T05:21:46.007 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T05:21:46.030 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-04-16T05:21:46.093 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T05:21:46.133 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T05:21:46.158 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T05:21:46.167 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T05:21:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:21:46.202 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T05:21:46.206 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:21:46.214 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T05:21:46.243 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:21:46.255 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T05:21:46.257 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T05:21:46.309 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T05:21:46.317 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:46.332 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T05:21:46.342 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T05:21:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T05:21:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:46.377 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T05:21:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T05:21:46.419 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T05:21:46.452 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T05:21:46.488 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T05:21:46.532 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T05:21:46.574 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T05:21:46.582 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:46.714 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T05:21:46.741 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T05:21:46.754 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T05:21:46.762 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T05:21:46.792 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T05:21:46.795 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:46.834 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T05:21:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T05:21:46.915 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:21:46.924 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T05:21:46.931 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T05:21:46.937 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:46.951 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T05:21:46.966 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T05:21:46.986 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T05:21:46.998 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T05:21:47.024 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T05:21:47.027 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:21:47.048 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T05:21:47.049 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:21:47.059 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T05:21:47.089 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T05:21:47.113 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T05:21:47.129 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T05:21:47.152 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T05:21:47.215 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T05:21:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T05:21:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T05:21:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T05:21:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T05:21:47.306 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T05:21:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T05:21:47.334 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:47.355 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T05:21:47.377 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T05:21:47.415 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T05:21:47.421 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:21:47.431 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T05:21:47.444 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:21:47.469 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 39/198 2024-04-16T05:21:47.474 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T05:21:47.504 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 40/198 2024-04-16T05:21:47.507 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 17/198 2024-04-16T05:21:47.532 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 41/198 2024-04-16T05:21:47.538 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T05:21:47.547 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T05:21:47.570 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:21:47.585 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T05:21:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T05:21:47.647 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T05:21:47.719 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:21:47.720 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T05:21:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T05:21:47.751 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T05:21:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T05:21:47.792 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T05:21:47.852 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T05:21:47.891 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T05:21:47.898 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T05:21:47.923 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T05:21:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T05:21:47.969 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:21:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T05:21:48.012 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T05:21:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:21:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T05:21:48.084 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T05:21:48.139 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T05:21:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T05:21:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T05:21:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T05:21:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T05:21:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T05:21:48.353 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T05:21:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T05:21:48.410 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T05:21:48.429 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T05:21:48.485 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T05:21:48.519 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T05:21:48.520 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T05:21:48.561 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T05:21:48.605 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:21:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T05:21:48.614 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T05:21:48.615 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:21:48.644 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T05:21:48.645 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T05:21:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T05:21:48.670 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T05:21:48.694 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T05:21:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T05:21:48.706 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T05:21:48.732 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 53/198 2024-04-16T05:21:48.746 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T05:21:48.762 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T05:21:48.779 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T05:21:48.834 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T05:21:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T05:21:48.873 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T05:21:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T05:21:48.936 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T05:21:48.972 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T05:21:48.979 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T05:21:49.015 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T05:21:49.048 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T05:21:49.076 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T05:21:49.145 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T05:21:49.169 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T05:21:49.180 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 39/198 2024-04-16T05:21:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T05:21:49.224 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 40/198 2024-04-16T05:21:49.244 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 41/198 2024-04-16T05:21:49.258 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T05:21:49.313 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T05:21:49.359 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T05:21:49.432 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T05:21:49.564 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T05:21:49.594 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T05:21:49.619 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T05:21:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T05:21:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T05:21:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T05:21:49.769 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T05:21:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T05:21:49.824 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T05:21:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T05:21:49.903 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T05:21:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T05:21:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T05:21:49.970 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T05:21:50.017 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T05:21:50.053 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T05:21:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T05:21:50.125 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T05:21:50.138 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T05:21:50.139 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T05:21:50.161 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T05:21:50.224 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T05:21:50.242 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T05:21:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T05:21:50.310 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T05:21:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T05:21:50.360 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:21:50.369 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T05:21:50.382 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T05:21:50.400 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T05:21:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T05:21:50.449 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T05:21:50.487 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 53/198 2024-04-16T05:21:50.517 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T05:21:50.517 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T05:21:50.557 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T05:21:50.597 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T05:21:50.622 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T05:21:50.647 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T05:21:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T05:21:50.723 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T05:21:50.847 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T05:21:50.895 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T05:21:50.921 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T05:21:50.947 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T05:21:50.978 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T05:21:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T05:21:51.022 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T05:21:51.036 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T05:21:51.093 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T05:21:51.165 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T05:21:51.211 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T05:21:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T05:21:51.266 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T05:21:51.274 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T05:21:51.338 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T05:21:51.342 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T05:21:51.370 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T05:21:51.407 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T05:21:51.453 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T05:21:51.481 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T05:21:51.489 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T05:21:51.523 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T05:21:51.532 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T05:21:51.555 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T05:21:51.574 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T05:21:51.586 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:21:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T05:21:51.649 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T05:21:51.688 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T05:21:51.707 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T05:21:51.719 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T05:21:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T05:21:51.755 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T05:21:51.785 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T05:21:51.826 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T05:21:51.827 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T05:21:51.868 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T05:21:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T05:21:51.921 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T05:21:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T05:21:51.960 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T05:21:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T05:21:52.000 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T05:21:52.026 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T05:21:52.050 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T05:21:52.066 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T05:21:52.122 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T05:21:52.155 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T05:21:52.208 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T05:21:52.239 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T05:21:52.243 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T05:21:52.286 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T05:21:52.351 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T05:21:52.382 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T05:21:52.392 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T05:21:52.440 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T05:21:52.441 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T05:21:52.496 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T05:21:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T05:21:52.586 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T05:21:52.617 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T05:21:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T05:21:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T05:21:52.678 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T05:21:52.730 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T05:21:52.859 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T05:21:52.894 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T05:21:52.926 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T05:21:52.958 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:21:52.986 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T05:21:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T05:21:53.020 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T05:21:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:21:53.060 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T05:21:53.090 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T05:21:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:21:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 125/198 2024-04-16T05:21:53.146 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T05:21:53.165 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T05:21:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T05:21:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T05:21:53.221 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T05:21:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T05:21:53.264 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T05:21:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T05:21:53.298 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T05:21:53.339 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T05:21:53.365 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T05:21:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T05:21:53.413 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T05:21:53.453 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T05:21:53.543 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T05:21:53.588 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T05:21:53.600 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T05:21:53.628 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T05:21:53.649 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T05:21:53.690 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T05:21:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T05:21:53.716 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:21:53.788 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:21:53.794 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T05:21:53.818 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 129/198 2024-04-16T05:21:54.030 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T05:21:54.137 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T05:21:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T05:21:54.458 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T05:21:54.859 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T05:21:54.910 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T05:21:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T05:21:54.991 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T05:21:55.070 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T05:21:55.111 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T05:21:55.281 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T05:21:55.309 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T05:21:55.387 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T05:21:55.661 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T05:21:55.744 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T05:21:55.775 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T05:21:55.806 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T05:21:55.834 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T05:21:56.018 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T05:21:56.121 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T05:21:56.145 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T05:21:56.172 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:21:56.174 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:56.214 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:56.262 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:21:56.293 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 125/198 2024-04-16T05:21:56.329 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T05:21:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:56.811 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T05:21:56.818 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:21:56.841 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:21:56.896 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:21:56.902 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:21:56.902 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T05:21:56.903 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T05:21:56.903 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:21:56.918 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 129/198 2024-04-16T05:21:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:21:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T05:21:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T05:21:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T05:21:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T05:21:57.508 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T05:21:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T05:21:57.847 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:21:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 124/198 2024-04-16T05:21:57.951 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 125/198 2024-04-16T05:21:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T05:21:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T05:21:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:21:58.553 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 128/198 2024-04-16T05:21:58.576 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 129/198 2024-04-16T05:21:59.172 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:59.752 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:21:59.783 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:21:59.865 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:21:59.865 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T05:21:59.866 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T05:21:59.866 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:21:59.893 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:00.783 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:22:00.822 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:22:01.359 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 130/198 2024-04-16T05:22:01.392 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:22:01.465 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 131/198 2024-04-16T05:22:01.465 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T05:22:01.465 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T05:22:01.465 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:01.493 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /root 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2024-04-16T05:22:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:09.564 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:09.579 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:09.579 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:09.579 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T05:22:09.579 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:11.126 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T05:22:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:11.385 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:11.407 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T05:22:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:11.408 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:11.732 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:11.732 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-04-16T05:22:11.732 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-04-16T05:22:11.732 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-04-16T05:22:11.733 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-04-16T05:22:11.733 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-04-16T05:22:11.733 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-04-16T05:22:11.733 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-04-16T05:22:11.733 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:11.869 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:11.891 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:11.891 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:11.891 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T05:22:11.891 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:11.892 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:11.892 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:11.919 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:11.920 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T05:22:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T05:22:12.424 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:12.485 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 139/198 2024-04-16T05:22:12.547 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 140/198 2024-04-16T05:22:12.575 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 132/198 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /root 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-04-16T05:22:13.192 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:13.358 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:13.373 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 133/198 2024-04-16T05:22:13.373 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:13.373 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T05:22:13.373 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:13.399 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:13.423 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:13.424 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:13.424 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T05:22:13.424 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:13.424 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:13.424 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:13.649 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:13.671 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:13.671 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:13.671 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T05:22:13.671 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:13.671 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:13.671 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:13.702 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:14.119 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:14.141 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:14.141 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:14.141 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T05:22:14.141 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:14.141 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:14.141 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:14.237 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T05:22:14.613 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:14.643 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:14.675 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 139/198 2024-04-16T05:22:14.716 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:14.736 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 140/198 2024-04-16T05:22:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:14.766 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:14.850 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:14.896 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:14.920 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 134/198 2024-04-16T05:22:14.921 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:14.921 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T05:22:14.921 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:14.921 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T05:22:14.921 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 144/198 2024-04-16T05:22:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:15.078 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:15.078 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T05:22:15.078 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:15.078 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:15.121 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:15.132 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:15.138 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:15.160 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 135/198 2024-04-16T05:22:15.160 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:15.160 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T05:22:15.160 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:15.160 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T05:22:15.160 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:15.182 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 147/198 2024-04-16T05:22:15.597 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:15.619 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 136/198 2024-04-16T05:22:15.619 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:15.619 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T05:22:15.619 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:15.619 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T05:22:15.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T05:22:15.799 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:15.842 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:16.154 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T05:22:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:16.190 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 138/198 2024-04-16T05:22:16.212 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:16.213 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:16.213 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T05:22:16.213 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:16.213 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:16.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 139/198 2024-04-16T05:22:16.257 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:16.279 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:16.279 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T05:22:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 140/198 2024-04-16T05:22:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:16.355 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:16.355 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T05:22:16.355 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:16.356 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:16.675 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:16.698 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:16.698 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:16.698 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T05:22:16.698 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:16.698 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:16.698 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:16.800 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:16.835 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:16.934 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 144/198 2024-04-16T05:22:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:17.142 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:17.142 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T05:22:17.142 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:17.142 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:17.198 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:17.243 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 147/198 2024-04-16T05:22:17.363 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 141/198 2024-04-16T05:22:17.397 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:18.165 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T05:22:18.189 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:18.212 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:18.212 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T05:22:18.212 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:18.212 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:18.278 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:18.278 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:18.278 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T05:22:18.278 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:18.324 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:18.345 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:18.346 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:18.346 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T05:22:18.346 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:18.346 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:18.346 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:18.356 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 142/198 2024-04-16T05:22:18.389 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:18.489 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 143/198 2024-04-16T05:22:18.653 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 144/198 2024-04-16T05:22:18.658 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:18.680 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:18.680 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:18.680 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T05:22:18.680 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:18.680 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:18.680 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:18.683 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:18.705 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 145/198 2024-04-16T05:22:18.705 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:18.705 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T05:22:18.705 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:18.705 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T05:22:18.705 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:18.750 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:18.760 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 146/198 2024-04-16T05:22:18.806 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 147/198 2024-04-16T05:22:19.730 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T05:22:19.761 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:19.783 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 149/198 2024-04-16T05:22:19.783 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:19.783 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T05:22:19.783 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:19.783 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T05:22:19.783 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:19.828 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:19.850 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 150/198 2024-04-16T05:22:19.850 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:19.850 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T05:22:19.850 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:19.896 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:19.917 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:19.917 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:19.917 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T05:22:19.917 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:19.918 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T05:22:19.918 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:20.221 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:20.243 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 152/198 2024-04-16T05:22:20.243 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T05:22:20.243 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T05:22:20.243 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:20.243 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T05:22:20.243 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 154/198 2024-04-16T05:22:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 155/198 2024-04-16T05:22:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T05:22:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 157/198 2024-04-16T05:22:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T05:22:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 159/198 2024-04-16T05:22:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T05:22:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T05:22:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T05:22:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 163/198 2024-04-16T05:22:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T05:22:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T05:22:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T05:22:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T05:22:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T05:22:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:23.287 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:23.328 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 154/198 2024-04-16T05:22:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 155/198 2024-04-16T05:22:23.386 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T05:22:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 157/198 2024-04-16T05:22:23.668 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T05:22:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 159/198 2024-04-16T05:22:23.756 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T05:22:23.809 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T05:22:23.852 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T05:22:23.876 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 163/198 2024-04-16T05:22:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T05:22:23.938 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T05:22:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 170/198 2024-04-16T05:22:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T05:22:24.052 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T05:22:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T05:22:24.086 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T05:22:24.096 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T05:22:24.182 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T05:22:24.215 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T05:22:24.241 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T05:22:24.266 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T05:22:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T05:22:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:24.382 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T05:22:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:24.497 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:24.523 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T05:22:24.556 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T05:22:24.581 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T05:22:24.606 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T05:22:24.641 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T05:22:24.687 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T05:22:24.696 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:24.719 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T05:22:24.734 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:24.745 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T05:22:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T05:22:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T05:22:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:24.833 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:24.874 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 154/198 2024-04-16T05:22:24.905 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 155/198 2024-04-16T05:22:24.933 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T05:22:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:25.097 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:25.110 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 157/198 2024-04-16T05:22:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:25.223 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T05:22:25.263 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 159/198 2024-04-16T05:22:25.303 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T05:22:25.310 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T05:22:25.355 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T05:22:25.399 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T05:22:25.415 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 163/198 2024-04-16T05:22:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T05:22:25.447 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T05:22:25.476 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T05:22:25.522 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T05:22:25.565 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T05:22:25.569 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 170/198 2024-04-16T05:22:25.592 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T05:22:25.625 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T05:22:25.662 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T05:22:25.694 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:22:25.702 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T05:22:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T05:22:25.823 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T05:22:25.848 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T05:22:25.874 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T05:22:25.900 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T05:22:25.900 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:25.996 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:26.026 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T05:22:26.026 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:26.110 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:26.137 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T05:22:26.170 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T05:22:26.188 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:26.195 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T05:22:26.220 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T05:22:26.223 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:26.242 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T05:22:26.253 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T05:22:26.293 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T05:22:26.324 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T05:22:26.350 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T05:22:26.375 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T05:22:26.400 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:26.426 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T05:22:26.431 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:26.432 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:26.603 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:26.603 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:26.687 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:26.718 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:26.745 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T05:22:26.771 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T05:22:26.796 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T05:22:26.859 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:22:27.073 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 170/198 2024-04-16T05:22:27.152 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T05:22:27.192 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T05:22:27.274 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T05:22:27.314 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T05:22:27.339 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T05:22:27.364 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T05:22:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T05:22:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:27.471 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T05:22:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T05:22:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:27.585 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T05:22:27.611 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T05:22:27.644 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T05:22:27.670 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T05:22:27.695 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T05:22:27.728 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T05:22:27.767 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T05:22:27.808 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T05:22:27.833 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T05:22:27.859 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T05:22:27.886 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:27.901 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T05:22:27.907 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:27.907 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:28.087 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T05:22:28.087 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:28.178 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T05:22:28.210 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:28.245 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T05:22:28.271 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T05:22:28.296 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T05:22:28.360 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:22:30.665 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:22:30.665 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 1/198 2024-04-16T05:22:30.665 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2/198 2024-04-16T05:22:30.665 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 3/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 4/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 5/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 6/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 7/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 8/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 9/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 13/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 14/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 15/198 2024-04-16T05:22:30.666 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 17/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 18/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 19/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 20/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 21/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 22/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 23/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 24/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 25/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 26/198 2024-04-16T05:22:30.667 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 27/198 2024-04-16T05:22:30.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 28/198 2024-04-16T05:22:30.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 29/198 2024-04-16T05:22:30.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 30/198 2024-04-16T05:22:30.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 31/198 2024-04-16T05:22:30.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 32/198 2024-04-16T05:22:30.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 33/198 2024-04-16T05:22:30.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 34/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 35/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 36/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 37/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 38/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 39/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T05:22:30.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T05:22:30.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T05:22:30.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T05:22:30.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T05:22:30.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T05:22:30.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T05:22:30.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T05:22:30.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T05:22:30.691 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T05:22:30.692 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:30.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T05:22:30.694 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T05:22:30.694 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T05:22:32.287 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:22:32.287 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 1/198 2024-04-16T05:22:32.287 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2/198 2024-04-16T05:22:32.287 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 3/198 2024-04-16T05:22:32.287 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 4/198 2024-04-16T05:22:32.287 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 5/198 2024-04-16T05:22:32.287 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 6/198 2024-04-16T05:22:32.287 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 7/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 8/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 9/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 13/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 14/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 15/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 17/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 18/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 19/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 20/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 21/198 2024-04-16T05:22:32.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 22/198 2024-04-16T05:22:32.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 23/198 2024-04-16T05:22:32.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 24/198 2024-04-16T05:22:32.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 25/198 2024-04-16T05:22:32.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 26/198 2024-04-16T05:22:32.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 27/198 2024-04-16T05:22:32.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 28/198 2024-04-16T05:22:32.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 29/198 2024-04-16T05:22:32.304 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 30/198 2024-04-16T05:22:32.304 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 31/198 2024-04-16T05:22:32.304 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 32/198 2024-04-16T05:22:32.304 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 33/198 2024-04-16T05:22:32.304 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 34/198 2024-04-16T05:22:32.304 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 35/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 36/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 37/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 38/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 39/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T05:22:32.305 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T05:22:32.306 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T05:22:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T05:22:32.308 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T05:22:32.309 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T05:22:32.310 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T05:22:32.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T05:22:32.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T05:22:32.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T05:22:32.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T05:22:32.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T05:22:33.554 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.555 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.556 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.557 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: jq-1.6-16.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T05:22:33.558 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T05:22:33.559 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T05:22:33.560 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T05:22:33.561 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T05:22:33.562 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T05:22:33.563 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T05:22:33.564 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T05:22:33.565 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T05:22:33.566 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T05:22:33.566 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:22:33.566 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 1/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 3/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 4/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.ga87fc 5/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 6/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 7/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 8/198 2024-04-16T05:22:33.723 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 9/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 10/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 11/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 12/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 13/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 14/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_ 15/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 16/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 17/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_6 18/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86 19/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 20/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el 21/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9. 22/198 2024-04-16T05:22:33.724 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_6 23/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 24/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 25/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 26/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 27/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 28/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 29/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6. 30/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noa 31/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.n 32/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87 33/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el 34/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 35/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.e 36/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 37/198 2024-04-16T05:22:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 38/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 39/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T05:22:33.726 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T05:22:33.727 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T05:22:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T05:22:33.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T05:22:33.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T05:22:33.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T05:22:33.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 151/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 153/198 2024-04-16T05:22:33.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T05:22:33.734 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T05:22:33.735 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T05:22:33.736 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T05:22:33.991 DEBUG:teuthology.parallel:result is None 2024-04-16T05:22:35.416 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T05:22:35.416 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.417 INFO:teuthology.orchestra.run.smithi134.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T05:22:35.418 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.419 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T05:22:35.420 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.6-16.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T05:22:35.421 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T05:22:35.422 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T05:22:35.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T05:22:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T05:22:35.425 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.426 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T05:22:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T05:22:35.428 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T05:22:35.429 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:22:35.429 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-16T05:22:35.927 DEBUG:teuthology.parallel:result is None 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T05:22:36.921 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.922 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.923 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.924 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:19.0.0-3101.ga87fcff6.el9.noarch 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-16.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T05:22:36.925 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.926 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T05:22:36.927 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T05:22:36.928 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T05:22:36.929 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T05:22:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T05:22:36.931 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T05:22:36.932 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:19.0.0-3101.ga87fcff6.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T05:22:36.933 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T05:22:36.934 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T05:22:36.934 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-16T05:22:36.934 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-04-16T05:22:37.391 DEBUG:teuthology.parallel:result is None 2024-04-16T05:22:37.392 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:22:37.621 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T05:22:37.646 INFO:teuthology.orchestra.run.smithi032.stdout:19.0.0-3101.ga87fcff6.el9 2024-04-16T05:22:37.647 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.ga87fcff6.el9 2024-04-16T05:22:37.647 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.ga87fcff6 is installed. 2024-04-16T05:22:37.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:22:37.798 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T05:22:37.819 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-3101.ga87fcff6.el9 2024-04-16T05:22:37.819 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.ga87fcff6.el9 2024-04-16T05:22:37.819 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.ga87fcff6 is installed. 2024-04-16T05:22:37.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:22:37.978 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T05:22:38.004 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-3101.ga87fcff6.el9 2024-04-16T05:22:38.004 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.ga87fcff6.el9 2024-04-16T05:22:38.004 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.ga87fcff6 is installed. 2024-04-16T05:22:38.006 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-16T05:22:38.006 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:22:38.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T05:22:38.042 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:22:38.042 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T05:22:38.072 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:22:38.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T05:22:38.106 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-16T05:22:38.106 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:22:38.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T05:22:38.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T05:22:38.209 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:22:38.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T05:22:38.237 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T05:22:38.305 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:22:38.305 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T05:22:38.337 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T05:22:38.408 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-16T05:22:38.408 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:22:38.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T05:22:38.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T05:22:38.511 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:22:38.511 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T05:22:38.538 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T05:22:38.605 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:22:38.605 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T05:22:38.636 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T05:22:38.707 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-16T05:22:38.708 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:22:38.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T05:22:38.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T05:22:38.810 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:22:38.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T05:22:38.836 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T05:22:38.902 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:22:38.903 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T05:22:38.934 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T05:22:39.005 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-16T05:22:39.097 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'subvols': {'create': 2}} 2024-04-16T05:22:39.098 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:22:39.098 INFO:tasks.cephadm:Cluster fsid is 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:22:39.098 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-16T05:22:39.099 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.32', 'mon.b': '172.21.15.82', 'mon.c': '172.21.15.134'} 2024-04-16T05:22:39.099 INFO:tasks.cephadm:First mon is mon.a on smithi032 2024-04-16T05:22:39.099 INFO:tasks.cephadm:First mgr is x 2024-04-16T05:22:39.099 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-16T05:22:39.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname $(hostname -s) 2024-04-16T05:22:39.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2024-04-16T05:22:39.158 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-04-16T05:22:39.190 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-16T05:22:39.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:22:39.408 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a87fcff6340baeedd43af126e793f3241b6400ea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78606/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3101-ga87fcff6', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3101.ga87fcff6'}, 'url': 'https://2.chacra.ceph.com/r/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 04:15:21.905790', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'qa-debug-resource-temporarily-unavailable-issue', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-16T05:22:39.551 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref qa-debug-resource-temporarily-unavailable-issue, sha1 a87fcff6340baeedd43af126e793f3241b6400ea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:22:39.553 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/x86_64/flavors/default/cephadm 2024-04-16T05:22:39.553 DEBUG:teuthology.orchestra.run.smithi032:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:41.223 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r--. 1 ubuntu ubuntu 781558 Apr 16 05:22 /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:41.224 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:43.279 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r--. 1 ubuntu ubuntu 781558 Apr 16 05:22 /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:43.279 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/qa-debug-resource-temporarily-unavailable-issue/a87fcff6340baeedd43af126e793f3241b6400ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:45.472 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 781558 Apr 16 05:22 /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:45.472 DEBUG:teuthology.orchestra.run.smithi032:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:45.492 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:45.510 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T05:22:45.545 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea on all hosts... 2024-04-16T05:22:45.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea pull 2024-04-16T05:22:45.548 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea pull 2024-04-16T05:22:45.553 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea pull 2024-04-16T05:22:46.369 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:22:46.433 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:22:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:23:54.625 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:23:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev)", 2024-04-16T05:23:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "image_id": "3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317", 2024-04-16T05:23:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "repo_digests": [ 2024-04-16T05:23:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1" 2024-04-16T05:23:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:23:54.626 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:24:24.144 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-16T05:24:24.145 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev)", 2024-04-16T05:24:24.145 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317", 2024-04-16T05:24:24.145 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-04-16T05:24:24.145 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1" 2024-04-16T05:24:24.145 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-04-16T05:24:24.145 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-16T05:24:28.621 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-16T05:24:28.621 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev)", 2024-04-16T05:24:28.621 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317", 2024-04-16T05:24:28.621 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2024-04-16T05:24:28.621 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1" 2024-04-16T05:24:28.621 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-16T05:24:28.621 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-16T05:24:28.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph 2024-04-16T05:24:28.684 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2024-04-16T05:24:28.715 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-04-16T05:24:28.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /etc/ceph 2024-04-16T05:24:28.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2024-04-16T05:24:28.811 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-04-16T05:24:28.841 INFO:tasks.cephadm:Writing seed config... 2024-04-16T05:24:28.843 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-16T05:24:28.843 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-16T05:24:28.843 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-16T05:24:28.843 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-16T05:24:28.843 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-16T05:24:28.843 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-16T05:24:28.843 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-16T05:24:28.843 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-16T05:24:28.844 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-16T05:24:28.845 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-16T05:24:28.846 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-16T05:24:28.846 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-16T05:24:28.846 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-16T05:24:28.846 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-16T05:24:28.846 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-16T05:24:28.846 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-16T05:24:28.846 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-16T05:24:28.847 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:24:28.847 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-16T05:24:28.865 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 578203a8-fbb1-11ee-bc8f-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-16T05:24:28.865 DEBUG:teuthology.orchestra.run.smithi032:mon.a> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.a.service 2024-04-16T05:24:28.907 DEBUG:teuthology.orchestra.run.smithi032:mgr.x> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service 2024-04-16T05:24:28.950 INFO:tasks.cephadm:Bootstrapping... 2024-04-16T05:24:28.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea -v bootstrap --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.32 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:24:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-04-16T05:24:29.173 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea', '-v', 'bootstrap', '--fsid', '578203a8-fbb1-11ee-bc8f-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.32', '--skip-admin-label'] 2024-04-16T05:24:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-16T05:24:29.174 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying podman|docker is present... 2024-04-16T05:24:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-16T05:24:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying lvm2 is present... 2024-04-16T05:24:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying time synchronization is in place... 2024-04-16T05:24:29.188 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-16T05:24:29.188 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-16T05:24:29.195 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-16T05:24:29.195 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-04-16T05:24:29.202 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-04-16T05:24:29.209 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-04-16T05:24:29.209 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-04-16T05:24:29.209 INFO:teuthology.orchestra.run.smithi032.stdout:Repeating the final host check... 2024-04-16T05:24:29.233 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-16T05:24:29.233 INFO:teuthology.orchestra.run.smithi032.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-16T05:24:29.233 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl is present 2024-04-16T05:24:29.233 INFO:teuthology.orchestra.run.smithi032.stdout:lvcreate is present 2024-04-16T05:24:29.240 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-16T05:24:29.240 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-16T05:24:29.247 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-16T05:24:29.247 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-04-16T05:24:29.253 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-04-16T05:24:29.260 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-04-16T05:24:29.260 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-04-16T05:24:29.261 INFO:teuthology.orchestra.run.smithi032.stdout:Host looks OK 2024-04-16T05:24:29.261 INFO:teuthology.orchestra.run.smithi032.stdout:Cluster fsid: 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:29.261 INFO:teuthology.orchestra.run.smithi032.stdout:Acquiring lock 140529279910816 on /run/cephadm/578203a8-fbb1-11ee-bc8f-c7b262605968.lock 2024-04-16T05:24:29.261 INFO:teuthology.orchestra.run.smithi032.stdout:Lock 140529279910816 acquired on /run/cephadm/578203a8-fbb1-11ee-bc8f-c7b262605968.lock 2024-04-16T05:24:29.261 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 3300 ... 2024-04-16T05:24:29.261 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 6789 ... 2024-04-16T05:24:29.261 INFO:teuthology.orchestra.run.smithi032.stdout:Base mon IP(s) is [172.21.15.32:3300, 172.21.15.32:6789], mon addrv is [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-04-16T05:24:29.265 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.32 metric 100 2024-04-16T05:24:29.265 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.32 metric 100 2024-04-16T05:24:29.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-16T05:24:29.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-16T05:24:29.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-16T05:24:29.270 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-16T05:24:29.271 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-16T05:24:29.271 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-16T05:24:29.271 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-16T05:24:29.271 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-16T05:24:29.271 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-16T05:24:29.271 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-04-16T05:24:29.272 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-04-16T05:24:29.272 INFO:teuthology.orchestra.run.smithi032.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-16T05:24:29.272 INFO:teuthology.orchestra.run.smithi032.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-16T05:24:29.273 INFO:teuthology.orchestra.run.smithi032.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:24:29.856 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 2024-04-16T05:24:29.857 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea... 2024-04-16T05:24:29.857 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Getting image source signatures 2024-04-16T05:24:29.857 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:831530ac7ca651ac084d96199c7d2480907501d88600b7dc48bade85dc1cca4f 2024-04-16T05:24:29.857 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:9793b5b5582798bde394f4383d142e9f248a25e4ebfd72ee05ea9ec6251a877d 2024-04-16T05:24:29.857 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying config sha256:3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 2024-04-16T05:24:29.857 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-16T05:24:30.727 INFO:teuthology.orchestra.run.smithi032.stdout:ceph: stdout ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev) 2024-04-16T05:24:30.729 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph version: ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev) 2024-04-16T05:24:30.729 INFO:teuthology.orchestra.run.smithi032.stdout:Extracting ceph user uid/gid from container image... 2024-04-16T05:24:31.751 INFO:teuthology.orchestra.run.smithi032.stdout:stat: stdout 167 167 2024-04-16T05:24:31.751 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial keys... 2024-04-16T05:24:32.614 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCQCx5m89fDBBAA6UL3oWOs+roIgKO/q7gg0w== 2024-04-16T05:24:33.638 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCQCx5mOmXcORAADWdC/ynl+g8ttnC+eFpZdA== 2024-04-16T05:24:34.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCRCx5mBeWSOBAAX3qcB8C0Zp3QFn0scKoy8Q== 2024-04-16T05:24:34.477 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial monmap... 2024-04-16T05:24:35.399 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-16T05:24:35.399 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-16T05:24:35.399 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:35.399 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-16T05:24:35.400 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool for a [v2:172.21.15.32:3300,v1:172.21.15.32:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-16T05:24:35.400 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = quincy 2024-04-16T05:24:35.400 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: set fsid to 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:35.400 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-16T05:24:35.400 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:24:35.400 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mon... 2024-04-16T05:24:36.355 INFO:teuthology.orchestra.run.smithi032.stdout:create mon.a on 2024-04-16T05:24:36.567 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-16T05:24:36.710 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-16T05:24:36.885 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968.target → /etc/systemd/system/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968.target. 2024-04-16T05:24:36.885 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968.target → /etc/systemd/system/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968.target. 2024-04-16T05:24:37.111 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.a 2024-04-16T05:24:37.111 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.a.service: Unit ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.a.service not loaded. 2024-04-16T05:24:37.276 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968.target.wants/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.a.service → /etc/systemd/system/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@.service. 2024-04-16T05:24:37.745 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T05:24:37.745 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-04-16T05:24:37.752 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T05:24:37.752 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-04-16T05:24:37.752 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-04-16T05:24:37.752 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-04-16T05:24:37.752 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon to start... 2024-04-16T05:24:37.752 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon... 2024-04-16T05:24:38.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:38 smithi032 ceph-mon[31976]: from='client.? 172.21.15.32:0/3236639666' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-16T05:24:38.881 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout cluster: 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout id: 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout services: 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.559701s) 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:38.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout data: 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pgs: 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.170+0000 7f116b809700 1 Processor -- start 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.171+0000 7f116b809700 1 -- start start 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.171+0000 7f116b809700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.171+0000 7f116b809700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f116407c8d0 con 0x7f116407de50 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.171+0000 7f11695a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.171+0000 7f11695a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45424/0 (socket says 172.21.15.32:45424) 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.171+0000 7f11695a5700 1 -- 172.21.15.32:0/2738522880 learned_addr learned my addr 172.21.15.32:0/2738522880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:38.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.172+0000 7f11695a5700 1 -- 172.21.15.32:0/2738522880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116407ca50 con 0x7f116407de50 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.172+0000 7f11695a5700 1 --2- 172.21.15.32:0/2738522880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116407c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11580098a0 tx=0x7f1158009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff5772818eff6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.172+0000 7f1157fff700 1 -- 172.21.15.32:0/2738522880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1158004160 con 0x7f116407de50 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.172+0000 7f1157fff700 1 -- 172.21.15.32:0/2738522880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1158004300 con 0x7f116407de50 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.172+0000 7f1157fff700 1 -- 172.21.15.32:0/2738522880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11580036a0 con 0x7f116407de50 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.172+0000 7f116b809700 1 -- 172.21.15.32:0/2738522880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 msgr2=0x7f116407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.172+0000 7f116b809700 1 --2- 172.21.15.32:0/2738522880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116407c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11580098a0 tx=0x7f1158009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 -- 172.21.15.32:0/2738522880 shutdown_connections 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 --2- 172.21.15.32:0/2738522880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116407c300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 -- 172.21.15.32:0/2738522880 >> 172.21.15.32:0/2738522880 conn(0x7f1164103b90 msgr2=0x7f1164105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 -- 172.21.15.32:0/2738522880 shutdown_connections 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 -- 172.21.15.32:0/2738522880 wait complete. 2024-04-16T05:24:38.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 Processor -- start 2024-04-16T05:24:38.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 -- start start 2024-04-16T05:24:38.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:38.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f116b809700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f116407cf80 con 0x7f116407de50 2024-04-16T05:24:38.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f11695a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.173+0000 7f11695a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45426/0 (socket says 172.21.15.32:45426) 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f11695a5700 1 -- 172.21.15.32:0/3236639666 learned_addr learned my addr 172.21.15.32:0/3236639666 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f11695a5700 1 -- 172.21.15.32:0/3236639666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116419e370 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f11695a5700 1 --2- 172.21.15.32:0/3236639666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116419de30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1158000c00 tx=0x7f1158004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f11567fc700 1 -- 172.21.15.32:0/3236639666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1158003db0 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f11567fc700 1 -- 172.21.15.32:0/3236639666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1158011630 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116419e600 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f11567fc700 1 -- 172.21.15.32:0/3236639666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1158011950 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11641a12f0 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f11567fc700 1 -- 172.21.15.32:0/3236639666 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1158011af0 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.174+0000 7f11567fc700 1 -- 172.21.15.32:0/3236639666 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f115801c450 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.175+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1164064150 con 0x7f116407de50 2024-04-16T05:24:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.177+0000 7f11567fc700 1 -- 172.21.15.32:0/3236639666 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1158032020 con 0x7f116407de50 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.250+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f11641a1750 con 0x7f116407de50 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.250+0000 7f11567fc700 1 -- 172.21.15.32:0/3236639666 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f1158004260 con 0x7f116407de50 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.251+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 msgr2=0x7f116419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.251+0000 7f116b809700 1 --2- 172.21.15.32:0/3236639666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116419de30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1158000c00 tx=0x7f1158004750 comp rx=0 tx=0).stop 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.251+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 shutdown_connections 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.251+0000 7f116b809700 1 --2- 172.21.15.32:0/3236639666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f116407de50 0x7f116419de30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.251+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 >> 172.21.15.32:0/3236639666 conn(0x7f1164103b90 msgr2=0x7f1164104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.251+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 shutdown_connections 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:38.251+0000 7f116b809700 1 -- 172.21.15.32:0/3236639666 wait complete. 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:mon is available 2024-04-16T05:24:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:Assimilating anything we can from ceph.conf... 2024-04-16T05:24:39.946 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:39 smithi032 ceph-mon[31976]: from='client.? 172.21.15.32:0/22741033' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-16T05:24:39.946 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:39 smithi032 ceph-mon[31976]: from='client.? 172.21.15.32:0/22741033' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-16T05:24:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-04-16T05:24:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-04-16T05:24:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-16T05:24:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-16T05:24:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.293+0000 7f49de5fe700 1 Processor -- start 2024-04-16T05:24:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.293+0000 7f49de5fe700 1 -- start start 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.293+0000 7f49de5fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d807c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.293+0000 7f49de5fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49d807c910 con 0x7f49d807de90 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.293+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d807c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.294+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d807c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44134/0 (socket says 172.21.15.32:44134) 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.294+0000 7f49d7fff700 1 -- 172.21.15.32:0/2766253576 learned_addr learned my addr 172.21.15.32:0/2766253576 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.294+0000 7f49d7fff700 1 -- 172.21.15.32:0/2766253576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d807ca90 con 0x7f49d807de90 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.294+0000 7f49d7fff700 1 --2- 172.21.15.32:0/2766253576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d807c340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f49c80098a0 tx=0x7f49c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3292fb8dd574209 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.294+0000 7f49d6ffd700 1 -- 172.21.15.32:0/2766253576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c8004160 con 0x7f49d807de90 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.294+0000 7f49d6ffd700 1 -- 172.21.15.32:0/2766253576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c8004300 con 0x7f49d807de90 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.294+0000 7f49d6ffd700 1 -- 172.21.15.32:0/2766253576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c80036a0 con 0x7f49d807de90 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 -- 172.21.15.32:0/2766253576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 msgr2=0x7f49d807c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 --2- 172.21.15.32:0/2766253576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d807c340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f49c80098a0 tx=0x7f49c8009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:39.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 -- 172.21.15.32:0/2766253576 shutdown_connections 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 --2- 172.21.15.32:0/2766253576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d807c340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 -- 172.21.15.32:0/2766253576 >> 172.21.15.32:0/2766253576 conn(0x7f49d8103b90 msgr2=0x7f49d8105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 -- 172.21.15.32:0/2766253576 shutdown_connections 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 -- 172.21.15.32:0/2766253576 wait complete. 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 Processor -- start 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.295+0000 7f49de5fe700 1 -- start start 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49de5fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49de5fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49d807cfc0 con 0x7f49d807de90 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d819de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44138/0 (socket says 172.21.15.32:44138) 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49d7fff700 1 -- 172.21.15.32:0/22741033 learned_addr learned my addr 172.21.15.32:0/22741033 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49d7fff700 1 -- 172.21.15.32:0/22741033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d819e3b0 con 0x7f49d807de90 2024-04-16T05:24:39.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49d7fff700 1 --2- 172.21.15.32:0/22741033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d819de70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f49c8000c00 tx=0x7f49c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49d57fa700 1 -- 172.21.15.32:0/22741033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c8003db0 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49d57fa700 1 -- 172.21.15.32:0/22741033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c8011630 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49d57fa700 1 -- 172.21.15.32:0/22741033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c8011950 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d819e640 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.296+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81a1330 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.297+0000 7f49d57fa700 1 -- 172.21.15.32:0/22741033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f49c8011af0 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.297+0000 7f49d57fa700 1 -- 172.21.15.32:0/22741033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f49c801c450 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.298+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d8064150 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.300+0000 7f49d57fa700 1 -- 172.21.15.32:0/22741033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f49c8032020 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.370+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f49d802e0b0 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.372+0000 7f49d57fa700 1 -- 172.21.15.32:0/22741033 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f49c8004260 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.373+0000 7f49d57fa700 1 -- 172.21.15.32:0/22741033 <== mon.0 v2:172.21.15.32:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49c801b450 con 0x7f49d807de90 2024-04-16T05:24:39.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.374+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 msgr2=0x7f49d819de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:39.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.374+0000 7f49de5fe700 1 --2- 172.21.15.32:0/22741033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d819de70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f49c8000c00 tx=0x7f49c8004750 comp rx=0 tx=0).stop 2024-04-16T05:24:39.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.374+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 shutdown_connections 2024-04-16T05:24:39.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.374+0000 7f49de5fe700 1 --2- 172.21.15.32:0/22741033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d807de90 0x7f49d819de70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:39.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.374+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 >> 172.21.15.32:0/22741033 conn(0x7f49d8103b90 msgr2=0x7f49d8103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:39.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.374+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 shutdown_connections 2024-04-16T05:24:39.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:39.374+0000 7f49de5fe700 1 -- 172.21.15.32:0/22741033 wait complete. 2024-04-16T05:24:39.958 INFO:teuthology.orchestra.run.smithi032.stdout:Generating new minimal ceph.conf... 2024-04-16T05:24:40.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:40 smithi032 ceph-mon[31976]: from='client.? 172.21.15.32:0/3490884630' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:24:41.083 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.332+0000 7ff63aa92700 1 Processor -- start 2024-04-16T05:24:41.083 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.332+0000 7ff63aa92700 1 -- start start 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff63aa92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff634108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff63aa92700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff634108fd0 con 0x7ff6341085e0 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff63882e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff634108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff63882e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff634108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44142/0 (socket says 172.21.15.32:44142) 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff63882e700 1 -- 172.21.15.32:0/4037714716 learned_addr learned my addr 172.21.15.32:0/4037714716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff63882e700 1 -- 172.21.15.32:0/4037714716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff634109790 con 0x7ff6341085e0 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff63882e700 1 --2- 172.21.15.32:0/4037714716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff634108a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff630009e80 tx=0x7ff63000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2969e753a9fd466a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff62b7fe700 1 -- 172.21.15.32:0/4037714716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff630017070 con 0x7ff6341085e0 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff62b7fe700 1 -- 172.21.15.32:0/4037714716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff63000b8c0 con 0x7ff6341085e0 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.333+0000 7ff62b7fe700 1 -- 172.21.15.32:0/4037714716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6300045c0 con 0x7ff6341085e0 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.334+0000 7ff63aa92700 1 -- 172.21.15.32:0/4037714716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 msgr2=0x7ff634108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.334+0000 7ff63aa92700 1 --2- 172.21.15.32:0/4037714716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff634108a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff630009e80 tx=0x7ff63000b330 comp rx=0 tx=0).stop 2024-04-16T05:24:41.084 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.334+0000 7ff63aa92700 1 -- 172.21.15.32:0/4037714716 shutdown_connections 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.334+0000 7ff63aa92700 1 --2- 172.21.15.32:0/4037714716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff634108a00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.334+0000 7ff63aa92700 1 -- 172.21.15.32:0/4037714716 >> 172.21.15.32:0/4037714716 conn(0x7ff634103b80 msgr2=0x7ff634105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.334+0000 7ff63aa92700 1 -- 172.21.15.32:0/4037714716 shutdown_connections 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.334+0000 7ff63aa92700 1 -- 172.21.15.32:0/4037714716 wait complete. 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63aa92700 1 Processor -- start 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63aa92700 1 -- start start 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63aa92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff63419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63aa92700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff63410cb40 con 0x7ff6341085e0 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63882e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff63419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63882e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff63419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44158/0 (socket says 172.21.15.32:44158) 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63882e700 1 -- 172.21.15.32:0/3490884630 learned_addr learned my addr 172.21.15.32:0/3490884630 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63882e700 1 -- 172.21.15.32:0/3490884630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63419cac0 con 0x7ff6341085e0 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.335+0000 7ff63882e700 1 --2- 172.21.15.32:0/3490884630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff63419c580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff630009e50 tx=0x7ff63000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.336+0000 7ff629ffb700 1 -- 172.21.15.32:0/3490884630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff630020020 con 0x7ff6341085e0 2024-04-16T05:24:41.085 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.336+0000 7ff629ffb700 1 -- 172.21.15.32:0/3490884630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff630003b50 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.336+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63419cd50 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.336+0000 7ff629ffb700 1 -- 172.21.15.32:0/3490884630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff630017040 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.336+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63419fa40 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.336+0000 7ff629ffb700 1 -- 172.21.15.32:0/3490884630 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff630023020 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.336+0000 7ff629ffb700 1 -- 172.21.15.32:0/3490884630 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff63000f050 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.337+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff634064150 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.339+0000 7ff629ffb700 1 -- 172.21.15.32:0/3490884630 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff63002a020 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.410+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff63419ff00 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.410+0000 7ff629ffb700 1 -- 172.21.15.32:0/3490884630 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff630003d00 con 0x7ff6341085e0 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.411+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 msgr2=0x7ff63419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.411+0000 7ff63aa92700 1 --2- 172.21.15.32:0/3490884630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff63419c580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff630009e50 tx=0x7ff63000bd30 comp rx=0 tx=0).stop 2024-04-16T05:24:41.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.411+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 shutdown_connections 2024-04-16T05:24:41.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.411+0000 7ff63aa92700 1 --2- 172.21.15.32:0/3490884630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6341085e0 0x7ff63419c580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:41.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.411+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 >> 172.21.15.32:0/3490884630 conn(0x7ff634103b80 msgr2=0x7ff634104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:41.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.411+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 shutdown_connections 2024-04-16T05:24:41.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:40.411+0000 7ff63aa92700 1 -- 172.21.15.32:0/3490884630 wait complete. 2024-04-16T05:24:41.087 INFO:teuthology.orchestra.run.smithi032.stdout:Restarting the monitor... 2024-04-16T05:24:41.346 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:41 smithi032 systemd[1]: Stopping Ceph mon.a for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:24:41.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:41 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a[31972]: 2024-04-16T05:24:41.347+0000 7f9322482700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:24:41.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:41 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a[31972]: 2024-04-16T05:24:41.347+0000 7f9322482700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-16T05:24:41.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:41 smithi032 podman[32220]: 2024-04-16 05:24:41.389217003 +0000 UTC m=+0.146188060 container died b2809b7015160aae58fee1573826e61b4b521409049e2df275e4291bc1de8023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:24:41.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:41 smithi032 podman[32220]: 2024-04-16 05:24:41.739240425 +0000 UTC m=+0.496211453 container cleanup b2809b7015160aae58fee1573826e61b4b521409049e2df275e4291bc1de8023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-16T05:24:41.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:41 smithi032 bash[32220]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a 2024-04-16T05:24:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:41 smithi032 podman[32233]: 2024-04-16 05:24:41.973312321 +0000 UTC m=+0.585185091 container remove b2809b7015160aae58fee1573826e61b4b521409049e2df275e4291bc1de8023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:24:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 systemd[1]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-16T05:24:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 systemd[1]: Stopped Ceph mon.a for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:24:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 systemd[1]: Starting Ceph mon.a for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:24:42.575 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 podman[32331]: 2024-04-16 05:24:42.307676164 +0000 UTC m=+0.097824561 container create 38b0b9542547dfa25b8186721c89cec5e63569a5ce8b972c57b9ed3f0f5db857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-16T05:24:42.575 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 podman[32331]: 2024-04-16 05:24:42.226731031 +0000 UTC m=+0.016879443 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:24:42.575 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 podman[32331]: 2024-04-16 05:24:42.501605515 +0000 UTC m=+0.291753915 container init 38b0b9542547dfa25b8186721c89cec5e63569a5ce8b972c57b9ed3f0f5db857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409) 2024-04-16T05:24:42.575 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 podman[32331]: 2024-04-16 05:24:42.504939674 +0000 UTC m=+0.295088074 container start 38b0b9542547dfa25b8186721c89cec5e63569a5ce8b972c57b9ed3f0f5db857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-16T05:24:42.575 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T05:24:42.575 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev), process ceph-mon, pid 2 2024-04-16T05:24:42.575 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: pidfile_write: ignore empty --pid-file 2024-04-16T05:24:42.575 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: load: jerasure load: lrc 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: RocksDB version: 7.9.2 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Git sha 0 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Compile date 2024-04-16 03:54:27 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: DB SUMMARY 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: DB Session ID: UD69J5UWIBO57HXEWZYJ 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: CURRENT file: CURRENT 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105070 ; 2024-04-16T05:24:42.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.error_if_exists: 0 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.create_if_missing: 0 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.paranoid_checks: 1 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.env: 0x5570de592cc0 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.info_log: 0x5570df26fc20 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.statistics: (nil) 2024-04-16T05:24:42.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.use_fsync: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_log_file_size: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.allow_fallocate: 1 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.use_direct_reads: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.db_log_dir: 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.wal_dir: 2024-04-16T05:24:42.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.write_buffer_manager: 0x5570df23e1e0 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T05:24:42.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T05:24:42.580 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.unordered_write: 0 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.row_cache: None 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.wal_filter: None 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T05:24:42.581 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.two_write_queues: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.wal_compression: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.atomic_flush: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.log_readahead_size: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T05:24:42.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_background_jobs: 2 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_background_compactions: -1 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_subcompactions: 1 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_open_files: -1 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T05:24:42.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_background_flushes: -1 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Compression algorithms supported: 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: kZSTD supported: 0 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: kXpressCompression supported: 0 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: kZlibCompression supported: 1 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: kSnappyCompression supported: 1 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: kLZ4Compression supported: 1 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: kBZip2Compression supported: 0 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T05:24:42.584 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.merge_operator: 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_filter: None 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5570df26fde0) 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks: 1 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_top_level_index_and_filter: 1 2024-04-16T05:24:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_type: 0 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_index_type: 0 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_shortening: 1 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: checksum: 4 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: no_block_cache: 0 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache: 0x5570df271090 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_name: BinnedLRUCache 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_options: 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: capacity : 536870912 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_shard_bits : 4 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: strict_capacity_limit : 0 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: high_pri_pool_ratio: 0.000 2024-04-16T05:24:42.586 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_compressed: (nil) 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: persistent_cache: (nil) 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size: 4096 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size_deviation: 10 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_restart_interval: 16 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_block_restart_interval: 1 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: metadata_block_size: 4096 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: partition_filters: 0 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: use_delta_encoding: 1 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: filter_policy: bloomfilter 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: whole_key_filtering: 1 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: verify_compression: 0 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: read_amp_bytes_per_bit: 0 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: format_version: 5 2024-04-16T05:24:42.587 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_index_compression: 1 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_align: 0 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout: max_auto_readahead_size: 262144 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout: prepopulate_block_cache: 0 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout: initial_auto_readahead_size: 8192 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression: NoCompression 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.num_levels: 7 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T05:24:42.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T05:24:42.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T05:24:42.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T05:24:42.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.inplace_update_support: 0 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.bloom_locality: 0 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.max_successive_merges: 0 2024-04-16T05:24:42.592 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.ttl: 2592000 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.enable_blob_files: false 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.min_blob_size: 0 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T05:24:42.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7682654a-8c87-46e1-9c99-6105ba33341a 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245082533978, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245082535020, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100188, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98076, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14165, "raw_average_key_size": 49, "raw_value_size": 90410, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713245082, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7682654a-8c87-46e1-9c99-6105ba33341a", "db_session_id": "UD69J5UWIBO57HXEWZYJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713245082535105, "job": 1, "event": "recovery_finished"} 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T05:24:42.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5570df35e000 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: DB pointer 0x5570df348000 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** DB Stats ** 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-04-16T05:24:42.595 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: L0 2/0 99.70 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: Sum 2/0 99.70 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 139.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T05:24:42.596 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative compaction: 0.00 GB write, 30.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval compaction: 0.00 GB write, 30.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache BinnedLRUCache@0x5570df271090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-16T05:24:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: starting mon.a rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a@-1(???) e1 preinit fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a@-1(???).mds e1 new map 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a@-1(???).mds e1 print_map 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout: e1 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout: legacy client fscid: -1 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout: No filesystems configured 2024-04-16T05:24:42.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T05:24:42.599 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:24:42.599 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:24:42.599 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T05:24:42.599 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-16T05:24:42.599 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 bash[32331]: 38b0b9542547dfa25b8186721c89cec5e63569a5ce8b972c57b9ed3f0f5db857 2024-04-16T05:24:42.599 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 systemd[1]: Started Ceph mon.a for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:24:42.599 INFO:teuthology.orchestra.run.smithi032.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: monmap epoch 1 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: last_changed 2024-04-16T05:24:34.788250+0000 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: created 2024-04-16T05:24:34.788250+0000 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: min_mon_release 19 (squid) 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: election_strategy: 1 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: fsmap 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: osdmap e1: 0 total, 0 up, 0 in 2024-04-16T05:24:42.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:42 smithi032 ceph-mon[32351]: mgrmap e1: no daemons active 2024-04-16T05:24:43.624 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:42.999+0000 7f599da3b700 1 Processor -- start 2024-04-16T05:24:43.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:42.999+0000 7f599da3b700 1 -- start start 2024-04-16T05:24:43.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:42.999+0000 7f599da3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f59981078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:43.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:42.999+0000 7f599da3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5998107eb0 con 0x7f59981074c0 2024-04-16T05:24:43.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.000+0000 7f5996ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f59981078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:43.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.000+0000 7f5996ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f59981078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44162/0 (socket says 172.21.15.32:44162) 2024-04-16T05:24:43.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.000+0000 7f5996ffd700 1 -- 172.21.15.32:0/3291558242 learned_addr learned my addr 172.21.15.32:0/3291558242 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:43.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.000+0000 7f5996ffd700 1 -- 172.21.15.32:0/3291558242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59981086c0 con 0x7f59981074c0 2024-04-16T05:24:43.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.000+0000 7f5996ffd700 1 --2- 172.21.15.32:0/3291558242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f59981078e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5988009e80 tx=0x7f598800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa61e6f87981c04c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.000+0000 7f5995ffb700 1 -- 172.21.15.32:0/3291558242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5988017070 con 0x7f59981074c0 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f5995ffb700 1 -- 172.21.15.32:0/3291558242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f598800b8c0 con 0x7f59981074c0 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f5995ffb700 1 -- 172.21.15.32:0/3291558242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59880045c0 con 0x7f59981074c0 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f599da3b700 1 -- 172.21.15.32:0/3291558242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 msgr2=0x7f59981078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f599da3b700 1 --2- 172.21.15.32:0/3291558242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f59981078e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5988009e80 tx=0x7f598800b330 comp rx=0 tx=0).stop 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f599da3b700 1 -- 172.21.15.32:0/3291558242 shutdown_connections 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f599da3b700 1 --2- 172.21.15.32:0/3291558242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f59981078e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f599da3b700 1 -- 172.21.15.32:0/3291558242 >> 172.21.15.32:0/3291558242 conn(0x7f5998102a40 msgr2=0x7f5998104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f599da3b700 1 -- 172.21.15.32:0/3291558242 shutdown_connections 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.001+0000 7f599da3b700 1 -- 172.21.15.32:0/3291558242 wait complete. 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.002+0000 7f599da3b700 1 Processor -- start 2024-04-16T05:24:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.002+0000 7f599da3b700 1 -- start start 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.002+0000 7f599da3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f599807ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.002+0000 7f599da3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f599810ba70 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.002+0000 7f5996ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f599807ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.002+0000 7f5996ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f599807ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44168/0 (socket says 172.21.15.32:44168) 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.002+0000 7f5996ffd700 1 -- 172.21.15.32:0/4038218438 learned_addr learned my addr 172.21.15.32:0/4038218438 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f5996ffd700 1 -- 172.21.15.32:0/4038218438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599819d040 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f5996ffd700 1 --2- 172.21.15.32:0/4038218438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f599807ba50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59880072a0 tx=0x7f598800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f599ca39700 1 -- 172.21.15.32:0/4038218438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5988020020 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59981a1650 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f599ca39700 1 -- 172.21.15.32:0/4038218438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5988003b50 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f599ca39700 1 -- 172.21.15.32:0/4038218438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5988017040 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59981a1a70 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f599ca39700 1 -- 172.21.15.32:0/4038218438 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5988023020 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.003+0000 7f599ca39700 1 -- 172.21.15.32:0/4038218438 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f598800f050 con 0x7f59981074c0 2024-04-16T05:24:43.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.004+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5998064150 con 0x7f59981074c0 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.006+0000 7f599ca39700 1 -- 172.21.15.32:0/4038218438 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f598802a020 con 0x7f59981074c0 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.076+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f59981a1c40 con 0x7f59981074c0 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.078+0000 7f599ca39700 1 -- 172.21.15.32:0/4038218438 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f59981a1c40 con 0x7f59981074c0 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.078+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 msgr2=0x7f599807ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.078+0000 7f599da3b700 1 --2- 172.21.15.32:0/4038218438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f599807ba50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59880072a0 tx=0x7f598800bd50 comp rx=0 tx=0).stop 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.078+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 shutdown_connections 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.078+0000 7f599da3b700 1 --2- 172.21.15.32:0/4038218438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59981074c0 0x7f599807ba50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.078+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 >> 172.21.15.32:0/4038218438 conn(0x7f5998102a40 msgr2=0x7f5998103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.079+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 shutdown_connections 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:43.079+0000 7f599da3b700 1 -- 172.21.15.32:0/4038218438 wait complete. 2024-04-16T05:24:43.633 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-16T05:24:43.655 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:24:43.655 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mgr... 2024-04-16T05:24:43.655 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-16T05:24:43.656 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-16T05:24:43.908 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.x 2024-04-16T05:24:43.908 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service: Unit ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service not loaded. 2024-04-16T05:24:44.053 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968.target.wants/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@.service. 2024-04-16T05:24:44.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:44 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4038218438' entity='client.admin' 2024-04-16T05:24:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T05:24:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-04-16T05:24:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T05:24:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-04-16T05:24:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-04-16T05:24:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-04-16T05:24:44.600 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T05:24:44.601 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-04-16T05:24:44.608 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T05:24:44.608 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-04-16T05:24:44.608 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-04-16T05:24:44.608 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-16T05:24:44.608 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr to start... 2024-04-16T05:24:44.608 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr... 2024-04-16T05:24:45.441 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:45 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/550125699' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T05:24:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T05:24:45.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T05:24:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T05:24:37.693000+0000", 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.058+0000 7f143e564700 1 Processor -- start 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.058+0000 7f143e564700 1 -- start start 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.059+0000 7f143e564700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f143810b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.059+0000 7f143e564700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f143807be30 con 0x7f14381090a0 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.059+0000 7f1437fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f143810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.059+0000 7f1437fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f143810b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44190/0 (socket says 172.21.15.32:44190) 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.059+0000 7f1437fff700 1 -- 172.21.15.32:0/987100587 learned_addr learned my addr 172.21.15.32:0/987100587 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.059+0000 7f1437fff700 1 -- 172.21.15.32:0/987100587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143810b9d0 con 0x7f14381090a0 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.059+0000 7f1437fff700 1 --2- 172.21.15.32:0/987100587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f143810b490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f14280098a0 tx=0x7f1428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e50a49147c1000 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.059+0000 7f1436ffd700 1 -- 172.21.15.32:0/987100587 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1428004160 con 0x7f14381090a0 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f1436ffd700 1 -- 172.21.15.32:0/987100587 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1428004300 con 0x7f14381090a0 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f1436ffd700 1 -- 172.21.15.32:0/987100587 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1428011b50 con 0x7f14381090a0 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f143e564700 1 -- 172.21.15.32:0/987100587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 msgr2=0x7f143810b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f143e564700 1 --2- 172.21.15.32:0/987100587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f143810b490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f14280098a0 tx=0x7f1428009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f143e564700 1 -- 172.21.15.32:0/987100587 shutdown_connections 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f143e564700 1 --2- 172.21.15.32:0/987100587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f143810b490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f143e564700 1 -- 172.21.15.32:0/987100587 >> 172.21.15.32:0/987100587 conn(0x7f1438102a70 msgr2=0x7f1438104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:45.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f143e564700 1 -- 172.21.15.32:0/987100587 shutdown_connections 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.060+0000 7f143e564700 1 -- 172.21.15.32:0/987100587 wait complete. 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.061+0000 7f143e564700 1 Processor -- start 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.061+0000 7f143e564700 1 -- start start 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.061+0000 7f143e564700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f1438108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.061+0000 7f143e564700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f143810c990 con 0x7f14381090a0 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.061+0000 7f1437fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f1438108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.061+0000 7f1437fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f1438108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44206/0 (socket says 172.21.15.32:44206) 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.061+0000 7f1437fff700 1 -- 172.21.15.32:0/550125699 learned_addr learned my addr 172.21.15.32:0/550125699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f1437fff700 1 -- 172.21.15.32:0/550125699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1438108e60 con 0x7f14381090a0 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f1437fff700 1 --2- 172.21.15.32:0/550125699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f1438108920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1428011b60 tx=0x7f1428011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f14357fa700 1 -- 172.21.15.32:0/550125699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1428003b20 con 0x7f14381090a0 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f14357fa700 1 -- 172.21.15.32:0/550125699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1428003cc0 con 0x7f14381090a0 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1438107050 con 0x7f14381090a0 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f14357fa700 1 -- 172.21.15.32:0/550125699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1428024840 con 0x7f14381090a0 2024-04-16T05:24:45.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1438107590 con 0x7f14381090a0 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f14357fa700 1 -- 172.21.15.32:0/550125699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1428022020 con 0x7f14381090a0 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.062+0000 7f14357fa700 1 -- 172.21.15.32:0/550125699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1428011e00 con 0x7f14381090a0 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.063+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1438064150 con 0x7f14381090a0 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.065+0000 7f14357fa700 1 -- 172.21.15.32:0/550125699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f14280193c0 con 0x7f14381090a0 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.139+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1438107e90 con 0x7f14381090a0 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.140+0000 7f14357fa700 1 -- 172.21.15.32:0/550125699 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f14280195a0 con 0x7f14381090a0 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.140+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 msgr2=0x7f1438108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.140+0000 7f143e564700 1 --2- 172.21.15.32:0/550125699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f1438108920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1428011b60 tx=0x7f1428011c40 comp rx=0 tx=0).stop 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.141+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 shutdown_connections 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.141+0000 7f143e564700 1 --2- 172.21.15.32:0/550125699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14381090a0 0x7f1438108920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.141+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 >> 172.21.15.32:0/550125699 conn(0x7f1438102a70 msgr2=0x7f1438103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.141+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 shutdown_connections 2024-04-16T05:24:45.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:45.141+0000 7f143e564700 1 -- 172.21.15.32:0/550125699 wait complete. 2024-04-16T05:24:45.746 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (1/15)... 2024-04-16T05:24:45.982 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:45 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:45.927+0000 7fb146581200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:24:46.482 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:46.070+0000 7fb146581200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:24:46.482 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:46.168+0000 7fb146581200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:24:46.482 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:46.253+0000 7fb146581200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:24:46.832 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:46.522+0000 7fb146581200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:24:46.833 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:46.643+0000 7fb146581200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:24:46.833 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:46.736+0000 7fb146581200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:24:46.833 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:46.833+0000 7fb146581200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:24:47.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:47 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:47.012+0000 7fb146581200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:24:48.136 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:48.027+0000 7fb146581200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:24:48.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:48 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2514777994' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T05:24:48.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:48.168+0000 7fb146581200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:24:48.811 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:48.514+0000 7fb146581200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:24:48.811 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:48.608+0000 7fb146581200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:24:48.811 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:48.811+0000 7fb146581200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:24:48.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:48.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:24:48.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:24:48.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T05:24:48.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T05:24:48.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T05:24:48.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:48.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T05:24:48.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T05:24:48.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T05:24:48.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T05:24:48.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T05:24:48.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:48.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T05:24:37.693000+0000", 2024-04-16T05:24:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.154+0000 7fa718ed7700 1 Processor -- start 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.154+0000 7fa718ed7700 1 -- start start 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.154+0000 7fa718ed7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa7141072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.154+0000 7fa718ed7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa71407be10 con 0x7fa714106eb0 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.155+0000 7fa71259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa7141072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.155+0000 7fa71259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa7141072d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44212/0 (socket says 172.21.15.32:44212) 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.155+0000 7fa71259c700 1 -- 172.21.15.32:0/2481170849 learned_addr learned my addr 172.21.15.32:0/2481170849 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.155+0000 7fa71259c700 1 -- 172.21.15.32:0/2481170849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa714107810 con 0x7fa714106eb0 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.155+0000 7fa71259c700 1 --2- 172.21.15.32:0/2481170849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa7141072d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0098a0 tx=0x7fa6fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f296d093501415 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.155+0000 7fa71159a700 1 -- 172.21.15.32:0/2481170849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6fc004160 con 0x7fa714106eb0 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.155+0000 7fa71159a700 1 -- 172.21.15.32:0/2481170849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6fc004300 con 0x7fa714106eb0 2024-04-16T05:24:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.155+0000 7fa71159a700 1 -- 172.21.15.32:0/2481170849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6fc011b50 con 0x7fa714106eb0 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.156+0000 7fa718ed7700 1 -- 172.21.15.32:0/2481170849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 msgr2=0x7fa7141072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.156+0000 7fa718ed7700 1 --2- 172.21.15.32:0/2481170849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa7141072d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0098a0 tx=0x7fa6fc009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.156+0000 7fa718ed7700 1 -- 172.21.15.32:0/2481170849 shutdown_connections 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.156+0000 7fa718ed7700 1 --2- 172.21.15.32:0/2481170849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa7141072d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.156+0000 7fa718ed7700 1 -- 172.21.15.32:0/2481170849 >> 172.21.15.32:0/2481170849 conn(0x7fa714102a70 msgr2=0x7fa714104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.156+0000 7fa718ed7700 1 -- 172.21.15.32:0/2481170849 shutdown_connections 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.156+0000 7fa718ed7700 1 -- 172.21.15.32:0/2481170849 wait complete. 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.157+0000 7fa718ed7700 1 Processor -- start 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.157+0000 7fa718ed7700 1 -- start start 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.157+0000 7fa718ed7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa71419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.157+0000 7fa718ed7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141088a0 con 0x7fa714106eb0 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.157+0000 7fa71259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa71419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.157+0000 7fa71259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa71419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44224/0 (socket says 172.21.15.32:44224) 2024-04-16T05:24:48.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.157+0000 7fa71259c700 1 -- 172.21.15.32:0/2514777994 learned_addr learned my addr 172.21.15.32:0/2514777994 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.157+0000 7fa71259c700 1 -- 172.21.15.32:0/2514777994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71419ec90 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.158+0000 7fa71259c700 1 --2- 172.21.15.32:0/2514777994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa71419e750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc011670 tx=0x7fa6fc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.158+0000 7fa70b7fe700 1 -- 172.21.15.32:0/2514777994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6fc0039c0 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.158+0000 7fa70b7fe700 1 -- 172.21.15.32:0/2514777994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6fc011e50 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.158+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71419ef20 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.158+0000 7fa70b7fe700 1 -- 172.21.15.32:0/2514777994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6fc01a5c0 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.158+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7141a1c10 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.158+0000 7fa70b7fe700 1 -- 172.21.15.32:0/2514777994 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6fc01ac50 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.158+0000 7fa70b7fe700 1 -- 172.21.15.32:0/2514777994 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa6fc019700 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.159+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa714064150 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.161+0000 7fa70b7fe700 1 -- 172.21.15.32:0/2514777994 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa6fc003b60 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.236+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa71402e120 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.236+0000 7fa70b7fe700 1 -- 172.21.15.32:0/2514777994 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa6fc0199a0 con 0x7fa714106eb0 2024-04-16T05:24:48.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.237+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 msgr2=0x7fa71419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:48.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.237+0000 7fa718ed7700 1 --2- 172.21.15.32:0/2514777994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa71419e750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc011670 tx=0x7fa6fc011b60 comp rx=0 tx=0).stop 2024-04-16T05:24:48.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.237+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 shutdown_connections 2024-04-16T05:24:48.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.237+0000 7fa718ed7700 1 --2- 172.21.15.32:0/2514777994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa714106eb0 0x7fa71419e750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:48.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.237+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 >> 172.21.15.32:0/2514777994 conn(0x7fa714102a70 msgr2=0x7fa7141953c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:48.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.237+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 shutdown_connections 2024-04-16T05:24:48.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:48.237+0000 7fa718ed7700 1 -- 172.21.15.32:0/2514777994 wait complete. 2024-04-16T05:24:48.835 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (2/15)... 2024-04-16T05:24:49.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:49.003+0000 7fb146581200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:24:49.558 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:49.297+0000 7fb146581200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:24:49.558 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:49.559+0000 7fb146581200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:24:49.982 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:49.804+0000 7fb146581200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:24:50.946 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:50.635+0000 7fb146581200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:24:51.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:51 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1673479953' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T05:24:51.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:51.228+0000 7fb146581200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:24:51.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:51.335+0000 7fb146581200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:24:51.938 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:51.564+0000 7fb146581200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:24:51.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:51.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:24:51.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T05:24:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T05:24:51.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T05:24:51.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T05:24:37.693000+0000", 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:24:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.242+0000 7fbd40ccf700 1 Processor -- start 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.242+0000 7fbd40ccf700 1 -- start start 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.243+0000 7fbd40ccf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c10b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.243+0000 7fbd40ccf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c07bdf0 con 0x7fbd3c1090e0 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.243+0000 7fbd3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c10b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.243+0000 7fbd3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c10b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37730/0 (socket says 172.21.15.32:37730) 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.243+0000 7fbd3a59c700 1 -- 172.21.15.32:0/2884962537 learned_addr learned my addr 172.21.15.32:0/2884962537 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.243+0000 7fbd3a59c700 1 -- 172.21.15.32:0/2884962537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c10ba10 con 0x7fbd3c1090e0 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.243+0000 7fbd3a59c700 1 --2- 172.21.15.32:0/2884962537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c10b4d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0098a0 tx=0x7fbd2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a1624589f1ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.244+0000 7fbd3959a700 1 -- 172.21.15.32:0/2884962537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c004160 con 0x7fbd3c1090e0 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.244+0000 7fbd3959a700 1 -- 172.21.15.32:0/2884962537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd2c004300 con 0x7fbd3c1090e0 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.244+0000 7fbd3959a700 1 -- 172.21.15.32:0/2884962537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c011b50 con 0x7fbd3c1090e0 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.244+0000 7fbd40ccf700 1 -- 172.21.15.32:0/2884962537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 msgr2=0x7fbd3c10b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.244+0000 7fbd40ccf700 1 --2- 172.21.15.32:0/2884962537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c10b4d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0098a0 tx=0x7fbd2c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.245+0000 7fbd40ccf700 1 -- 172.21.15.32:0/2884962537 shutdown_connections 2024-04-16T05:24:51.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.245+0000 7fbd40ccf700 1 --2- 172.21.15.32:0/2884962537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c10b4d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.245+0000 7fbd40ccf700 1 -- 172.21.15.32:0/2884962537 >> 172.21.15.32:0/2884962537 conn(0x7fbd3c102a70 msgr2=0x7fbd3c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.245+0000 7fbd40ccf700 1 -- 172.21.15.32:0/2884962537 shutdown_connections 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.245+0000 7fbd40ccf700 1 -- 172.21.15.32:0/2884962537 wait complete. 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.245+0000 7fbd40ccf700 1 Processor -- start 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.245+0000 7fbd40ccf700 1 -- start start 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.245+0000 7fbd40ccf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd40ccf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c10c980 con 0x7fbd3c1090e0 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37738/0 (socket says 172.21.15.32:37738) 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd3a59c700 1 -- 172.21.15.32:0/1673479953 learned_addr learned my addr 172.21.15.32:0/1673479953 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd3a59c700 1 -- 172.21.15.32:0/1673479953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c19ca60 con 0x7fbd3c1090e0 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd3a59c700 1 --2- 172.21.15.32:0/1673479953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c19c520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c011670 tx=0x7fbd2c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd337fe700 1 -- 172.21.15.32:0/1673479953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c0039c0 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd337fe700 1 -- 172.21.15.32:0/1673479953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd2c011e50 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c19ccf0 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd337fe700 1 -- 172.21.15.32:0/1673479953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c01a5c0 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c19f9e0 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.246+0000 7fbd337fe700 1 -- 172.21.15.32:0/1673479953 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd2c01ac50 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.247+0000 7fbd337fe700 1 -- 172.21.15.32:0/1673479953 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd2c00bb20 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.247+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3c064150 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.251+0000 7fbd337fe700 1 -- 172.21.15.32:0/1673479953 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbd2c043e50 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.325+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd3c19fbb0 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.326+0000 7fbd337fe700 1 -- 172.21.15.32:0/1673479953 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbd2c032020 con 0x7fbd3c1090e0 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.327+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 msgr2=0x7fbd3c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.327+0000 7fbd40ccf700 1 --2- 172.21.15.32:0/1673479953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c19c520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c011670 tx=0x7fbd2c011b60 comp rx=0 tx=0).stop 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.327+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 shutdown_connections 2024-04-16T05:24:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.327+0000 7fbd40ccf700 1 --2- 172.21.15.32:0/1673479953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c1090e0 0x7fbd3c19c520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:51.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.327+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 >> 172.21.15.32:0/1673479953 conn(0x7fbd3c102a70 msgr2=0x7fbd3c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:51.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.327+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 shutdown_connections 2024-04-16T05:24:51.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:51.327+0000 7fbd40ccf700 1 -- 172.21.15.32:0/1673479953 wait complete. 2024-04-16T05:24:51.958 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (3/15)... 2024-04-16T05:24:52.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: Activating manager daemon x 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: mgrmap e2: x(active, starting, since 0.00281048s) 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: Manager daemon x is now available 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' 2024-04-16T05:24:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:52 smithi032 ceph-mon[32351]: from='mgr.14100 172.21.15.32:0/1264621490' entity='mgr.x' 2024-04-16T05:24:53.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:53 smithi032 ceph-mon[32351]: mgrmap e3: x(active, since 1.00712s) 2024-04-16T05:24:54.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:54 smithi032 ceph-mon[32351]: mgrmap e4: x(active, since 2s) 2024-04-16T05:24:55.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:55.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:24:55.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:24:55.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-16T05:24:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T05:24:55.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T05:24:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T05:24:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T05:24:37.693000+0000", 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.378+0000 7f18b5798700 1 Processor -- start 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.378+0000 7f18b5798700 1 -- start start 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.378+0000 7f18b5798700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b010b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.378+0000 7f18b5798700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18b007be50 con 0x7f18b0109090 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.378+0000 7f18aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b010b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:55.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.378+0000 7f18aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b010b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37806/0 (socket says 172.21.15.32:37806) 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.378+0000 7f18aeffd700 1 -- 172.21.15.32:0/1292670556 learned_addr learned my addr 172.21.15.32:0/1292670556 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.379+0000 7f18aeffd700 1 -- 172.21.15.32:0/1292670556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b010b9c0 con 0x7f18b0109090 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.379+0000 7f18aeffd700 1 --2- 172.21.15.32:0/1292670556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b010b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f18a00098a0 tx=0x7f18a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5c27145aa42ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.379+0000 7f18adffb700 1 -- 172.21.15.32:0/1292670556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0004160 con 0x7f18b0109090 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.379+0000 7f18adffb700 1 -- 172.21.15.32:0/1292670556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f18a0004300 con 0x7f18b0109090 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.379+0000 7f18adffb700 1 -- 172.21.15.32:0/1292670556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0011b50 con 0x7f18b0109090 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.379+0000 7f18b5798700 1 -- 172.21.15.32:0/1292670556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 msgr2=0x7f18b010b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.379+0000 7f18b5798700 1 --2- 172.21.15.32:0/1292670556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b010b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f18a00098a0 tx=0x7f18a0009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.380+0000 7f18b5798700 1 -- 172.21.15.32:0/1292670556 shutdown_connections 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.380+0000 7f18b5798700 1 --2- 172.21.15.32:0/1292670556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b010b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.380+0000 7f18b5798700 1 -- 172.21.15.32:0/1292670556 >> 172.21.15.32:0/1292670556 conn(0x7f18b0102a60 msgr2=0x7f18b0104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.380+0000 7f18b5798700 1 -- 172.21.15.32:0/1292670556 shutdown_connections 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.380+0000 7f18b5798700 1 -- 172.21.15.32:0/1292670556 wait complete. 2024-04-16T05:24:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.380+0000 7f18b5798700 1 Processor -- start 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.380+0000 7f18b5798700 1 -- start start 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18b5798700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18b5798700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18b010c990 con 0x7f18b0109090 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37822/0 (socket says 172.21.15.32:37822) 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18aeffd700 1 -- 172.21.15.32:0/449398872 learned_addr learned my addr 172.21.15.32:0/449398872 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18aeffd700 1 -- 172.21.15.32:0/449398872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b019ca90 con 0x7f18b0109090 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18aeffd700 1 --2- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b019c550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f18a0011b60 tx=0x7f18a0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f1897fff700 1 -- 172.21.15.32:0/449398872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0003b20 con 0x7f18b0109090 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f1897fff700 1 -- 172.21.15.32:0/449398872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f18a0003cc0 con 0x7f18b0109090 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b019cd20 con 0x7f18b0109090 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f1897fff700 1 -- 172.21.15.32:0/449398872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0024840 con 0x7f18b0109090 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.381+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b019fa10 con 0x7f18b0109090 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.382+0000 7f1897fff700 1 -- 172.21.15.32:0/449398872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f18a0022020 con 0x7f18b0109090 2024-04-16T05:24:55.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.382+0000 7f1897fff700 1 --2- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f18980395b0 0x7f189803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.382+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b0064150 con 0x7f18b0109090 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.383+0000 7f18ae7fc700 1 --2- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f18980395b0 0x7f189803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.383+0000 7f1897fff700 1 -- 172.21.15.32:0/449398872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18a0011e00 con 0x7f18b0109090 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.383+0000 7f18ae7fc700 1 --2- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f18980395b0 0x7f189803ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f18a4006fd0 tx=0x7f18a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.388+0000 7f1897fff700 1 -- 172.21.15.32:0/449398872 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18a00172b0 con 0x7f18b0109090 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.680+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f18b019fbe0 con 0x7f18b0109090 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.681+0000 7f1897fff700 1 -- 172.21.15.32:0/449398872 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f18a0013110 con 0x7f18b0109090 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f18980395b0 msgr2=0x7f189803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 --2- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f18980395b0 0x7f189803ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f18a4006fd0 tx=0x7f18a4006e40 comp rx=0 tx=0).stop 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 msgr2=0x7f18b019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 --2- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b019c550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f18a0011b60 tx=0x7f18a0011c40 comp rx=0 tx=0).stop 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 shutdown_connections 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 --2- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f18980395b0 0x7f189803ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 --2- 172.21.15.32:0/449398872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b0109090 0x7f18b019c550 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:55.257 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 >> 172.21.15.32:0/449398872 conn(0x7f18b0102a60 msgr2=0x7f18b01035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:55.257 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 shutdown_connections 2024-04-16T05:24:55.257 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:54.683+0000 7f18b5798700 1 -- 172.21.15.32:0/449398872 wait complete. 2024-04-16T05:24:55.257 INFO:teuthology.orchestra.run.smithi032.stdout:mgr is available 2024-04-16T05:24:55.622 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:55 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/449398872' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T05:24:56.461 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-16T05:24:56.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.648+0000 7f076558b700 1 Processor -- start 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.649+0000 7f076558b700 1 -- start start 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.649+0000 7f076558b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076010b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.649+0000 7f076558b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f076007beb0 con 0x7f0760109020 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.649+0000 7f075effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076010b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.649+0000 7f075effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076010b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37834/0 (socket says 172.21.15.32:37834) 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.649+0000 7f075effd700 1 -- 172.21.15.32:0/2122865607 learned_addr learned my addr 172.21.15.32:0/2122865607 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.649+0000 7f075effd700 1 -- 172.21.15.32:0/2122865607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076010b950 con 0x7f0760109020 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.649+0000 7f075effd700 1 --2- 172.21.15.32:0/2122865607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076010b410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f07500098a0 tx=0x7f0750009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fba000a19cb2dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f075dffb700 1 -- 172.21.15.32:0/2122865607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0750004160 con 0x7f0760109020 2024-04-16T05:24:56.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f075dffb700 1 -- 172.21.15.32:0/2122865607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0750004300 con 0x7f0760109020 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f075dffb700 1 -- 172.21.15.32:0/2122865607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0750011b50 con 0x7f0760109020 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f076558b700 1 -- 172.21.15.32:0/2122865607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 msgr2=0x7f076010b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f076558b700 1 --2- 172.21.15.32:0/2122865607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076010b410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f07500098a0 tx=0x7f0750009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f076558b700 1 -- 172.21.15.32:0/2122865607 shutdown_connections 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f076558b700 1 --2- 172.21.15.32:0/2122865607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076010b410 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f076558b700 1 -- 172.21.15.32:0/2122865607 >> 172.21.15.32:0/2122865607 conn(0x7f0760102a70 msgr2=0x7f0760104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f076558b700 1 -- 172.21.15.32:0/2122865607 shutdown_connections 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.650+0000 7f076558b700 1 -- 172.21.15.32:0/2122865607 wait complete. 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.651+0000 7f076558b700 1 Processor -- start 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.651+0000 7f076558b700 1 -- start start 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.651+0000 7f076558b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.651+0000 7f076558b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f076010c980 con 0x7f0760109020 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.651+0000 7f075effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.651+0000 7f075effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37846/0 (socket says 172.21.15.32:37846) 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.651+0000 7f075effd700 1 -- 172.21.15.32:0/234778083 learned_addr learned my addr 172.21.15.32:0/234778083 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.652+0000 7f075effd700 1 -- 172.21.15.32:0/234778083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076019ca80 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.652+0000 7f075effd700 1 --2- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076019c540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0750011670 tx=0x7f0750011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.652+0000 7f0757fff700 1 -- 172.21.15.32:0/234778083 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07500039c0 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.652+0000 7f0757fff700 1 -- 172.21.15.32:0/234778083 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0750011e50 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.652+0000 7f0757fff700 1 -- 172.21.15.32:0/234778083 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f075001a5c0 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.652+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076019cd10 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.652+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076019fa00 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.653+0000 7f0757fff700 1 -- 172.21.15.32:0/234778083 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f075001a870 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.653+0000 7f0757fff700 1 --2- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f074c039560 0x7f074c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.653+0000 7f0757fff700 1 -- 172.21.15.32:0/234778083 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f075004dcd0 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.653+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0760064150 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.653+0000 7f075e7fc700 1 --2- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f074c039560 0x7f074c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.654+0000 7f075e7fc700 1 --2- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f074c039560 0x7f074c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0748006fd0 tx=0x7f0748006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.658+0000 7f0757fff700 1 -- 172.21.15.32:0/234778083 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0750019c70 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.866+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f076019fbd0 con 0x7f0760109020 2024-04-16T05:24:56.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.867+0000 7f0757fff700 1 -- 172.21.15.32:0/234778083 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f075004d6f0 con 0x7f0760109020 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f074c039560 msgr2=0x7f074c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 --2- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f074c039560 0x7f074c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0748006fd0 tx=0x7f0748006e40 comp rx=0 tx=0).stop 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 msgr2=0x7f076019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 --2- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076019c540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0750011670 tx=0x7f0750011b60 comp rx=0 tx=0).stop 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 shutdown_connections 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 --2- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f074c039560 0x7f074c03ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 --2- 172.21.15.32:0/234778083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0760109020 0x7f076019c540 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 >> 172.21.15.32:0/234778083 conn(0x7f0760102a70 msgr2=0x7f0760103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 shutdown_connections 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:55.869+0000 7f076558b700 1 -- 172.21.15.32:0/234778083 wait complete. 2024-04-16T05:24:56.466 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling cephadm module... 2024-04-16T05:24:56.878 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:56 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/234778083' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-16T05:24:57.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2178405969' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-16T05:24:57.890 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:57 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: ignoring --setuser ceph since I am not root 2024-04-16T05:24:57.890 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:57 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: ignoring --setgroup ceph since I am not root 2024-04-16T05:24:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.914+0000 7f6a644cd700 1 Processor -- start 2024-04-16T05:24:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.914+0000 7f6a644cd700 1 -- start start 2024-04-16T05:24:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a644cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c1078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a644cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c107ea0 con 0x7f6a5c1074b0 2024-04-16T05:24:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a62269700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c1078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a62269700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c1078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37858/0 (socket says 172.21.15.32:37858) 2024-04-16T05:24:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a62269700 1 -- 172.21.15.32:0/900329427 learned_addr learned my addr 172.21.15.32:0/900329427 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a62269700 1 -- 172.21.15.32:0/900329427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c1086b0 con 0x7f6a5c1074b0 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a62269700 1 --2- 172.21.15.32:0/900329427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c1078d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009e80 tx=0x7f6a4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ba2ff87860bf8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a61267700 1 -- 172.21.15.32:0/900329427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c017070 con 0x7f6a5c1074b0 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a61267700 1 -- 172.21.15.32:0/900329427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a4c00b8c0 con 0x7f6a5c1074b0 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.915+0000 7f6a61267700 1 -- 172.21.15.32:0/900329427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c0045c0 con 0x7f6a5c1074b0 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.916+0000 7f6a644cd700 1 -- 172.21.15.32:0/900329427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 msgr2=0x7f6a5c1078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.916+0000 7f6a644cd700 1 --2- 172.21.15.32:0/900329427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c1078d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009e80 tx=0x7f6a4c00b330 comp rx=0 tx=0).stop 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.916+0000 7f6a644cd700 1 -- 172.21.15.32:0/900329427 shutdown_connections 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.916+0000 7f6a644cd700 1 --2- 172.21.15.32:0/900329427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c1078d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.916+0000 7f6a644cd700 1 -- 172.21.15.32:0/900329427 >> 172.21.15.32:0/900329427 conn(0x7f6a5c102a70 msgr2=0x7f6a5c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.916+0000 7f6a644cd700 1 -- 172.21.15.32:0/900329427 shutdown_connections 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.916+0000 7f6a644cd700 1 -- 172.21.15.32:0/900329427 wait complete. 2024-04-16T05:24:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a644cd700 1 Processor -- start 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a644cd700 1 -- start start 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a644cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a644cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10ba60 con 0x7f6a5c1074b0 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a62269700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a62269700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37864/0 (socket says 172.21.15.32:37864) 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a62269700 1 -- 172.21.15.32:0/2178405969 learned_addr learned my addr 172.21.15.32:0/2178405969 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a62269700 1 -- 172.21.15.32:0/2178405969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c19ca90 con 0x7f6a5c1074b0 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a62269700 1 --2- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c19c550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009530 tx=0x7f6a4c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.917+0000 7f6a5b7fe700 1 -- 172.21.15.32:0/2178405969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c020020 con 0x7f6a5c1074b0 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.918+0000 7f6a5b7fe700 1 -- 172.21.15.32:0/2178405969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a4c003b50 con 0x7f6a5c1074b0 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.918+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c19cd20 con 0x7f6a5c1074b0 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.918+0000 7f6a5b7fe700 1 -- 172.21.15.32:0/2178405969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c017040 con 0x7f6a5c1074b0 2024-04-16T05:24:58.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.918+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c19fa10 con 0x7f6a5c1074b0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.918+0000 7f6a5b7fe700 1 -- 172.21.15.32:0/2178405969 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f6a4c023020 con 0x7f6a5c1074b0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.919+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5c064150 con 0x7f6a5c1074b0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.919+0000 7f6a5b7fe700 1 --2- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f6a440395b0 0x7f6a4403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.919+0000 7f6a5b7fe700 1 -- 172.21.15.32:0/2178405969 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6a4c00f050 con 0x7f6a5c1074b0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.919+0000 7f6a61a68700 1 --2- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f6a440395b0 0x7f6a4403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.919+0000 7f6a61a68700 1 --2- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f6a440395b0 0x7f6a4403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a50006fd0 tx=0x7f6a50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:56.924+0000 7f6a5b7fe700 1 -- 172.21.15.32:0/2178405969 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a4c02a020 con 0x7f6a5c1074b0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.190+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f6a5c0008d0 con 0x7f6a5c1074b0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.578+0000 7f6a5b7fe700 1 -- 172.21.15.32:0/2178405969 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f6a4c003d70 con 0x7f6a5c1074b0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.579+0000 7f6a5b7fe700 1 -- 172.21.15.32:0/2178405969 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f6a4c025400 con 0x7f6a5c1074b0 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f6a440395b0 msgr2=0x7f6a4403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 --2- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f6a440395b0 0x7f6a4403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a50006fd0 tx=0x7f6a50006e40 comp rx=0 tx=0).stop 2024-04-16T05:24:58.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 msgr2=0x7f6a5c19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 --2- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c19c550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009530 tx=0x7f6a4c00bd50 comp rx=0 tx=0).stop 2024-04-16T05:24:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 shutdown_connections 2024-04-16T05:24:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 --2- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7f6a440395b0 0x7f6a4403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 --2- 172.21.15.32:0/2178405969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a5c1074b0 0x7f6a5c19c550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 >> 172.21.15.32:0/2178405969 conn(0x7f6a5c102a70 msgr2=0x7f6a5c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 shutdown_connections 2024-04-16T05:24:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:57.581+0000 7f6a644cd700 1 -- 172.21.15.32:0/2178405969 wait complete. 2024-04-16T05:24:58.467 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:58 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:58.159+0000 7f0f8899d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:24:58.468 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:58 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:58.262+0000 7f0f8899d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:24:58.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2178405969' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-16T05:24:58.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:58 smithi032 ceph-mon[32351]: mgrmap e5: x(active, since 6s) 2024-04-16T05:24:59.039 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:58 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:58.945+0000 7f0f8899d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:24:59.039 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:59.040+0000 7f0f8899d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:24:59.369 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:59.186+0000 7f0f8899d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:24:59.369 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:59.285+0000 7f0f8899d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:24:59.369 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:59.370+0000 7f0f8899d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-16T05:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T05:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-16T05:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-16T05:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff1255e0700 1 Processor -- start 2024-04-16T05:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff1255e0700 1 -- start start 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff1255e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12010b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff1255e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff12007be10 con 0x7ff1201090e0 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff11effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12010b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff11effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12010b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49608/0 (socket says 172.21.15.32:49608) 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff11effd700 1 -- 172.21.15.32:0/3337769021 learned_addr learned my addr 172.21.15.32:0/3337769021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff11effd700 1 -- 172.21.15.32:0/3337769021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12010ba10 con 0x7ff1201090e0 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.571+0000 7ff11effd700 1 --2- 172.21.15.32:0/3337769021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12010b4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff1100098a0 tx=0x7ff110009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d41c0117a4bcf67 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff11dffb700 1 -- 172.21.15.32:0/3337769021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff110004160 con 0x7ff1201090e0 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff11dffb700 1 -- 172.21.15.32:0/3337769021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff110004300 con 0x7ff1201090e0 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff11dffb700 1 -- 172.21.15.32:0/3337769021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff110011b50 con 0x7ff1201090e0 2024-04-16T05:24:59.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff1255e0700 1 -- 172.21.15.32:0/3337769021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 msgr2=0x7ff12010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff1255e0700 1 --2- 172.21.15.32:0/3337769021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12010b4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff1100098a0 tx=0x7ff110009bb0 comp rx=0 tx=0).stop 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff1255e0700 1 -- 172.21.15.32:0/3337769021 shutdown_connections 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff1255e0700 1 --2- 172.21.15.32:0/3337769021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12010b4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff1255e0700 1 -- 172.21.15.32:0/3337769021 >> 172.21.15.32:0/3337769021 conn(0x7ff120102a50 msgr2=0x7ff120104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff1255e0700 1 -- 172.21.15.32:0/3337769021 shutdown_connections 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.572+0000 7ff1255e0700 1 -- 172.21.15.32:0/3337769021 wait complete. 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.573+0000 7ff1255e0700 1 Processor -- start 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.573+0000 7ff1255e0700 1 -- start start 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.573+0000 7ff1255e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.573+0000 7ff1255e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff12010c9d0 con 0x7ff1201090e0 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.573+0000 7ff11effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.573+0000 7ff11effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12019c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49614/0 (socket says 172.21.15.32:49614) 2024-04-16T05:24:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.573+0000 7ff11effd700 1 -- 172.21.15.32:0/30268756 learned_addr learned my addr 172.21.15.32:0/30268756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.574+0000 7ff11effd700 1 -- 172.21.15.32:0/30268756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12019caf0 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.574+0000 7ff11effd700 1 --2- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12019c5b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff110011b60 tx=0x7ff110011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.574+0000 7ff107fff700 1 -- 172.21.15.32:0/30268756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff110003970 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.574+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12019cd80 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.574+0000 7ff107fff700 1 -- 172.21.15.32:0/30268756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff110003b10 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.574+0000 7ff107fff700 1 -- 172.21.15.32:0/30268756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff110024610 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.574+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12019fa70 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.575+0000 7ff107fff700 1 -- 172.21.15.32:0/30268756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff1100248c0 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.575+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff120064150 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.575+0000 7ff107fff700 1 --2- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7ff108039600 0x7ff10803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.575+0000 7ff107fff700 1 -- 172.21.15.32:0/30268756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff110011e00 con 0x7ff1201090e0 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.575+0000 7ff11e7fc700 1 -- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7ff108039600 msgr2=0x7ff10803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3517252554 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.575+0000 7ff11e7fc700 1 --2- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7ff108039600 0x7ff10803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:24:59.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.580+0000 7ff107fff700 1 -- 172.21.15.32:0/30268756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1100172b0 con 0x7ff1201090e0 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.776+0000 7ff11e7fc700 1 -- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7ff108039600 msgr2=0x7ff10803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3517252554 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.776+0000 7ff11e7fc700 1 --2- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7ff108039600 0x7ff10803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.851+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff12019fc40 con 0x7ff1201090e0 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.851+0000 7ff107fff700 1 -- 172.21.15.32:0/30268756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff11000ba00 con 0x7ff1201090e0 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.853+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7ff108039600 msgr2=0x7ff10803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.853+0000 7ff1255e0700 1 --2- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7ff108039600 0x7ff10803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.853+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 msgr2=0x7ff12019c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.853+0000 7ff1255e0700 1 --2- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12019c5b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff110011b60 tx=0x7ff110011c40 comp rx=0 tx=0).stop 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.853+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 shutdown_connections 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.853+0000 7ff1255e0700 1 --2- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7ff108039600 0x7ff10803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.853+0000 7ff1255e0700 1 --2- 172.21.15.32:0/30268756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1201090e0 0x7ff12019c5b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.853+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 >> 172.21.15.32:0/30268756 conn(0x7ff120102a50 msgr2=0x7ff120103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.854+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 shutdown_connections 2024-04-16T05:24:59.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:58.854+0000 7ff1255e0700 1 -- 172.21.15.32:0/30268756 wait complete. 2024-04-16T05:24:59.494 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-04-16T05:24:59.494 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 5... 2024-04-16T05:24:59.648 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/30268756' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-16T05:24:59.961 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:59.650+0000 7f0f8899d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:24:59.961 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:59.766+0000 7f0f8899d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:24:59.961 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:59.860+0000 7f0f8899d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:25:00.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:24:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:24:59.962+0000 7f0f8899d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:25:00.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:00.148+0000 7f0f8899d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:25:01.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:01.170+0000 7f0f8899d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:25:01.643 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:01.305+0000 7f0f8899d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:25:01.643 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:01.645+0000 7f0f8899d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:25:01.937 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:01.737+0000 7f0f8899d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:25:01.938 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:01.939+0000 7f0f8899d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:02.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:02.129+0000 7f0f8899d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:25:02.684 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:02.422+0000 7f0f8899d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:02.684 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:02.685+0000 7f0f8899d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:02.982 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:02.930+0000 7f0f8899d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:25:04.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:03 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:03.761+0000 7f0f8899d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:25:04.668 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:04.345+0000 7f0f8899d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:25:04.668 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:04.446+0000 7f0f8899d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:25:04.669 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:04.670+0000 7f0f8899d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:25:04.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: Active manager daemon x restarted 2024-04-16T05:25:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: Activating manager daemon x 2024-04-16T05:25:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: osdmap e2: 0 total, 0 up, 0 in 2024-04-16T05:25:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: mgrmap e6: x(active, starting, since 0.00325736s) 2024-04-16T05:25:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:25:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:25:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:25:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:25:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:25:04.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: Manager daemon x is now available 2024-04-16T05:25:04.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:04.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:04.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:04.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:04.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:04 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:25:05.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:05 smithi032 ceph-mon[32351]: Found migration_current of "None". Setting to last migration. 2024-04-16T05:25:05.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:05 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:25:05.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:05 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:05.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:05 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:05.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:05 smithi032 ceph-mon[32351]: mgrmap e7: x(active, since 1.00513s) 2024-04-16T05:25:06.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:06.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-16T05:25:06.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-16T05:25:06.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.890+0000 7efe8851a700 1 Processor -- start 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.890+0000 7efe8851a700 1 -- start start 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.890+0000 7efe8851a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe80108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.890+0000 7efe8851a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efe80108fd0 con 0x7efe801085e0 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.891+0000 7efe862b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe80108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.891+0000 7efe862b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe80108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49626/0 (socket says 172.21.15.32:49626) 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.891+0000 7efe862b6700 1 -- 172.21.15.32:0/244313661 learned_addr learned my addr 172.21.15.32:0/244313661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.891+0000 7efe862b6700 1 -- 172.21.15.32:0/244313661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe80109790 con 0x7efe801085e0 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.891+0000 7efe862b6700 1 --2- 172.21.15.32:0/244313661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe80108a00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0098a0 tx=0x7efe7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a4983c9b7ba0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.891+0000 7efe852b4700 1 -- 172.21.15.32:0/244313661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe7c004160 con 0x7efe801085e0 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.891+0000 7efe852b4700 1 -- 172.21.15.32:0/244313661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe7c004300 con 0x7efe801085e0 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.891+0000 7efe852b4700 1 -- 172.21.15.32:0/244313661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe7c011b50 con 0x7efe801085e0 2024-04-16T05:25:06.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.892+0000 7efe8851a700 1 -- 172.21.15.32:0/244313661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 msgr2=0x7efe80108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.892+0000 7efe8851a700 1 --2- 172.21.15.32:0/244313661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe80108a00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0098a0 tx=0x7efe7c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.892+0000 7efe8851a700 1 -- 172.21.15.32:0/244313661 shutdown_connections 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.892+0000 7efe8851a700 1 --2- 172.21.15.32:0/244313661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe80108a00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.892+0000 7efe8851a700 1 -- 172.21.15.32:0/244313661 >> 172.21.15.32:0/244313661 conn(0x7efe80103b80 msgr2=0x7efe80105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.892+0000 7efe8851a700 1 -- 172.21.15.32:0/244313661 shutdown_connections 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.892+0000 7efe8851a700 1 -- 172.21.15.32:0/244313661 wait complete. 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.893+0000 7efe8851a700 1 Processor -- start 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.893+0000 7efe8851a700 1 -- start start 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.893+0000 7efe8851a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe8007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.893+0000 7efe8851a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efe8010cb40 con 0x7efe801085e0 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.893+0000 7efe862b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe8007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.893+0000 7efe862b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe8007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49630/0 (socket says 172.21.15.32:49630) 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.893+0000 7efe862b6700 1 -- 172.21.15.32:0/3776406931 learned_addr learned my addr 172.21.15.32:0/3776406931 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:06.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.893+0000 7efe862b6700 1 -- 172.21.15.32:0/3776406931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8007e0b0 con 0x7efe801085e0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.894+0000 7efe862b6700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe8007db70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009fb0 tx=0x7efe7c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.894+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe7c0038b0 con 0x7efe801085e0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.894+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe7c011e50 con 0x7efe801085e0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.894+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8007c2a0 con 0x7efe801085e0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.894+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe7c019550 con 0x7efe801085e0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.894+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8007c7e0 con 0x7efe801085e0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.895+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7efe7c019850 con 0x7efe801085e0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.895+0000 7efe777fe700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 0x7efe6c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.895+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 --> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efe6c03c130 con 0x7efe6c039560 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.895+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efe7c04d980 con 0x7efe801085e0 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.895+0000 7efe85ab5700 1 -- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 msgr2=0x7efe6c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3517252554 2024-04-16T05:25:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:24:59.895+0000 7efe85ab5700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 0x7efe6c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.096+0000 7efe85ab5700 1 -- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 msgr2=0x7efe6c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3517252554 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.096+0000 7efe85ab5700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 0x7efe6c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.497+0000 7efe85ab5700 1 -- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 msgr2=0x7efe6c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3517252554 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:00.497+0000 7efe85ab5700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 0x7efe6c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:01.298+0000 7efe85ab5700 1 -- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 msgr2=0x7efe6c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3517252554 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:01.298+0000 7efe85ab5700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 0x7efe6c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:02.899+0000 7efe85ab5700 1 -- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 msgr2=0x7efe6c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3517252554 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:02.899+0000 7efe85ab5700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 0x7efe6c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:04.673+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7efe7c019b90 con 0x7efe801085e0 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:04.673+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 msgr2=0x7efe6c03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:04.673+0000 7efe777fe700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 0x7efe6c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.674+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7efe7c0205d0 con 0x7efe801085e0 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.674+0000 7efe777fe700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7efe6c03ce50 0x7efe6c03f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.674+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efe7c003bf0 con 0x7efe6c03ce50 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.674+0000 7efe85ab5700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7efe6c03ce50 0x7efe6c03f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.675+0000 7efe85ab5700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7efe6c03ce50 0x7efe6c03f240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efe70003a10 tx=0x7efe70009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.676+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mgr.14120 v2:172.21.15.32:6800/1590295441 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7efe7c003bf0 con 0x7efe6c03ce50 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.682+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7efe80064150 con 0x7efe6c03ce50 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe777fe700 1 -- 172.21.15.32:0/3776406931 <== mgr.14120 v2:172.21.15.32:6800/1590295441 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7efe80064150 con 0x7efe6c03ce50 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7efe6c03ce50 msgr2=0x7efe6c03f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7efe6c03ce50 0x7efe6c03f240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efe70003a10 tx=0x7efe70009290 comp rx=0 tx=0).stop 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 msgr2=0x7efe8007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe8007db70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009fb0 tx=0x7efe7c011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 shutdown_connections 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7efe6c03ce50 0x7efe6c03f240 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe801085e0 0x7efe8007db70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:06.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 --2- 172.21.15.32:0/3776406931 >> [v2:172.21.15.32:6800/3517252554,v1:172.21.15.32:6801/3517252554] conn(0x7efe6c039560 0x7efe6c03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:06.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 >> 172.21.15.32:0/3776406931 conn(0x7efe80103b80 msgr2=0x7efe801047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:06.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 shutdown_connections 2024-04-16T05:25:06.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:05.683+0000 7efe8851a700 1 -- 172.21.15.32:0/3776406931 wait complete. 2024-04-16T05:25:06.265 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 5 is available 2024-04-16T05:25:06.265 INFO:teuthology.orchestra.run.smithi032.stdout:Setting orchestrator backend to cephadm... 2024-04-16T05:25:06.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:06 smithi032 ceph-mon[32351]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-16T05:25:06.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:06 smithi032 ceph-mon[32351]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-16T05:25:07.626 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.703+0000 7fea2fff5700 1 Processor -- start 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.703+0000 7fea2fff5700 1 -- start start 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.703+0000 7fea2fff5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.703+0000 7fea2fff5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fea2807c8f0 con 0x7fea2807de70 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.703+0000 7fea2dd91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.704+0000 7fea2dd91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49716/0 (socket says 172.21.15.32:49716) 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.704+0000 7fea2dd91700 1 -- 172.21.15.32:0/2222217323 learned_addr learned my addr 172.21.15.32:0/2222217323 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.704+0000 7fea2dd91700 1 -- 172.21.15.32:0/2222217323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2807ca70 con 0x7fea2807de70 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.704+0000 7fea2dd91700 1 --2- 172.21.15.32:0/2222217323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2807c320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fea240098a0 tx=0x7fea24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280a8c60f8e2e369 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.704+0000 7fea2cd8f700 1 -- 172.21.15.32:0/2222217323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea24004160 con 0x7fea2807de70 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.704+0000 7fea2cd8f700 1 -- 172.21.15.32:0/2222217323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea24004300 con 0x7fea2807de70 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.704+0000 7fea2cd8f700 1 -- 172.21.15.32:0/2222217323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea24011b50 con 0x7fea2807de70 2024-04-16T05:25:07.627 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.705+0000 7fea2fff5700 1 -- 172.21.15.32:0/2222217323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 msgr2=0x7fea2807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.705+0000 7fea2fff5700 1 --2- 172.21.15.32:0/2222217323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2807c320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fea240098a0 tx=0x7fea24009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.705+0000 7fea2fff5700 1 -- 172.21.15.32:0/2222217323 shutdown_connections 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.705+0000 7fea2fff5700 1 --2- 172.21.15.32:0/2222217323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2807c320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.705+0000 7fea2fff5700 1 -- 172.21.15.32:0/2222217323 >> 172.21.15.32:0/2222217323 conn(0x7fea28103bf0 msgr2=0x7fea28106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.705+0000 7fea2fff5700 1 -- 172.21.15.32:0/2222217323 shutdown_connections 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.705+0000 7fea2fff5700 1 -- 172.21.15.32:0/2222217323 wait complete. 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2fff5700 1 Processor -- start 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2fff5700 1 -- start start 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2fff5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2fff5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fea2810eb60 con 0x7fea2807de70 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2dd91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2dd91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49722/0 (socket says 172.21.15.32:49722) 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2dd91700 1 -- 172.21.15.32:0/3815532990 learned_addr learned my addr 172.21.15.32:0/3815532990 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2dd91700 1 -- 172.21.15.32:0/3815532990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2819ed00 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.706+0000 7fea2dd91700 1 --2- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2819e7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fea24009fb0 tx=0x7fea24011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.707+0000 7fea1affd700 1 -- 172.21.15.32:0/3815532990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea240038b0 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.707+0000 7fea1affd700 1 -- 172.21.15.32:0/3815532990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea24011e50 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.707+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2819ef90 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.707+0000 7fea1affd700 1 -- 172.21.15.32:0/3815532990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea24019420 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.707+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea281a1c80 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.707+0000 7fea1affd700 1 -- 172.21.15.32:0/3815532990 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fea240196d0 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.708+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea28064150 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.708+0000 7fea1affd700 1 --2- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fea140394a0 0x7fea1403b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.708+0000 7fea1affd700 1 -- 172.21.15.32:0/3815532990 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea2404d910 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.708+0000 7fea2d590700 1 --2- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fea140394a0 0x7fea1403b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.708+0000 7fea2d590700 1 --2- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fea140394a0 0x7fea1403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea1c006fd0 tx=0x7fea1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.713+0000 7fea1affd700 1 -- 172.21.15.32:0/3815532990 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea2400bc70 con 0x7fea2807de70 2024-04-16T05:25:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:06.727+0000 7fea1affd700 1 -- 172.21.15.32:0/3815532990 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fea2402d7c0 con 0x7fea2807de70 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.007+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fea28104ee0 con 0x7fea140394a0 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.012+0000 7fea1affd700 1 -- 172.21.15.32:0/3815532990 <== mgr.14120 v2:172.21.15.32:6800/1590295441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fea28104ee0 con 0x7fea140394a0 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fea140394a0 msgr2=0x7fea1403b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 --2- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fea140394a0 0x7fea1403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea1c006fd0 tx=0x7fea1c006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 msgr2=0x7fea2819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 --2- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2819e7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fea24009fb0 tx=0x7fea24011bf0 comp rx=0 tx=0).stop 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 shutdown_connections 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 --2- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fea140394a0 0x7fea1403b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 --2- 172.21.15.32:0/3815532990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2807de70 0x7fea2819e7c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 >> 172.21.15.32:0/3815532990 conn(0x7fea28103bf0 msgr2=0x7fea281047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 shutdown_connections 2024-04-16T05:25:07.630 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:07.014+0000 7fea2fff5700 1 -- 172.21.15.32:0/3815532990 wait complete. 2024-04-16T05:25:07.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: mgrmap e8: x(active, since 2s) 2024-04-16T05:25:07.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:07.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:07.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:07.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: [16/Apr/2024:05:25:07] ENGINE Bus STARTING 2024-04-16T05:25:07.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: [16/Apr/2024:05:25:07] ENGINE Serving on https://172.21.15.32:7150 2024-04-16T05:25:07.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: [16/Apr/2024:05:25:07] ENGINE Serving on http://172.21.15.32:8765 2024-04-16T05:25:07.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: [16/Apr/2024:05:25:07] ENGINE Bus STARTED 2024-04-16T05:25:07.956 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:07 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:08.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:08 smithi032 ceph-mon[32351]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:08.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-16T05:25:08.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.062+0000 7f0558efe700 1 Processor -- start 2024-04-16T05:25:08.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.062+0000 7f0558efe700 1 -- start start 2024-04-16T05:25:08.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.062+0000 7f0558efe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f05541078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:08.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.062+0000 7f0558efe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0554107e70 con 0x7f0554107480 2024-04-16T05:25:08.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.062+0000 7f055259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f05541078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:08.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.062+0000 7f055259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f05541078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49736/0 (socket says 172.21.15.32:49736) 2024-04-16T05:25:08.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.062+0000 7f055259c700 1 -- 172.21.15.32:0/2714926456 learned_addr learned my addr 172.21.15.32:0/2714926456 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:08.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.062+0000 7f055259c700 1 -- 172.21.15.32:0/2714926456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0554108680 con 0x7f0554107480 2024-04-16T05:25:08.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.063+0000 7f055259c700 1 --2- 172.21.15.32:0/2714926456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f05541078a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f054400bb40 tx=0x7f054400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96f6ef440004da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.063+0000 7f055159a700 1 -- 172.21.15.32:0/2714926456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f054400dbe0 con 0x7f0554107480 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.063+0000 7f055159a700 1 -- 172.21.15.32:0/2714926456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05440044f0 con 0x7f0554107480 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.063+0000 7f055159a700 1 -- 172.21.15.32:0/2714926456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0544010470 con 0x7f0554107480 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.063+0000 7f0558efe700 1 -- 172.21.15.32:0/2714926456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 msgr2=0x7f05541078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.063+0000 7f0558efe700 1 --2- 172.21.15.32:0/2714926456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f05541078a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f054400bb40 tx=0x7f054400be50 comp rx=0 tx=0).stop 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.064+0000 7f0558efe700 1 -- 172.21.15.32:0/2714926456 shutdown_connections 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.064+0000 7f0558efe700 1 --2- 172.21.15.32:0/2714926456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f05541078a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.064+0000 7f0558efe700 1 -- 172.21.15.32:0/2714926456 >> 172.21.15.32:0/2714926456 conn(0x7f05541029e0 msgr2=0x7f0554104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.064+0000 7f0558efe700 1 -- 172.21.15.32:0/2714926456 shutdown_connections 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.064+0000 7f0558efe700 1 -- 172.21.15.32:0/2714926456 wait complete. 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.064+0000 7f0558efe700 1 Processor -- start 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.064+0000 7f0558efe700 1 -- start start 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f0558efe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f055419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f0558efe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f055407afd0 con 0x7f0554107480 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f055259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f055419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f055259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f055419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49744/0 (socket says 172.21.15.32:49744) 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f055259c700 1 -- 172.21.15.32:0/3219675882 learned_addr learned my addr 172.21.15.32:0/3219675882 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f055259c700 1 -- 172.21.15.32:0/3219675882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055419ec90 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f055259c700 1 --2- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f055419e750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f054400b340 tx=0x7f05440040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f05437fe700 1 -- 172.21.15.32:0/3219675882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0544010720 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f05437fe700 1 -- 172.21.15.32:0/3219675882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f054401d070 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055419ef20 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f05437fe700 1 -- 172.21.15.32:0/3219675882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0544021c20 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.065+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05541a1c10 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.066+0000 7f05437fe700 1 -- 172.21.15.32:0/3219675882 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f054401f030 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.066+0000 7f05437fe700 1 --2- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f053c039560 0x7f053c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.066+0000 7f05437fe700 1 -- 172.21.15.32:0/3219675882 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f054404da70 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.066+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0554064150 con 0x7f0554107480 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.066+0000 7f0551d9b700 1 --2- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f053c039560 0x7f053c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.067+0000 7f0551d9b700 1 --2- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f053c039560 0x7f053c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0548006fd0 tx=0x7f0548006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:08.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.072+0000 7f05437fe700 1 -- 172.21.15.32:0/3219675882 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0544015070 con 0x7f0554107480 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.281+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f0554062f30 con 0x7f053c039560 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.281+0000 7f05437fe700 1 -- 172.21.15.32:0/3219675882 <== mgr.14120 v2:172.21.15.32:6800/1590295441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f0554062f30 con 0x7f053c039560 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.283+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f053c039560 msgr2=0x7f053c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.283+0000 7f0558efe700 1 --2- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f053c039560 0x7f053c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0548006fd0 tx=0x7f0548006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.283+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 msgr2=0x7f055419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.283+0000 7f0558efe700 1 --2- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f055419e750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f054400b340 tx=0x7f05440040b0 comp rx=0 tx=0).stop 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.283+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 shutdown_connections 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.283+0000 7f0558efe700 1 --2- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f053c039560 0x7f053c03ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.283+0000 7f0558efe700 1 --2- 172.21.15.32:0/3219675882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0554107480 0x7f055419e750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.283+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 >> 172.21.15.32:0/3219675882 conn(0x7f05541029e0 msgr2=0x7f0554190980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.284+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 shutdown_connections 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:08.284+0000 7f0558efe700 1 -- 172.21.15.32:0/3219675882 wait complete. 2024-04-16T05:25:08.930 INFO:teuthology.orchestra.run.smithi032.stdout:Generating ssh key... 2024-04-16T05:25:09.906 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-mon[32351]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:09.906 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-mon[32351]: Generating ssh key... 2024-04-16T05:25:10.198 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: Generating public/private rsa key pair. 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: Your identification has been saved in /tmp/tmp7jiovs6c/key. 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: Your public key has been saved in /tmp/tmp7jiovs6c/key.pub. 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: The key fingerprint is: 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: SHA256:ykwKfhojF9OUH7od3XSPqSjDt6EKkyBr8eW9fJ1mbiY ceph-578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: The key's randomart image is: 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: +---[RSA 3072]----+ 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: | | 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: | . | 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: | o . . . | 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: | o o o o . + | 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: |o.+ o.+ S . o . | 2024-04-16T05:25:10.199 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: |.+o=oO.o . . | 2024-04-16T05:25:10.200 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: |o.O.+.O.+ o . | 2024-04-16T05:25:10.200 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: |.o B .=.E B | 2024-04-16T05:25:10.200 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: | . ...oo B. | 2024-04-16T05:25:10.200 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: +----[SHA256]-----+ 2024-04-16T05:25:10.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.335+0000 7f749cbb3700 1 Processor -- start 2024-04-16T05:25:10.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.335+0000 7f749cbb3700 1 -- start start 2024-04-16T05:25:10.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749cbb3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f74981072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:10.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749cbb3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f749807be30 con 0x7f7498106eb0 2024-04-16T05:25:10.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f74981072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:10.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f74981072d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41852/0 (socket says 172.21.15.32:41852) 2024-04-16T05:25:10.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749659c700 1 -- 172.21.15.32:0/4124226150 learned_addr learned my addr 172.21.15.32:0/4124226150 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:10.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749659c700 1 -- 172.21.15.32:0/4124226150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7498107810 con 0x7f7498106eb0 2024-04-16T05:25:10.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749659c700 1 --2- 172.21.15.32:0/4124226150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f74981072d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7488009e80 tx=0x7f748800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51dbf5d33894290 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749559a700 1 -- 172.21.15.32:0/4124226150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488017070 con 0x7f7498106eb0 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749559a700 1 -- 172.21.15.32:0/4124226150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f748800b8c0 con 0x7f7498106eb0 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.336+0000 7f749559a700 1 -- 172.21.15.32:0/4124226150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74880045c0 con 0x7f7498106eb0 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.337+0000 7f749cbb3700 1 -- 172.21.15.32:0/4124226150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 msgr2=0x7f74981072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.337+0000 7f749cbb3700 1 --2- 172.21.15.32:0/4124226150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f74981072d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7488009e80 tx=0x7f748800b330 comp rx=0 tx=0).stop 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.337+0000 7f749cbb3700 1 -- 172.21.15.32:0/4124226150 shutdown_connections 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.337+0000 7f749cbb3700 1 --2- 172.21.15.32:0/4124226150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f74981072d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.337+0000 7f749cbb3700 1 -- 172.21.15.32:0/4124226150 >> 172.21.15.32:0/4124226150 conn(0x7f7498102a90 msgr2=0x7f7498104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.337+0000 7f749cbb3700 1 -- 172.21.15.32:0/4124226150 shutdown_connections 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.337+0000 7f749cbb3700 1 -- 172.21.15.32:0/4124226150 wait complete. 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.338+0000 7f749cbb3700 1 Processor -- start 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.338+0000 7f749cbb3700 1 -- start start 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.338+0000 7f749cbb3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f749819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.338+0000 7f749cbb3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f74981088a0 con 0x7f7498106eb0 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.338+0000 7f749659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f749819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:10.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.338+0000 7f749659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f749819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41864/0 (socket says 172.21.15.32:41864) 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.338+0000 7f749659c700 1 -- 172.21.15.32:0/473131946 learned_addr learned my addr 172.21.15.32:0/473131946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.339+0000 7f749659c700 1 -- 172.21.15.32:0/473131946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749819ec90 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.339+0000 7f749659c700 1 --2- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f749819e750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7488009e50 tx=0x7f748800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.339+0000 7f748f7fe700 1 -- 172.21.15.32:0/473131946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488020020 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.339+0000 7f748f7fe700 1 -- 172.21.15.32:0/473131946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7488003b50 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.339+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749819ef20 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.339+0000 7f748f7fe700 1 -- 172.21.15.32:0/473131946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488017040 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.339+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74981a1c10 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.340+0000 7f748f7fe700 1 -- 172.21.15.32:0/473131946 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7488023020 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.340+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7498064150 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.340+0000 7f748f7fe700 1 --2- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f74840395b0 0x7f748403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.340+0000 7f748f7fe700 1 -- 172.21.15.32:0/473131946 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f748800f050 con 0x7f7498106eb0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.340+0000 7f7495d9b700 1 --2- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f74840395b0 0x7f748403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:10.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.341+0000 7f7495d9b700 1 --2- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f74840395b0 0x7f748403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7480006fd0 tx=0x7f7480006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.345+0000 7f748f7fe700 1 -- 172.21.15.32:0/473131946 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f748802a020 con 0x7f7498106eb0 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.554+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f74980008d0 con 0x7f74840395b0 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.912+0000 7f748f7fe700 1 -- 172.21.15.32:0/473131946 <== mgr.14120 v2:172.21.15.32:6800/1590295441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f74980008d0 con 0x7f74840395b0 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.914+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f74840395b0 msgr2=0x7f748403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.914+0000 7f749cbb3700 1 --2- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f74840395b0 0x7f748403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7480006fd0 tx=0x7f7480006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.914+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 msgr2=0x7f749819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.914+0000 7f749cbb3700 1 --2- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f749819e750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7488009e50 tx=0x7f748800bd30 comp rx=0 tx=0).stop 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.915+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 shutdown_connections 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.915+0000 7f749cbb3700 1 --2- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f74840395b0 0x7f748403ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.915+0000 7f749cbb3700 1 --2- 172.21.15.32:0/473131946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7498106eb0 0x7f749819e750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.915+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 >> 172.21.15.32:0/473131946 conn(0x7f7498102a90 msgr2=0x7f74981953c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.915+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 shutdown_connections 2024-04-16T05:25:10.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:09.915+0000 7f749cbb3700 1 -- 172.21.15.32:0/473131946 wait complete. 2024-04-16T05:25:11.038 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:10 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:11.038 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:10 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:11.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDECWHLnrRq/LOdaxvkAzZWFljhZMIY9fFpWhy0n7MmYyaMZA3/4KJWSTNkgNKgCZh+9M0uiqNRWQBrBSJIs+6qfy8F2sHMq4tC42a4S8lf44MoJ9iKqXgcvuj0RugziJg0r4HjJt1sGCynXk8lzOepQCGDQRDooZjg6NziAyUlov56ec/jN2ZjH593ioakbUtxEPyt/E6g5YXLWoVUf94h2QX+iPb7Ba5xFHstno/g38vVR+nPmjVGkHnQ9fvkplDSaPohDxXUIXXdhhOktavjpvoxYBgBhscpJ/wgnqdJpKz/HGk0krmkenZNtqNmxGGUI5hfiifoXss5FCcqqH7syDk0gpLKAcm5LL+1TauZt6gYXLmPFFKeKnvN+R4tWQuG7EGEp3hwwyOJJZsgflQQxYTepZxdKYrVVqNXNj1ydr3J4DSDj13SNtUCQ+EA0GJAPgxqOCeNd1z/eIqZSH8TtGAZGaqnzgJgG9yxBlmmRQ/EK4YBr5cm8RH0W9hrZY0= ceph-578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.891+0000 7f64a7f11700 1 Processor -- start 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.892+0000 7f64a7f11700 1 -- start start 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.892+0000 7f64a7f11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a010b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.892+0000 7f64a7f11700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f64a007be50 con 0x7f64a0109080 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.892+0000 7f64a5cad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a010b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.892+0000 7f64a5cad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a010b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41874/0 (socket says 172.21.15.32:41874) 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.892+0000 7f64a5cad700 1 -- 172.21.15.32:0/1976679752 learned_addr learned my addr 172.21.15.32:0/1976679752 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.892+0000 7f64a5cad700 1 -- 172.21.15.32:0/1976679752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a010b9b0 con 0x7f64a0109080 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.892+0000 7f64a5cad700 1 --2- 172.21.15.32:0/1976679752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a010b470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f649c0098a0 tx=0x7f649c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19697d6053a6fb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a4cab700 1 -- 172.21.15.32:0/1976679752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f649c004160 con 0x7f64a0109080 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a4cab700 1 -- 172.21.15.32:0/1976679752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f649c004300 con 0x7f64a0109080 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a4cab700 1 -- 172.21.15.32:0/1976679752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f649c011b50 con 0x7f64a0109080 2024-04-16T05:25:11.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a7f11700 1 -- 172.21.15.32:0/1976679752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 msgr2=0x7f64a010b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a7f11700 1 --2- 172.21.15.32:0/1976679752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a010b470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f649c0098a0 tx=0x7f649c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a7f11700 1 -- 172.21.15.32:0/1976679752 shutdown_connections 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a7f11700 1 --2- 172.21.15.32:0/1976679752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a010b470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a7f11700 1 -- 172.21.15.32:0/1976679752 >> 172.21.15.32:0/1976679752 conn(0x7f64a0102a50 msgr2=0x7f64a0104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a7f11700 1 -- 172.21.15.32:0/1976679752 shutdown_connections 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.893+0000 7f64a7f11700 1 -- 172.21.15.32:0/1976679752 wait complete. 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.894+0000 7f64a7f11700 1 Processor -- start 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.894+0000 7f64a7f11700 1 -- start start 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.894+0000 7f64a7f11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.894+0000 7f64a7f11700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f64a010c980 con 0x7f64a0109080 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.894+0000 7f64a5cad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.894+0000 7f64a5cad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41886/0 (socket says 172.21.15.32:41886) 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.894+0000 7f64a5cad700 1 -- 172.21.15.32:0/1186009152 learned_addr learned my addr 172.21.15.32:0/1186009152 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.895+0000 7f64a5cad700 1 -- 172.21.15.32:0/1186009152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a019ec90 con 0x7f64a0109080 2024-04-16T05:25:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.895+0000 7f64a5cad700 1 --2- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a019e750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f649c011b60 tx=0x7f649c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.895+0000 7f6492ffd700 1 -- 172.21.15.32:0/1186009152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f649c003b20 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.895+0000 7f6492ffd700 1 -- 172.21.15.32:0/1186009152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f649c003cc0 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.895+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a019ef20 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.895+0000 7f6492ffd700 1 -- 172.21.15.32:0/1186009152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f649c024840 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.895+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a01a1c10 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.896+0000 7f6492ffd700 1 -- 172.21.15.32:0/1186009152 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f649c022020 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.896+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a0064150 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.896+0000 7f6492ffd700 1 --2- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f648c0395b0 0x7f648c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.896+0000 7f6492ffd700 1 -- 172.21.15.32:0/1186009152 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f649c011e00 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.896+0000 7f64a54ac700 1 --2- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f648c0395b0 0x7f648c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.896+0000 7f64a54ac700 1 --2- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f648c0395b0 0x7f648c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6494006fd0 tx=0x7f6494006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:10.901+0000 7f6492ffd700 1 -- 172.21.15.32:0/1186009152 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f649c02d3f0 con 0x7f64a0109080 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.114+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f64a0103bf0 con 0x7f648c0395b0 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.115+0000 7f6492ffd700 1 -- 172.21.15.32:0/1186009152 <== mgr.14120 v2:172.21.15.32:6800/1590295441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f64a0103bf0 con 0x7f648c0395b0 2024-04-16T05:25:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f648c0395b0 msgr2=0x7f648c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 --2- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f648c0395b0 0x7f648c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6494006fd0 tx=0x7f6494006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 msgr2=0x7f64a019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 --2- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a019e750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f649c011b60 tx=0x7f649c011c40 comp rx=0 tx=0).stop 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 shutdown_connections 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 --2- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f648c0395b0 0x7f648c03ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 --2- 172.21.15.32:0/1186009152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0109080 0x7f64a019e750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 >> 172.21.15.32:0/1186009152 conn(0x7f64a0102a50 msgr2=0x7f64a01034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 shutdown_connections 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:11.117+0000 7f64a7f11700 1 -- 172.21.15.32:0/1186009152 wait complete. 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:Adding key to root@localhost authorized_keys... 2024-04-16T05:25:11.717 INFO:teuthology.orchestra.run.smithi032.stdout:Adding host smithi032... 2024-04-16T05:25:11.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:11 smithi032 ceph-mon[32351]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:12.945 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:12 smithi032 ceph-mon[32351]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:13 smithi032 ceph-mon[32351]: Deploying cephadm binary to smithi032 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Added host 'smithi032' with addr '172.21.15.32' 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.137+0000 7f02b0db0700 1 Processor -- start 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.137+0000 7f02b0db0700 1 -- start start 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.138+0000 7f02b0db0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac1078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.138+0000 7f02b0db0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac107eb0 con 0x7f02ac1074c0 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.138+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac1078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.138+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac1078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41898/0 (socket says 172.21.15.32:41898) 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.138+0000 7f02aa59c700 1 -- 172.21.15.32:0/3302675154 learned_addr learned my addr 172.21.15.32:0/3302675154 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:17.405 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.138+0000 7f02aa59c700 1 -- 172.21.15.32:0/3302675154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ac1086c0 con 0x7f02ac1074c0 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.138+0000 7f02aa59c700 1 --2- 172.21.15.32:0/3302675154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac1078e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02940098a0 tx=0x7f0294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43be3eb16f13fe3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.138+0000 7f02a959a700 1 -- 172.21.15.32:0/3302675154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0294004160 con 0x7f02ac1074c0 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02a959a700 1 -- 172.21.15.32:0/3302675154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0294004300 con 0x7f02ac1074c0 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02a959a700 1 -- 172.21.15.32:0/3302675154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0294011b50 con 0x7f02ac1074c0 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02b0db0700 1 -- 172.21.15.32:0/3302675154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 msgr2=0x7f02ac1078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02b0db0700 1 --2- 172.21.15.32:0/3302675154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac1078e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02940098a0 tx=0x7f0294009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02b0db0700 1 -- 172.21.15.32:0/3302675154 shutdown_connections 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02b0db0700 1 --2- 172.21.15.32:0/3302675154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac1078e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02b0db0700 1 -- 172.21.15.32:0/3302675154 >> 172.21.15.32:0/3302675154 conn(0x7f02ac102a60 msgr2=0x7f02ac104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02b0db0700 1 -- 172.21.15.32:0/3302675154 shutdown_connections 2024-04-16T05:25:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.139+0000 7f02b0db0700 1 -- 172.21.15.32:0/3302675154 wait complete. 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.140+0000 7f02b0db0700 1 Processor -- start 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.140+0000 7f02b0db0700 1 -- start start 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.140+0000 7f02b0db0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.140+0000 7f02b0db0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac10ba70 con 0x7f02ac1074c0 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.140+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.140+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac19c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41900/0 (socket says 172.21.15.32:41900) 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.140+0000 7f02aa59c700 1 -- 172.21.15.32:0/4208885455 learned_addr learned my addr 172.21.15.32:0/4208885455 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.141+0000 7f02aa59c700 1 -- 172.21.15.32:0/4208885455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ac19cae0 con 0x7f02ac1074c0 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.141+0000 7f02aa59c700 1 --2- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac19c5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0294000c00 tx=0x7f0294011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.141+0000 7f02a37fe700 1 -- 172.21.15.32:0/4208885455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0294003b20 con 0x7f02ac1074c0 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.141+0000 7f02a37fe700 1 -- 172.21.15.32:0/4208885455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0294003cc0 con 0x7f02ac1074c0 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.141+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ac19cd70 con 0x7f02ac1074c0 2024-04-16T05:25:17.407 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.141+0000 7f02a37fe700 1 -- 172.21.15.32:0/4208885455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0294024910 con 0x7f02ac1074c0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.141+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ac19fa60 con 0x7f02ac1074c0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.142+0000 7f02a37fe700 1 -- 172.21.15.32:0/4208885455 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0294022020 con 0x7f02ac1074c0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.142+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ac064150 con 0x7f02ac1074c0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.142+0000 7f02a37fe700 1 --2- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f02980395b0 0x7f029803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.142+0000 7f02a37fe700 1 -- 172.21.15.32:0/4208885455 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0294011e00 con 0x7f02ac1074c0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.142+0000 7f02a9d9b700 1 --2- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f02980395b0 0x7f029803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.143+0000 7f02a9d9b700 1 --2- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f02980395b0 0x7f029803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f029c006fd0 tx=0x7f029c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.147+0000 7f02a37fe700 1 -- 172.21.15.32:0/4208885455 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f029404d1b0 con 0x7f02ac1074c0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:12.358+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}) -- 0x7f02ac02e2c0 con 0x7f02980395b0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.763+0000 7f02a37fe700 1 -- 172.21.15.32:0/4208885455 <== mgr.14120 v2:172.21.15.32:6800/1590295441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f02ac02e2c0 con 0x7f02980395b0 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f02980395b0 msgr2=0x7f029803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 --2- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f02980395b0 0x7f029803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f029c006fd0 tx=0x7f029c006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 msgr2=0x7f02ac19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 --2- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac19c5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0294000c00 tx=0x7f0294011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 shutdown_connections 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 --2- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f02980395b0 0x7f029803ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 --2- 172.21.15.32:0/4208885455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02ac1074c0 0x7f02ac19c5a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 >> 172.21.15.32:0/4208885455 conn(0x7f02ac102a60 msgr2=0x7f02ac1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 shutdown_connections 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:16.766+0000 7f02b0db0700 1 -- 172.21.15.32:0/4208885455 wait complete. 2024-04-16T05:25:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mon service... 2024-04-16T05:25:17.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:17 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:17.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:17 smithi032 ceph-mon[32351]: Added host smithi032 2024-04-16T05:25:17.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:17 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:19.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:19 smithi032 ceph-mon[32351]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:19.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:19 smithi032 ceph-mon[32351]: Saving service mon spec with placement count:5 2024-04-16T05:25:19.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:19 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.898+0000 7f1f8a172700 1 Processor -- start 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.898+0000 7f1f8a172700 1 -- start start 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.899+0000 7f1f8a172700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8410b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.899+0000 7f1f8a172700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8407beb0 con 0x7f1f84109020 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.899+0000 7f1f837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8410b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.899+0000 7f1f837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8410b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41914/0 (socket says 172.21.15.32:41914) 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.899+0000 7f1f837fe700 1 -- 172.21.15.32:0/2342463659 learned_addr learned my addr 172.21.15.32:0/2342463659 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:19.796 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.899+0000 7f1f837fe700 1 -- 172.21.15.32:0/2342463659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8410b950 con 0x7f1f84109020 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.899+0000 7f1f837fe700 1 --2- 172.21.15.32:0/2342463659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8410b410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f740098a0 tx=0x7f1f74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5b9c705d8a3061 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f827fc700 1 -- 172.21.15.32:0/2342463659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f74004160 con 0x7f1f84109020 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f827fc700 1 -- 172.21.15.32:0/2342463659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f74004300 con 0x7f1f84109020 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f827fc700 1 -- 172.21.15.32:0/2342463659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f74011b50 con 0x7f1f84109020 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f8a172700 1 -- 172.21.15.32:0/2342463659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 msgr2=0x7f1f8410b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f8a172700 1 --2- 172.21.15.32:0/2342463659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8410b410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f740098a0 tx=0x7f1f74009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f8a172700 1 -- 172.21.15.32:0/2342463659 shutdown_connections 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f8a172700 1 --2- 172.21.15.32:0/2342463659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8410b410 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f8a172700 1 -- 172.21.15.32:0/2342463659 >> 172.21.15.32:0/2342463659 conn(0x7f1f84102a70 msgr2=0x7f1f84104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f8a172700 1 -- 172.21.15.32:0/2342463659 shutdown_connections 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.900+0000 7f1f8a172700 1 -- 172.21.15.32:0/2342463659 wait complete. 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.901+0000 7f1f8a172700 1 Processor -- start 2024-04-16T05:25:19.797 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.901+0000 7f1f8a172700 1 -- start start 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.901+0000 7f1f8a172700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.901+0000 7f1f8a172700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8410c980 con 0x7f1f84109020 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.901+0000 7f1f837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.901+0000 7f1f837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41922/0 (socket says 172.21.15.32:41922) 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.901+0000 7f1f837fe700 1 -- 172.21.15.32:0/3268546308 learned_addr learned my addr 172.21.15.32:0/3268546308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.902+0000 7f1f837fe700 1 -- 172.21.15.32:0/3268546308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8419ca80 con 0x7f1f84109020 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.902+0000 7f1f837fe700 1 --2- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8419c540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1f74011670 tx=0x7f1f74011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.902+0000 7f1f80ff9700 1 -- 172.21.15.32:0/3268546308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f740039c0 con 0x7f1f84109020 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.902+0000 7f1f80ff9700 1 -- 172.21.15.32:0/3268546308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f74011e50 con 0x7f1f84109020 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.902+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8419cd10 con 0x7f1f84109020 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.902+0000 7f1f80ff9700 1 -- 172.21.15.32:0/3268546308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f7401a5c0 con 0x7f1f84109020 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.902+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8419fa00 con 0x7f1f84109020 2024-04-16T05:25:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.903+0000 7f1f80ff9700 1 -- 172.21.15.32:0/3268546308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1f74024430 con 0x7f1f84109020 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.903+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f84064150 con 0x7f1f84109020 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.903+0000 7f1f80ff9700 1 --2- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f1f6c0395b0 0x7f1f6c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.903+0000 7f1f80ff9700 1 -- 172.21.15.32:0/3268546308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f7404dab0 con 0x7f1f84109020 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.903+0000 7f1f82ffd700 1 --2- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f1f6c0395b0 0x7f1f6c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.904+0000 7f1f82ffd700 1 --2- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f1f6c0395b0 0x7f1f6c03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f78006fd0 tx=0x7f1f78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:17.908+0000 7f1f80ff9700 1 -- 172.21.15.32:0/3268546308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f74024720 con 0x7f1f84109020 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.134+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1f84103c70 con 0x7f1f6c0395b0 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.139+0000 7f1f80ff9700 1 -- 172.21.15.32:0/3268546308 <== mgr.14120 v2:172.21.15.32:6800/1590295441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1f84103c70 con 0x7f1f6c0395b0 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f1f6c0395b0 msgr2=0x7f1f6c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 --2- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f1f6c0395b0 0x7f1f6c03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f78006fd0 tx=0x7f1f78006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 msgr2=0x7f1f8419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 --2- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8419c540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1f74011670 tx=0x7f1f74011b60 comp rx=0 tx=0).stop 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 shutdown_connections 2024-04-16T05:25:19.799 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 --2- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f1f6c0395b0 0x7f1f6c03ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:19.800 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 --2- 172.21.15.32:0/3268546308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f84109020 0x7f1f8419c540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:19.800 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 >> 172.21.15.32:0/3268546308 conn(0x7f1f84102a70 msgr2=0x7f1f84103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:19.800 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 shutdown_connections 2024-04-16T05:25:19.800 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:18.141+0000 7f1f8a172700 1 -- 172.21.15.32:0/3268546308 wait complete. 2024-04-16T05:25:19.800 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mgr service... 2024-04-16T05:25:20.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:20 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:20.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:20 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:21.043 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-16T05:25:21.043 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.239+0000 7fd7c793f700 1 Processor -- start 2024-04-16T05:25:21.043 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.239+0000 7fd7c793f700 1 -- start start 2024-04-16T05:25:21.043 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.239+0000 7fd7c793f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c010b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.239+0000 7fd7c793f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c007bdf0 con 0x7fd7c0109100 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.240+0000 7fd7c56db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c010b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.240+0000 7fd7c56db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c010b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48422/0 (socket says 172.21.15.32:48422) 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.240+0000 7fd7c56db700 1 -- 172.21.15.32:0/4203081866 learned_addr learned my addr 172.21.15.32:0/4203081866 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.240+0000 7fd7c56db700 1 -- 172.21.15.32:0/4203081866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c010ba30 con 0x7fd7c0109100 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.240+0000 7fd7c56db700 1 --2- 172.21.15.32:0/4203081866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c010b4f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098a0 tx=0x7fd7bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2eb13b1c30b304b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.240+0000 7fd7b3fff700 1 -- 172.21.15.32:0/4203081866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc004160 con 0x7fd7c0109100 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.240+0000 7fd7b3fff700 1 -- 172.21.15.32:0/4203081866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7bc004300 con 0x7fd7c0109100 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.240+0000 7fd7b3fff700 1 -- 172.21.15.32:0/4203081866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc011b20 con 0x7fd7c0109100 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.241+0000 7fd7c793f700 1 -- 172.21.15.32:0/4203081866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 msgr2=0x7fd7c010b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.241+0000 7fd7c793f700 1 --2- 172.21.15.32:0/4203081866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c010b4f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098a0 tx=0x7fd7bc009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.241+0000 7fd7c793f700 1 -- 172.21.15.32:0/4203081866 shutdown_connections 2024-04-16T05:25:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.241+0000 7fd7c793f700 1 --2- 172.21.15.32:0/4203081866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c010b4f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.241+0000 7fd7c793f700 1 -- 172.21.15.32:0/4203081866 >> 172.21.15.32:0/4203081866 conn(0x7fd7c0102a90 msgr2=0x7fd7c0104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.241+0000 7fd7c793f700 1 -- 172.21.15.32:0/4203081866 shutdown_connections 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.241+0000 7fd7c793f700 1 -- 172.21.15.32:0/4203081866 wait complete. 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.242+0000 7fd7c793f700 1 Processor -- start 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.242+0000 7fd7c793f700 1 -- start start 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.242+0000 7fd7c793f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c0108980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.242+0000 7fd7c793f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c010c9e0 con 0x7fd7c0109100 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.242+0000 7fd7c56db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c0108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.242+0000 7fd7c56db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c0108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48434/0 (socket says 172.21.15.32:48434) 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.242+0000 7fd7c56db700 1 -- 172.21.15.32:0/3325615051 learned_addr learned my addr 172.21.15.32:0/3325615051 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.242+0000 7fd7c56db700 1 -- 172.21.15.32:0/3325615051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c0108ec0 con 0x7fd7c0109100 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.243+0000 7fd7c56db700 1 --2- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c0108980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0116a0 tx=0x7fd7bc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.243+0000 7fd7b27fc700 1 -- 172.21.15.32:0/3325615051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc003a40 con 0x7fd7c0109100 2024-04-16T05:25:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.243+0000 7fd7b27fc700 1 -- 172.21.15.32:0/3325615051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7bc003be0 con 0x7fd7c0109100 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.243+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c01070b0 con 0x7fd7c0109100 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.243+0000 7fd7b27fc700 1 -- 172.21.15.32:0/3325615051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc01a750 con 0x7fd7c0109100 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.243+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c0107570 con 0x7fd7c0109100 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.244+0000 7fd7b27fc700 1 -- 172.21.15.32:0/3325615051 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd7bc01aa00 con 0x7fd7c0109100 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.244+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c0064150 con 0x7fd7c0109100 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.244+0000 7fd7b27fc700 1 --2- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fd7ac0395b0 0x7fd7ac03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.244+0000 7fd7b27fc700 1 -- 172.21.15.32:0/3325615051 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd7bc04dc90 con 0x7fd7c0109100 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.244+0000 7fd7c4eda700 1 --2- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fd7ac0395b0 0x7fd7ac03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.245+0000 7fd7c4eda700 1 --2- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fd7ac0395b0 0x7fd7ac03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006fd0 tx=0x7fd7b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.249+0000 7fd7b27fc700 1 -- 172.21.15.32:0/3325615051 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7bc011e20 con 0x7fd7c0109100 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.473+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd7c0107e80 con 0x7fd7ac0395b0 2024-04-16T05:25:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.477+0000 7fd7b27fc700 1 -- 172.21.15.32:0/3325615051 <== mgr.14120 v2:172.21.15.32:6800/1590295441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd7c0107e80 con 0x7fd7ac0395b0 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fd7ac0395b0 msgr2=0x7fd7ac03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 --2- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fd7ac0395b0 0x7fd7ac03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006fd0 tx=0x7fd7b4006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 msgr2=0x7fd7c0108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 --2- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c0108980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0116a0 tx=0x7fd7bc011ba0 comp rx=0 tx=0).stop 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 shutdown_connections 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 --2- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fd7ac0395b0 0x7fd7ac03ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 --2- 172.21.15.32:0/3325615051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7c0109100 0x7fd7c0108980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 >> 172.21.15.32:0/3325615051 conn(0x7fd7c0102a90 msgr2=0x7fd7c01035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 shutdown_connections 2024-04-16T05:25:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:20.480+0000 7fd7c793f700 1 -- 172.21.15.32:0/3325615051 wait complete. 2024-04-16T05:25:21.723 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:21 smithi032 ceph-mon[32351]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:21.724 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:21 smithi032 ceph-mon[32351]: Saving service mgr spec with placement count:2 2024-04-16T05:25:21.724 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:21 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.517+0000 7f6e63a95700 1 Processor -- start 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.518+0000 7f6e63a95700 1 -- start start 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.518+0000 7f6e63a95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c10a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.518+0000 7f6e63a95700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c10a600 con 0x7f6e5c109c10 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.518+0000 7f6e61831700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c10a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.518+0000 7f6e61831700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c10a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48440/0 (socket says 172.21.15.32:48440) 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.518+0000 7f6e61831700 1 -- 172.21.15.32:0/3094708985 learned_addr learned my addr 172.21.15.32:0/3094708985 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.518+0000 7f6e61831700 1 -- 172.21.15.32:0/3094708985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5c10a780 con 0x7f6e5c109c10 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.518+0000 7f6e61831700 1 --2- 172.21.15.32:0/3094708985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c10a030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6e580098a0 tx=0x7f6e58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ee65489cbe0ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e6082f700 1 -- 172.21.15.32:0/3094708985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e58004160 con 0x7f6e5c109c10 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e6082f700 1 -- 172.21.15.32:0/3094708985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e58004300 con 0x7f6e5c109c10 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e6082f700 1 -- 172.21.15.32:0/3094708985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e58011b20 con 0x7f6e5c109c10 2024-04-16T05:25:22.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e63a95700 1 -- 172.21.15.32:0/3094708985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 msgr2=0x7f6e5c10a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e63a95700 1 --2- 172.21.15.32:0/3094708985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c10a030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6e580098a0 tx=0x7f6e58009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e63a95700 1 -- 172.21.15.32:0/3094708985 shutdown_connections 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e63a95700 1 --2- 172.21.15.32:0/3094708985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c10a030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e63a95700 1 -- 172.21.15.32:0/3094708985 >> 172.21.15.32:0/3094708985 conn(0x7f6e5c1021f0 msgr2=0x7f6e5c104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e63a95700 1 -- 172.21.15.32:0/3094708985 shutdown_connections 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.519+0000 7f6e63a95700 1 -- 172.21.15.32:0/3094708985 wait complete. 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.520+0000 7f6e63a95700 1 Processor -- start 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.520+0000 7f6e63a95700 1 -- start start 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.520+0000 7f6e63a95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.520+0000 7f6e63a95700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c10cbe0 con 0x7f6e5c109c10 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.520+0000 7f6e61831700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.520+0000 7f6e61831700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48456/0 (socket says 172.21.15.32:48456) 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.520+0000 7f6e61831700 1 -- 172.21.15.32:0/2576075446 learned_addr learned my addr 172.21.15.32:0/2576075446 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.521+0000 7f6e61831700 1 -- 172.21.15.32:0/2576075446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5c19caa0 con 0x7f6e5c109c10 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.521+0000 7f6e61831700 1 --2- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c19c560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e580116a0 tx=0x7f6e58011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.521+0000 7f6e52ffd700 1 -- 172.21.15.32:0/2576075446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e58003a40 con 0x7f6e5c109c10 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.521+0000 7f6e52ffd700 1 -- 172.21.15.32:0/2576075446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e58003be0 con 0x7f6e5c109c10 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.521+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5c19cd90 con 0x7f6e5c109c10 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.521+0000 7f6e52ffd700 1 -- 172.21.15.32:0/2576075446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5801a720 con 0x7f6e5c109c10 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.521+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e5c19fa20 con 0x7f6e5c109c10 2024-04-16T05:25:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.522+0000 7f6e52ffd700 1 -- 172.21.15.32:0/2576075446 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6e5801a9d0 con 0x7f6e5c109c10 2024-04-16T05:25:22.748 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.522+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e5c064150 con 0x7f6e5c109c10 2024-04-16T05:25:22.748 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.522+0000 7f6e52ffd700 1 --2- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f6e480395b0 0x7f6e4803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:22.748 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.523+0000 7f6e52ffd700 1 -- 172.21.15.32:0/2576075446 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6e5804dba0 con 0x7f6e5c109c10 2024-04-16T05:25:22.748 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.523+0000 7f6e61030700 1 --2- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f6e480395b0 0x7f6e4803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:22.748 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.523+0000 7f6e61030700 1 --2- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f6e480395b0 0x7f6e4803ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006fd0 tx=0x7f6e4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:22.748 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.528+0000 7f6e52ffd700 1 -- 172.21.15.32:0/2576075446 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e58011e20 con 0x7f6e5c109c10 2024-04-16T05:25:22.748 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.749+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6e5c19ffa0 con 0x7f6e5c109c10 2024-04-16T05:25:22.749 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.752+0000 7f6e52ffd700 1 -- 172.21.15.32:0/2576075446 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6e58019c70 con 0x7f6e5c109c10 2024-04-16T05:25:22.749 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.755+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f6e480395b0 msgr2=0x7f6e4803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:22.749 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.755+0000 7f6e63a95700 1 --2- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f6e480395b0 0x7f6e4803ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006fd0 tx=0x7f6e4c006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:22.749 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.755+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 msgr2=0x7f6e5c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:22.749 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.755+0000 7f6e63a95700 1 --2- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c19c560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e580116a0 tx=0x7f6e58011ba0 comp rx=0 tx=0).stop 2024-04-16T05:25:22.749 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.756+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 shutdown_connections 2024-04-16T05:25:22.749 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.756+0000 7f6e63a95700 1 --2- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f6e480395b0 0x7f6e4803ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:22.750 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.756+0000 7f6e63a95700 1 --2- 172.21.15.32:0/2576075446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5c109c10 0x7f6e5c19c560 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:22.750 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.756+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 >> 172.21.15.32:0/2576075446 conn(0x7f6e5c1021f0 msgr2=0x7f6e5c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:22.750 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.756+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 shutdown_connections 2024-04-16T05:25:22.750 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:21.756+0000 7f6e63a95700 1 -- 172.21.15.32:0/2576075446 wait complete. 2024-04-16T05:25:22.988 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:22 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2576075446' entity='client.admin' 2024-04-16T05:25:22.988 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:22 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:23.975 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.208+0000 7f0cee72a700 1 Processor -- start 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.209+0000 7f0cee72a700 1 -- start start 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.209+0000 7f0cee72a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce810b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.209+0000 7f0cee72a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce807be70 con 0x7f0ce8109070 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.209+0000 7f0ce7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce810b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.209+0000 7f0ce7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce810b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48472/0 (socket says 172.21.15.32:48472) 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.209+0000 7f0ce7fff700 1 -- 172.21.15.32:0/76453641 learned_addr learned my addr 172.21.15.32:0/76453641 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.209+0000 7f0ce7fff700 1 -- 172.21.15.32:0/76453641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce810b9a0 con 0x7f0ce8109070 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.210+0000 7f0ce7fff700 1 --2- 172.21.15.32:0/76453641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce810b460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80098a0 tx=0x7f0cd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1552085ca3fe17c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.210+0000 7f0ce6ffd700 1 -- 172.21.15.32:0/76453641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd8004160 con 0x7f0ce8109070 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.210+0000 7f0ce6ffd700 1 -- 172.21.15.32:0/76453641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8004300 con 0x7f0ce8109070 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.210+0000 7f0ce6ffd700 1 -- 172.21.15.32:0/76453641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd8011b20 con 0x7f0ce8109070 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.210+0000 7f0cee72a700 1 -- 172.21.15.32:0/76453641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 msgr2=0x7f0ce810b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.210+0000 7f0cee72a700 1 --2- 172.21.15.32:0/76453641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce810b460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80098a0 tx=0x7f0cd8009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.210+0000 7f0cee72a700 1 -- 172.21.15.32:0/76453641 shutdown_connections 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.210+0000 7f0cee72a700 1 --2- 172.21.15.32:0/76453641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce810b460 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.211+0000 7f0cee72a700 1 -- 172.21.15.32:0/76453641 >> 172.21.15.32:0/76453641 conn(0x7f0ce8102a60 msgr2=0x7f0ce8104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.211+0000 7f0cee72a700 1 -- 172.21.15.32:0/76453641 shutdown_connections 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.211+0000 7f0cee72a700 1 -- 172.21.15.32:0/76453641 wait complete. 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.211+0000 7f0cee72a700 1 Processor -- start 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.211+0000 7f0cee72a700 1 -- start start 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0cee72a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0cee72a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810ca00 con 0x7f0ce8109070 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0ce7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0ce7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48486/0 (socket says 172.21.15.32:48486) 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0ce7fff700 1 -- 172.21.15.32:0/1040960888 learned_addr learned my addr 172.21.15.32:0/1040960888 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0ce7fff700 1 -- 172.21.15.32:0/1040960888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce819ca30 con 0x7f0ce8109070 2024-04-16T05:25:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0ce7fff700 1 --2- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce819c4f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80117e0 tx=0x7f0cd8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0ce57fa700 1 -- 172.21.15.32:0/1040960888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd8003bf0 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0ce57fa700 1 -- 172.21.15.32:0/1040960888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd801a910 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce819ccc0 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.212+0000 7f0ce57fa700 1 -- 172.21.15.32:0/1040960888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd8024970 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.213+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce819d120 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.214+0000 7f0ce57fa700 1 -- 172.21.15.32:0/1040960888 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0cd8022020 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.214+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce8064150 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.214+0000 7f0ce57fa700 1 --2- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f0cd0039560 0x7f0cd003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.214+0000 7f0ce57fa700 1 -- 172.21.15.32:0/1040960888 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0cd804da90 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.214+0000 7f0ce77fe700 1 --2- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f0cd0039560 0x7f0cd003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.214+0000 7f0ce77fe700 1 --2- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f0cd0039560 0x7f0cd003ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc006fd0 tx=0x7f0cdc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.219+0000 7f0ce57fa700 1 -- 172.21.15.32:0/1040960888 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cd8011e20 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.426+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0ce819faf0 con 0x7f0ce8109070 2024-04-16T05:25:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.429+0000 7f0ce57fa700 1 -- 172.21.15.32:0/1040960888 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0cd80172b0 con 0x7f0ce8109070 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.431+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f0cd0039560 msgr2=0x7f0cd003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.431+0000 7f0cee72a700 1 --2- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f0cd0039560 0x7f0cd003ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc006fd0 tx=0x7f0cdc006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.431+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 msgr2=0x7f0ce819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.431+0000 7f0cee72a700 1 --2- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce819c4f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80117e0 tx=0x7f0cd8011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.431+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 shutdown_connections 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.431+0000 7f0cee72a700 1 --2- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f0cd0039560 0x7f0cd003ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.431+0000 7f0cee72a700 1 --2- 172.21.15.32:0/1040960888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce8109070 0x7f0ce819c4f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.431+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 >> 172.21.15.32:0/1040960888 conn(0x7f0ce8102a60 msgr2=0x7f0ce81035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.432+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 shutdown_connections 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:23.432+0000 7f0cee72a700 1 -- 172.21.15.32:0/1040960888 wait complete. 2024-04-16T05:25:23.979 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling the dashboard module... 2024-04-16T05:25:24.663 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:24 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1040960888' entity='client.admin' 2024-04-16T05:25:24.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:24 smithi032 ceph-mon[32351]: from='mgr.14120 172.21.15.32:0/456837402' entity='mgr.x' 2024-04-16T05:25:25.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:25 smithi032 ceph-mon[32351]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:25:25.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:25 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2738550868' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-16T05:25:25.983 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:25 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: ignoring --setuser ceph since I am not root 2024-04-16T05:25:25.983 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:25 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: ignoring --setgroup ceph since I am not root 2024-04-16T05:25:26.296 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.401+0000 7f7804e14700 1 Processor -- start 2024-04-16T05:25:26.296 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.401+0000 7f7804e14700 1 -- start start 2024-04-16T05:25:26.296 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.401+0000 7f7804e14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f7800108ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.401+0000 7f7804e14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78001094b0 con 0x7f7800108ac0 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.402+0000 7f77fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f7800108ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.402+0000 7f77fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f7800108ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48494/0 (socket says 172.21.15.32:48494) 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.402+0000 7f77fe59c700 1 -- 172.21.15.32:0/510065649 learned_addr learned my addr 172.21.15.32:0/510065649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.402+0000 7f77fe59c700 1 -- 172.21.15.32:0/510065649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7800107720 con 0x7f7800108ac0 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.402+0000 7f77fe59c700 1 --2- 172.21.15.32:0/510065649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f7800108ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009e80 tx=0x7f77f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d30493a8597716 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.402+0000 7f77fd59a700 1 -- 172.21.15.32:0/510065649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f0017070 con 0x7f7800108ac0 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.402+0000 7f77fd59a700 1 -- 172.21.15.32:0/510065649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f000b8c0 con 0x7f7800108ac0 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.402+0000 7f77fd59a700 1 -- 172.21.15.32:0/510065649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f00045c0 con 0x7f7800108ac0 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.403+0000 7f7804e14700 1 -- 172.21.15.32:0/510065649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 msgr2=0x7f7800108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.403+0000 7f7804e14700 1 --2- 172.21.15.32:0/510065649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f7800108ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009e80 tx=0x7f77f000b330 comp rx=0 tx=0).stop 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.403+0000 7f7804e14700 1 -- 172.21.15.32:0/510065649 shutdown_connections 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.403+0000 7f7804e14700 1 --2- 172.21.15.32:0/510065649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f7800108ee0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.403+0000 7f7804e14700 1 -- 172.21.15.32:0/510065649 >> 172.21.15.32:0/510065649 conn(0x7f78001021f0 msgr2=0x7f7800104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:26.297 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.403+0000 7f7804e14700 1 -- 172.21.15.32:0/510065649 shutdown_connections 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.403+0000 7f7804e14700 1 -- 172.21.15.32:0/510065649 wait complete. 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.404+0000 7f7804e14700 1 Processor -- start 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.404+0000 7f7804e14700 1 -- start start 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.404+0000 7f7804e14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f780019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.404+0000 7f7804e14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f780010bae0 con 0x7f7800108ac0 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.404+0000 7f77fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f780019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.404+0000 7f77fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f780019c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48506/0 (socket says 172.21.15.32:48506) 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.404+0000 7f77fe59c700 1 -- 172.21.15.32:0/2738550868 learned_addr learned my addr 172.21.15.32:0/2738550868 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.404+0000 7f77fe59c700 1 -- 172.21.15.32:0/2738550868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780019cb40 con 0x7f7800108ac0 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.405+0000 7f77fe59c700 1 --2- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f780019c600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77f0003fa0 tx=0x7f77f0004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.405+0000 7f77f77fe700 1 -- 172.21.15.32:0/2738550868 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f0020020 con 0x7f7800108ac0 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.405+0000 7f77f77fe700 1 -- 172.21.15.32:0/2738550868 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f000bba0 con 0x7f7800108ac0 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.405+0000 7f77f77fe700 1 -- 172.21.15.32:0/2738550868 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f0017040 con 0x7f7800108ac0 2024-04-16T05:25:26.298 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.405+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780019cdd0 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.405+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780019fac0 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.406+0000 7f77f77fe700 1 -- 172.21.15.32:0/2738550868 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f77f0023020 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.406+0000 7f77f77fe700 1 --2- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f77ec0395b0 0x7f77ec03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.406+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7800064150 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.406+0000 7f77f77fe700 1 -- 172.21.15.32:0/2738550868 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f77f000f050 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.406+0000 7f77fdd9b700 1 --2- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f77ec0395b0 0x7f77ec03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.406+0000 7f77fdd9b700 1 --2- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f77ec0395b0 0x7f77ec03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77e8006fd0 tx=0x7f77e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.414+0000 7f77f77fe700 1 -- 172.21.15.32:0/2738550868 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77f002a020 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:24.693+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f780002e160 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.682+0000 7f77f77fe700 1 -- 172.21.15.32:0/2738550868 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f77f001b770 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.684+0000 7f77f77fe700 1 -- 172.21.15.32:0/2738550868 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f77f0025400 con 0x7f7800108ac0 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.686+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f77ec0395b0 msgr2=0x7f77ec03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.686+0000 7f7804e14700 1 --2- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f77ec0395b0 0x7f77ec03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77e8006fd0 tx=0x7f77e8006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.686+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 msgr2=0x7f780019c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:26.299 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.686+0000 7f7804e14700 1 --2- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f780019c600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77f0003fa0 tx=0x7f77f0004080 comp rx=0 tx=0).stop 2024-04-16T05:25:26.300 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.687+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 shutdown_connections 2024-04-16T05:25:26.300 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.687+0000 7f7804e14700 1 --2- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f77ec0395b0 0x7f77ec03ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.300 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.687+0000 7f7804e14700 1 --2- 172.21.15.32:0/2738550868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7800108ac0 0x7f780019c600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:26.300 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.687+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 >> 172.21.15.32:0/2738550868 conn(0x7f78001021f0 msgr2=0x7f7800103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:26.300 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.687+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 shutdown_connections 2024-04-16T05:25:26.300 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:25.687+0000 7f7804e14700 1 -- 172.21.15.32:0/2738550868 wait complete. 2024-04-16T05:25:26.541 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:26 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:26.280+0000 7f7f70eed200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:25:26.542 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:26 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:26.382+0000 7f7f70eed200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:25:26.932 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:26 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2738550868' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-16T05:25:26.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:26 smithi032 ceph-mon[32351]: mgrmap e9: x(active, since 21s) 2024-04-16T05:25:27.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:27.064+0000 7f7f70eed200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:25:27.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:27.159+0000 7f7f70eed200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:25:27.596 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:27.306+0000 7f7f70eed200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:25:27.596 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:27.405+0000 7f7f70eed200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:25:27.596 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:27.492+0000 7f7f70eed200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:25:27.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:27.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-16T05:25:27.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T05:25:27.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.728+0000 7fddb07d3700 1 Processor -- start 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.728+0000 7fddb07d3700 1 -- start start 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.728+0000 7fddb07d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.728+0000 7fddb07d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdda807c8f0 con 0x7fdda807de70 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.728+0000 7fddae56f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.728+0000 7fddae56f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48528/0 (socket says 172.21.15.32:48528) 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.728+0000 7fddae56f700 1 -- 172.21.15.32:0/3904739798 learned_addr learned my addr 172.21.15.32:0/3904739798 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.729+0000 7fddae56f700 1 -- 172.21.15.32:0/3904739798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda807ca70 con 0x7fdda807de70 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.729+0000 7fddae56f700 1 --2- 172.21.15.32:0/3904739798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda807c320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdda40098a0 tx=0x7fdda4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b889fea9753723fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.729+0000 7fddad56d700 1 -- 172.21.15.32:0/3904739798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdda4004160 con 0x7fdda807de70 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.729+0000 7fddad56d700 1 -- 172.21.15.32:0/3904739798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda4004300 con 0x7fdda807de70 2024-04-16T05:25:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.729+0000 7fddad56d700 1 -- 172.21.15.32:0/3904739798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdda4011b20 con 0x7fdda807de70 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 -- 172.21.15.32:0/3904739798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 msgr2=0x7fdda807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 --2- 172.21.15.32:0/3904739798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda807c320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdda40098a0 tx=0x7fdda4009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 -- 172.21.15.32:0/3904739798 shutdown_connections 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 --2- 172.21.15.32:0/3904739798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda807c320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 -- 172.21.15.32:0/3904739798 >> 172.21.15.32:0/3904739798 conn(0x7fdda8103b80 msgr2=0x7fdda8105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 -- 172.21.15.32:0/3904739798 shutdown_connections 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 -- 172.21.15.32:0/3904739798 wait complete. 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 Processor -- start 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.730+0000 7fddb07d3700 1 -- start start 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fddb07d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fddb07d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdda810ebc0 con 0x7fdda807de70 2024-04-16T05:25:27.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fddae56f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fddae56f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48530/0 (socket says 172.21.15.32:48530) 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fddae56f700 1 -- 172.21.15.32:0/3483061783 learned_addr learned my addr 172.21.15.32:0/3483061783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fddae56f700 1 -- 172.21.15.32:0/3483061783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda819ec70 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fddae56f700 1 --2- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda819e730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdda40116a0 tx=0x7fdda4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fdd9f7fe700 1 -- 172.21.15.32:0/3483061783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdda4003a40 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fdd9f7fe700 1 -- 172.21.15.32:0/3483061783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda4003be0 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fdd9f7fe700 1 -- 172.21.15.32:0/3483061783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdda401a750 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.731+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda819ef00 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.732+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda819f360 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.732+0000 7fdd9f7fe700 1 -- 172.21.15.32:0/3483061783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fdda4024430 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.732+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda8064150 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.733+0000 7fdd9f7fe700 1 --2- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fdd940395b0 0x7fdd9403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.733+0000 7fdd9f7fe700 1 -- 172.21.15.32:0/3483061783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdda404ddc0 con 0x7fdda807de70 2024-04-16T05:25:27.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.733+0000 7fddadd6e700 1 -- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fdd940395b0 msgr2=0x7fdd9403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1590295441 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.733+0000 7fddadd6e700 1 --2- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fdd940395b0 0x7fdd9403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.738+0000 7fdd9f7fe700 1 -- 172.21.15.32:0/3483061783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdda4011e20 con 0x7fdda807de70 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.934+0000 7fddadd6e700 1 -- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fdd940395b0 msgr2=0x7fdd9403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1590295441 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:26.934+0000 7fddadd6e700 1 --2- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fdd940395b0 0x7fdd9403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.006+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdda81a1d10 con 0x7fdda807de70 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.006+0000 7fdd9f7fe700 1 -- 172.21.15.32:0/3483061783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fdda402d3f0 con 0x7fdda807de70 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fdd940395b0 msgr2=0x7fdd9403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 --2- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fdd940395b0 0x7fdd9403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 msgr2=0x7fdda819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 --2- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda819e730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdda40116a0 tx=0x7fdda4011ba0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 shutdown_connections 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 --2- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7fdd940395b0 0x7fdd9403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 --2- 172.21.15.32:0/3483061783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdda807de70 0x7fdda819e730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:27.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 >> 172.21.15.32:0/3483061783 conn(0x7fdda8103b80 msgr2=0x7fdda81046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:27.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 shutdown_connections 2024-04-16T05:25:27.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:27.009+0000 7fddb07d3700 1 -- 172.21.15.32:0/3483061783 wait complete. 2024-04-16T05:25:27.609 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-04-16T05:25:27.609 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 9... 2024-04-16T05:25:27.885 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3483061783' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-16T05:25:27.885 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:27.770+0000 7f7f70eed200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:25:27.885 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:27.886+0000 7f7f70eed200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:28.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:27.978+0000 7f7f70eed200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:25:28.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:28 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:28.078+0000 7f7f70eed200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:25:28.640 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:28 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:28.260+0000 7f7f70eed200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:25:29.511 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:29 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:29.260+0000 7f7f70eed200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:25:29.511 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:29 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:29.399+0000 7f7f70eed200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:25:29.845 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:29 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:29.752+0000 7f7f70eed200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:25:29.845 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:29 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:29.846+0000 7f7f70eed200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:25:30.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:30 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:30.056+0000 7f7f70eed200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:30.549 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:30 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:30.251+0000 7f7f70eed200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:25:30.813 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:30 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:30.551+0000 7f7f70eed200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:25:30.813 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:30 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:30.814+0000 7f7f70eed200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:25:31.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:31 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:31.062+0000 7f7f70eed200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:25:32.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:31 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:31.900+0000 7f7f70eed200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:25:32.813 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:32.487+0000 7f7f70eed200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:25:32.813 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:32.588+0000 7f7f70eed200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:25:32.813 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:25:32.814+0000 7f7f70eed200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:25:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: Active manager daemon x restarted 2024-04-16T05:25:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: Activating manager daemon x 2024-04-16T05:25:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: osdmap e3: 0 total, 0 up, 0 in 2024-04-16T05:25:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: mgrmap e10: x(active, starting, since 0.00335058s) 2024-04-16T05:25:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:25:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:25:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:25:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:25:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:25:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: Manager daemon x is now available 2024-04-16T05:25:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:25:34.212 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:25:34.212 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:33 smithi032 ceph-mon[32351]: [16/Apr/2024:05:25:33] ENGINE Bus STARTING 2024-04-16T05:25:34.212 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:33 smithi032 ceph-mon[32351]: mgrmap e11: x(active, since 1.0073s) 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.035+0000 7f50eb43b700 1 Processor -- start 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.035+0000 7f50eb43b700 1 -- start start 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.035+0000 7f50eb43b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e410b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.035+0000 7f50eb43b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50e407be70 con 0x7f50e4109070 2024-04-16T05:25:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.035+0000 7f50e91d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e410b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.035+0000 7f50e91d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e410b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48540/0 (socket says 172.21.15.32:48540) 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.035+0000 7f50e91d7700 1 -- 172.21.15.32:0/2585586399 learned_addr learned my addr 172.21.15.32:0/2585586399 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.036+0000 7f50e91d7700 1 -- 172.21.15.32:0/2585586399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e410b9a0 con 0x7f50e4109070 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.036+0000 7f50e91d7700 1 --2- 172.21.15.32:0/2585586399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e410b460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50e00098a0 tx=0x7f50e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81710bd8c3bf8666 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.036+0000 7f50dbfff700 1 -- 172.21.15.32:0/2585586399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e0004160 con 0x7f50e4109070 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.036+0000 7f50dbfff700 1 -- 172.21.15.32:0/2585586399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e0004300 con 0x7f50e4109070 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.036+0000 7f50dbfff700 1 -- 172.21.15.32:0/2585586399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e0011b20 con 0x7f50e4109070 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.037+0000 7f50eb43b700 1 -- 172.21.15.32:0/2585586399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 msgr2=0x7f50e410b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.037+0000 7f50eb43b700 1 --2- 172.21.15.32:0/2585586399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e410b460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50e00098a0 tx=0x7f50e0009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.037+0000 7f50eb43b700 1 -- 172.21.15.32:0/2585586399 shutdown_connections 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.037+0000 7f50eb43b700 1 --2- 172.21.15.32:0/2585586399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e410b460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.037+0000 7f50eb43b700 1 -- 172.21.15.32:0/2585586399 >> 172.21.15.32:0/2585586399 conn(0x7f50e4102a60 msgr2=0x7f50e4104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.037+0000 7f50eb43b700 1 -- 172.21.15.32:0/2585586399 shutdown_connections 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.037+0000 7f50eb43b700 1 -- 172.21.15.32:0/2585586399 wait complete. 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.037+0000 7f50eb43b700 1 Processor -- start 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50eb43b700 1 -- start start 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50eb43b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50eb43b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50e410ca00 con 0x7f50e4109070 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50e91d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50e91d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48546/0 (socket says 172.21.15.32:48546) 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50e91d7700 1 -- 172.21.15.32:0/3319706511 learned_addr learned my addr 172.21.15.32:0/3319706511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50e91d7700 1 -- 172.21.15.32:0/3319706511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e419ca10 con 0x7f50e4109070 2024-04-16T05:25:34.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50e91d7700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e419c4d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50e00117e0 tx=0x7f50e0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e0003bf0 con 0x7f50e4109070 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e001a910 con 0x7f50e4109070 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e419cca0 con 0x7f50e4109070 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.038+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e0024970 con 0x7f50e4109070 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.039+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e419d100 con 0x7f50e4109070 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.039+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f50e0022020 con 0x7f50e4109070 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.040+0000 7f50da7fc700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 0x7f50d003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.040+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 --> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f50d003c180 con 0x7f50d00395b0 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.040+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f50e004da90 con 0x7f50e4109070 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.040+0000 7f50e89d6700 1 -- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 msgr2=0x7f50d003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1590295441 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.040+0000 7f50e89d6700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 0x7f50d003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.241+0000 7f50e89d6700 1 -- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 msgr2=0x7f50d003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1590295441 2024-04-16T05:25:34.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.241+0000 7f50e89d6700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 0x7f50d003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.642+0000 7f50e89d6700 1 -- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 msgr2=0x7f50d003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1590295441 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:28.642+0000 7f50e89d6700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 0x7f50d003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.443+0000 7f50e89d6700 1 -- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 msgr2=0x7f50d003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1590295441 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:29.443+0000 7f50e89d6700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 0x7f50d003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.045+0000 7f50e89d6700 1 -- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 msgr2=0x7f50d003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1590295441 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:31.045+0000 7f50e89d6700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 0x7f50d003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:32.818+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f50e004d050 con 0x7f50e4109070 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:32.818+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 msgr2=0x7f50d003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:32.818+0000 7f50da7fc700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 0x7f50d003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.820+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f50e000ec80 con 0x7f50e4109070 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.820+0000 7f50da7fc700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50d003cfe0 0x7f50d003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.820+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f50e0014e50 con 0x7f50d003cfe0 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.821+0000 7f50e89d6700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50d003cfe0 0x7f50d003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:34.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.822+0000 7f50e89d6700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50d003cfe0 0x7f50d003f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50d4003a10 tx=0x7f50d40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.822+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f50e0014e50 con 0x7f50d003cfe0 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.830+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f50e4064150 con 0x7f50d003cfe0 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50da7fc700 1 -- 172.21.15.32:0/3319706511 <== mgr.14152 v2:172.21.15.32:6800/2875882657 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f50e4064150 con 0x7f50d003cfe0 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50d003cfe0 msgr2=0x7f50d003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50d003cfe0 0x7f50d003f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50d4003a10 tx=0x7f50d40092b0 comp rx=0 tx=0).stop 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 msgr2=0x7f50e419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e419c4d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50e00117e0 tx=0x7f50e0011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 shutdown_connections 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e4109070 0x7f50e419c4d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50d003cfe0 0x7f50d003f3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 --2- 172.21.15.32:0/3319706511 >> [v2:172.21.15.32:6800/1590295441,v1:172.21.15.32:6801/1590295441] conn(0x7f50d00395b0 0x7f50d003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 >> 172.21.15.32:0/3319706511 conn(0x7f50e4102a60 msgr2=0x7f50e4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 shutdown_connections 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:33.831+0000 7f50eb43b700 1 -- 172.21.15.32:0/3319706511 wait complete. 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 9 is available 2024-04-16T05:25:34.496 INFO:teuthology.orchestra.run.smithi032.stdout:Generating a dashboard self-signed certificate... 2024-04-16T05:25:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:34 smithi032 ceph-mon[32351]: [16/Apr/2024:05:25:33] ENGINE Serving on http://172.21.15.32:8765 2024-04-16T05:25:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:34 smithi032 ceph-mon[32351]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-16T05:25:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:34 smithi032 ceph-mon[32351]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-16T05:25:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:34 smithi032 ceph-mon[32351]: [16/Apr/2024:05:25:33] ENGINE Serving on https://172.21.15.32:7150 2024-04-16T05:25:34.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:34 smithi032 ceph-mon[32351]: [16/Apr/2024:05:25:33] ENGINE Bus STARTED 2024-04-16T05:25:35.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:35 smithi032 ceph-mon[32351]: mgrmap e12: x(active, since 2s) 2024-04-16T05:25:35.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:35 smithi032 ceph-mon[32351]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:35.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:35.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:35.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.942+0000 7fb8c4a7b700 1 Processor -- start 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.942+0000 7fb8c4a7b700 1 -- start start 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.942+0000 7fb8c4a7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c010b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.942+0000 7fb8c4a7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c007be70 con 0x7fb8c0109080 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.943+0000 7fb8be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c010b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.943+0000 7fb8be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c010b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37098/0 (socket says 172.21.15.32:37098) 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.943+0000 7fb8be59c700 1 -- 172.21.15.32:0/554798844 learned_addr learned my addr 172.21.15.32:0/554798844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.943+0000 7fb8be59c700 1 -- 172.21.15.32:0/554798844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c010b9b0 con 0x7fb8c0109080 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.943+0000 7fb8be59c700 1 --2- 172.21.15.32:0/554798844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c010b470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00098a0 tx=0x7fb8b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3597a72218986b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.943+0000 7fb8bd59a700 1 -- 172.21.15.32:0/554798844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8b0004160 con 0x7fb8c0109080 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.943+0000 7fb8bd59a700 1 -- 172.21.15.32:0/554798844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b0004300 con 0x7fb8c0109080 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.943+0000 7fb8bd59a700 1 -- 172.21.15.32:0/554798844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8b0011b20 con 0x7fb8c0109080 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.944+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/554798844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 msgr2=0x7fb8c010b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.944+0000 7fb8c4a7b700 1 --2- 172.21.15.32:0/554798844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c010b470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00098a0 tx=0x7fb8b0009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.944+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/554798844 shutdown_connections 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.944+0000 7fb8c4a7b700 1 --2- 172.21.15.32:0/554798844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c010b470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.944+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/554798844 >> 172.21.15.32:0/554798844 conn(0x7fb8c0102a70 msgr2=0x7fb8c0104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.944+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/554798844 shutdown_connections 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.944+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/554798844 wait complete. 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.945+0000 7fb8c4a7b700 1 Processor -- start 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.945+0000 7fb8c4a7b700 1 -- start start 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.945+0000 7fb8c4a7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.945+0000 7fb8c4a7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c010ca10 con 0x7fb8c0109080 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.945+0000 7fb8be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.945+0000 7fb8be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37108/0 (socket says 172.21.15.32:37108) 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.945+0000 7fb8be59c700 1 -- 172.21.15.32:0/1065206376 learned_addr learned my addr 172.21.15.32:0/1065206376 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.945+0000 7fb8be59c700 1 -- 172.21.15.32:0/1065206376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c019c9f0 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.946+0000 7fb8be59c700 1 --2- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c019c4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00117e0 tx=0x7fb8b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.946+0000 7fb8af7fe700 1 -- 172.21.15.32:0/1065206376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8b0003bf0 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.946+0000 7fb8af7fe700 1 -- 172.21.15.32:0/1065206376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b001a910 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.946+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c019cc80 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.946+0000 7fb8af7fe700 1 -- 172.21.15.32:0/1065206376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8b0024970 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.946+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c019d0e0 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.947+0000 7fb8af7fe700 1 -- 172.21.15.32:0/1065206376 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fb8b0022020 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.947+0000 7fb8af7fe700 1 --2- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb8a80395b0 0x7fb8a803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.947+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c0064150 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.947+0000 7fb8af7fe700 1 -- 172.21.15.32:0/1065206376 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb8b004db80 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.947+0000 7fb8bdd9b700 1 --2- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb8a80395b0 0x7fb8a803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.947+0000 7fb8bdd9b700 1 --2- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb8a80395b0 0x7fb8a803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4006fd0 tx=0x7fb8b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:34.952+0000 7fb8af7fe700 1 -- 172.21.15.32:0/1065206376 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8b0011e20 con 0x7fb8c0109080 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.167+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb8c0103c20 con 0x7fb8a80395b0 2024-04-16T05:25:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.291+0000 7fb8af7fe700 1 -- 172.21.15.32:0/1065206376 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb8c0103c20 con 0x7fb8a80395b0 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb8a80395b0 msgr2=0x7fb8a803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 --2- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb8a80395b0 0x7fb8a803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4006fd0 tx=0x7fb8b4006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 msgr2=0x7fb8c019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 --2- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c019c4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00117e0 tx=0x7fb8b0011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 shutdown_connections 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 --2- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb8a80395b0 0x7fb8a803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 --2- 172.21.15.32:0/1065206376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c0109080 0x7fb8c019c4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 >> 172.21.15.32:0/1065206376 conn(0x7fb8c0102a70 msgr2=0x7fb8c0103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 shutdown_connections 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:35.292+0000 7fb8c4a7b700 1 -- 172.21.15.32:0/1065206376 wait complete. 2024-04-16T05:25:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial admin user... 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jNkxAzzhTQa4lmt.35twIeyUzSLhkxbJHVGau5hN19gmWvPHGIqqa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713245136, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.358+0000 7fc42eea2700 1 Processor -- start 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.358+0000 7fc42eea2700 1 -- start start 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.358+0000 7fc42eea2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc4281078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.358+0000 7fc42eea2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc428107ea0 con 0x7fc4281074b0 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.358+0000 7fc42cc3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc4281078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.358+0000 7fc42cc3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc4281078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37118/0 (socket says 172.21.15.32:37118) 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.358+0000 7fc42cc3e700 1 -- 172.21.15.32:0/2722320343 learned_addr learned my addr 172.21.15.32:0/2722320343 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.359+0000 7fc42cc3e700 1 -- 172.21.15.32:0/2722320343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4281086b0 con 0x7fc4281074b0 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.359+0000 7fc42cc3e700 1 --2- 172.21.15.32:0/2722320343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc4281078d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0098a0 tx=0x7fc41c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882507474d6fc6b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.359+0000 7fc4277fe700 1 -- 172.21.15.32:0/2722320343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41c004160 con 0x7fc4281074b0 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.359+0000 7fc4277fe700 1 -- 172.21.15.32:0/2722320343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c004300 con 0x7fc4281074b0 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.359+0000 7fc4277fe700 1 -- 172.21.15.32:0/2722320343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41c011b20 con 0x7fc4281074b0 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 -- 172.21.15.32:0/2722320343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 msgr2=0x7fc4281078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 --2- 172.21.15.32:0/2722320343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc4281078d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0098a0 tx=0x7fc41c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 -- 172.21.15.32:0/2722320343 shutdown_connections 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 --2- 172.21.15.32:0/2722320343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc4281078d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 -- 172.21.15.32:0/2722320343 >> 172.21.15.32:0/2722320343 conn(0x7fc428102a70 msgr2=0x7fc428104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 -- 172.21.15.32:0/2722320343 shutdown_connections 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 -- 172.21.15.32:0/2722320343 wait complete. 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 Processor -- start 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.360+0000 7fc42eea2700 1 -- start start 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc42eea2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc42819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc42eea2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc42807afd0 con 0x7fc4281074b0 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc42cc3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc42819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc42cc3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc42819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37130/0 (socket says 172.21.15.32:37130) 2024-04-16T05:25:37.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc42cc3e700 1 -- 172.21.15.32:0/1291434636 learned_addr learned my addr 172.21.15.32:0/1291434636 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc42cc3e700 1 -- 172.21.15.32:0/1291434636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42819a830 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc42cc3e700 1 --2- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc42819a2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0117e0 tx=0x7fc41c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc425ffb700 1 -- 172.21.15.32:0/1291434636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41c003bf0 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc425ffb700 1 -- 172.21.15.32:0/1291434636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c01a910 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42819aac0 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.361+0000 7fc425ffb700 1 -- 172.21.15.32:0/1291434636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41c0249a0 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.362+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42819af20 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.362+0000 7fc425ffb700 1 -- 172.21.15.32:0/1291434636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc41c022020 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.363+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc428064150 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.363+0000 7fc425ffb700 1 --2- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc41803b7e0 0x7fc41803dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.363+0000 7fc425ffb700 1 -- 172.21.15.32:0/1291434636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc41c04dcb0 con 0x7fc4281074b0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.363+0000 7fc427fff700 1 --2- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc41803b7e0 0x7fc41803dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:37.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.363+0000 7fc427fff700 1 --2- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc41803b7e0 0x7fc41803dca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc414006fd0 tx=0x7fc414006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.368+0000 7fc425ffb700 1 -- 172.21.15.32:0/1291434636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc41c053020 con 0x7fc4281074b0 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.613+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc42819d930 con 0x7fc41803b7e0 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.858+0000 7fc425ffb700 1 -- 172.21.15.32:0/1291434636 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc42819d930 con 0x7fc41803b7e0 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc41803b7e0 msgr2=0x7fc41803dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 --2- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc41803b7e0 0x7fc41803dca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc414006fd0 tx=0x7fc414006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 msgr2=0x7fc42819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 --2- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc42819a2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0117e0 tx=0x7fc41c011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 shutdown_connections 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 --2- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc41803b7e0 0x7fc41803dca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 --2- 172.21.15.32:0/1291434636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4281074b0 0x7fc42819a2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 >> 172.21.15.32:0/1291434636 conn(0x7fc428102a70 msgr2=0x7fc428190930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 shutdown_connections 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:36.862+0000 7fc42eea2700 1 -- 172.21.15.32:0/1291434636 wait complete. 2024-04-16T05:25:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:Fetching dashboard port number... 2024-04-16T05:25:37.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:37.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:37 smithi032 ceph-mon[32351]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:37.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:38.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:38 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1625392592' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-16T05:25:39.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 8443 2024-04-16T05:25:39.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.863+0000 7fe219eb9700 1 Processor -- start 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.863+0000 7fe219eb9700 1 -- start start 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.863+0000 7fe219eb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe2141078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.863+0000 7fe219eb9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe214107eb0 con 0x7fe2141074c0 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.864+0000 7fe2137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe2141078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.864+0000 7fe2137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe2141078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37136/0 (socket says 172.21.15.32:37136) 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.864+0000 7fe2137fe700 1 -- 172.21.15.32:0/1368260661 learned_addr learned my addr 172.21.15.32:0/1368260661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.864+0000 7fe2137fe700 1 -- 172.21.15.32:0/1368260661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2141086c0 con 0x7fe2141074c0 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.864+0000 7fe2137fe700 1 --2- 172.21.15.32:0/1368260661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe2141078e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098a0 tx=0x7fe204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e60d672d5c3ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.864+0000 7fe2127fc700 1 -- 172.21.15.32:0/1368260661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe204004160 con 0x7fe2141074c0 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.864+0000 7fe2127fc700 1 -- 172.21.15.32:0/1368260661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe204004300 con 0x7fe2141074c0 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.864+0000 7fe2127fc700 1 -- 172.21.15.32:0/1368260661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe204011b20 con 0x7fe2141074c0 2024-04-16T05:25:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.865+0000 7fe219eb9700 1 -- 172.21.15.32:0/1368260661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 msgr2=0x7fe2141078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.865+0000 7fe219eb9700 1 --2- 172.21.15.32:0/1368260661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe2141078e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098a0 tx=0x7fe204009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.865+0000 7fe219eb9700 1 -- 172.21.15.32:0/1368260661 shutdown_connections 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.865+0000 7fe219eb9700 1 --2- 172.21.15.32:0/1368260661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe2141078e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.865+0000 7fe219eb9700 1 -- 172.21.15.32:0/1368260661 >> 172.21.15.32:0/1368260661 conn(0x7fe214102a60 msgr2=0x7fe214104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.865+0000 7fe219eb9700 1 -- 172.21.15.32:0/1368260661 shutdown_connections 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.865+0000 7fe219eb9700 1 -- 172.21.15.32:0/1368260661 wait complete. 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.866+0000 7fe219eb9700 1 Processor -- start 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.866+0000 7fe219eb9700 1 -- start start 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.866+0000 7fe219eb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe21419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.866+0000 7fe219eb9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe21410ba70 con 0x7fe2141074c0 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.866+0000 7fe2137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe21419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.866+0000 7fe2137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe21419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37138/0 (socket says 172.21.15.32:37138) 2024-04-16T05:25:39.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.866+0000 7fe2137fe700 1 -- 172.21.15.32:0/1625392592 learned_addr learned my addr 172.21.15.32:0/1625392592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.866+0000 7fe2137fe700 1 -- 172.21.15.32:0/1625392592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21419ca80 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.867+0000 7fe2137fe700 1 --2- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe21419c540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2040117e0 tx=0x7fe204011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.867+0000 7fe210ff9700 1 -- 172.21.15.32:0/1625392592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe204003bf0 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.867+0000 7fe210ff9700 1 -- 172.21.15.32:0/1625392592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20401a910 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.867+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21419cd10 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.867+0000 7fe210ff9700 1 -- 172.21.15.32:0/1625392592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe204024970 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.867+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21419d170 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.868+0000 7fe210ff9700 1 -- 172.21.15.32:0/1625392592 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fe204022020 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.868+0000 7fe210ff9700 1 --2- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe200039600 0x7fe20003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.868+0000 7fe210ff9700 1 -- 172.21.15.32:0/1625392592 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe20404efa0 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.868+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe214064150 con 0x7fe2141074c0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.868+0000 7fe212ffd700 1 --2- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe200039600 0x7fe20003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.869+0000 7fe212ffd700 1 --2- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe200039600 0x7fe20003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006fd0 tx=0x7fe1fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:39.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:37.873+0000 7fe210ff9700 1 -- 172.21.15.32:0/1625392592 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe20402cdf0 con 0x7fe2141074c0 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.089+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fe214063700 con 0x7fe2141074c0 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.089+0000 7fe210ff9700 1 -- 172.21.15.32:0/1625392592 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fe204020c10 con 0x7fe2141074c0 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe200039600 msgr2=0x7fe20003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 --2- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe200039600 0x7fe20003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006fd0 tx=0x7fe1fc006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 msgr2=0x7fe21419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 --2- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe21419c540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2040117e0 tx=0x7fe204011ba0 comp rx=0 tx=0).stop 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 shutdown_connections 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 --2- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe200039600 0x7fe20003bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 --2- 172.21.15.32:0/1625392592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2141074c0 0x7fe21419c540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 >> 172.21.15.32:0/1625392592 conn(0x7fe214102a60 msgr2=0x7fe2141038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 shutdown_connections 2024-04-16T05:25:39.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:38.091+0000 7fe219eb9700 1 -- 172.21.15.32:0/1625392592 wait complete. 2024-04-16T05:25:39.673 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T05:25:39.674 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-04-16T05:25:39.682 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T05:25:39.682 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-04-16T05:25:39.682 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-04-16T05:25:39.683 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-16T05:25:39.683 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph Dashboard is now available at: 2024-04-16T05:25:39.684 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:39.684 INFO:teuthology.orchestra.run.smithi032.stdout: URL: https://smithi032.front.sepia.ceph.com:8443/ 2024-04-16T05:25:39.684 INFO:teuthology.orchestra.run.smithi032.stdout: User: admin 2024-04-16T05:25:39.684 INFO:teuthology.orchestra.run.smithi032.stdout: Password: pa7jxnency 2024-04-16T05:25:39.684 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:39.684 INFO:teuthology.orchestra.run.smithi032.stdout:Saving cluster configuration to /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config directory 2024-04-16T05:25:39.685 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling autotune for osd_memory_target 2024-04-16T05:25:40.631 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:40 smithi032 ceph-mon[32351]: mgrmap e13: x(active, since 6s) 2024-04-16T05:25:40.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1441874700 1 Processor -- start 2024-04-16T05:25:40.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1441874700 1 -- start start 2024-04-16T05:25:40.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1441874700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c10b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:40.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1441874700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f143c07be30 con 0x7f143c1090a0 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f143affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c10b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f143affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c10b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42428/0 (socket says 172.21.15.32:42428) 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f143affd700 1 -- 172.21.15.32:0/2719190934 learned_addr learned my addr 172.21.15.32:0/2719190934 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f143affd700 1 -- 172.21.15.32:0/2719190934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c10b9d0 con 0x7f143c1090a0 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f143affd700 1 --2- 172.21.15.32:0/2719190934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c10b490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f142c0098a0 tx=0x7f142c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5387cb1fa23331e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1439ffb700 1 -- 172.21.15.32:0/2719190934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f142c004160 con 0x7f143c1090a0 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1439ffb700 1 -- 172.21.15.32:0/2719190934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c004300 con 0x7f143c1090a0 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1439ffb700 1 -- 172.21.15.32:0/2719190934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f142c011b20 con 0x7f143c1090a0 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1441874700 1 -- 172.21.15.32:0/2719190934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 msgr2=0x7f143c10b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.095+0000 7f1441874700 1 --2- 172.21.15.32:0/2719190934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c10b490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f142c0098a0 tx=0x7f142c009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.096+0000 7f1441874700 1 -- 172.21.15.32:0/2719190934 shutdown_connections 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.096+0000 7f1441874700 1 --2- 172.21.15.32:0/2719190934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c10b490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.096+0000 7f1441874700 1 -- 172.21.15.32:0/2719190934 >> 172.21.15.32:0/2719190934 conn(0x7f143c102a70 msgr2=0x7f143c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.096+0000 7f1441874700 1 -- 172.21.15.32:0/2719190934 shutdown_connections 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.096+0000 7f1441874700 1 -- 172.21.15.32:0/2719190934 wait complete. 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.096+0000 7f1441874700 1 Processor -- start 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.096+0000 7f1441874700 1 -- start start 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f1441874700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f1441874700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f143c10c9d0 con 0x7f143c1090a0 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f143affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f143affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42440/0 (socket says 172.21.15.32:42440) 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f143affd700 1 -- 172.21.15.32:0/3137051639 learned_addr learned my addr 172.21.15.32:0/3137051639 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f143affd700 1 -- 172.21.15.32:0/3137051639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c19ec90 con 0x7f143c1090a0 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f143affd700 1 --2- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c19e750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f142c0117e0 tx=0x7f142c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f1440872700 1 -- 172.21.15.32:0/3137051639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f142c003bf0 con 0x7f143c1090a0 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f1440872700 1 -- 172.21.15.32:0/3137051639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c01a910 con 0x7f143c1090a0 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c19ef20 con 0x7f143c1090a0 2024-04-16T05:25:40.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f1440872700 1 -- 172.21.15.32:0/3137051639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f142c024970 con 0x7f143c1090a0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.097+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c19f380 con 0x7f143c1090a0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.098+0000 7f1440872700 1 -- 172.21.15.32:0/3137051639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f142c022020 con 0x7f143c1090a0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.098+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f143c064150 con 0x7f143c1090a0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.098+0000 7f1440872700 1 --2- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f14240396b0 0x7f142403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.098+0000 7f1440872700 1 -- 172.21.15.32:0/3137051639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f142c04dd50 con 0x7f143c1090a0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.098+0000 7f143a7fc700 1 --2- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f14240396b0 0x7f142403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.099+0000 7f143a7fc700 1 --2- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f14240396b0 0x7f142403bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1430006fd0 tx=0x7f1430006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.104+0000 7f1440872700 1 -- 172.21.15.32:0/3137051639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f142c011e20 con 0x7f143c1090a0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.307+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f143c1a1db0 con 0x7f143c1090a0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.308+0000 7f1440872700 1 -- 172.21.15.32:0/3137051639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f142c00bc70 con 0x7f143c1090a0 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f14240396b0 msgr2=0x7f142403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 --2- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f14240396b0 0x7f142403bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1430006fd0 tx=0x7f1430006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 msgr2=0x7f143c19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 --2- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c19e750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f142c0117e0 tx=0x7f142c011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:40.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 shutdown_connections 2024-04-16T05:25:40.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 --2- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f14240396b0 0x7f142403bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 --2- 172.21.15.32:0/3137051639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c1090a0 0x7f143c19e750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:40.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 >> 172.21.15.32:0/3137051639 conn(0x7f143c102a70 msgr2=0x7f143c103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:40.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 shutdown_connections 2024-04-16T05:25:40.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:40.311+0000 7f1441874700 1 -- 172.21.15.32:0/3137051639 wait complete. 2024-04-16T05:25:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.321+0000 7fc68d123700 1 Processor -- start 2024-04-16T05:25:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.321+0000 7fc68d123700 1 -- start start 2024-04-16T05:25:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc68d123700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68810b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc68d123700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc68807be50 con 0x7fc688109080 2024-04-16T05:25:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc686d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68810b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc686d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68810b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42454/0 (socket says 172.21.15.32:42454) 2024-04-16T05:25:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc686d9d700 1 -- 172.21.15.32:0/325845674 learned_addr learned my addr 172.21.15.32:0/325845674 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc686d9d700 1 -- 172.21.15.32:0/325845674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68810b9b0 con 0x7fc688109080 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc686d9d700 1 --2- 172.21.15.32:0/325845674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68810b470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6780098a0 tx=0x7fc678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67250b05c55ab23 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc685d9b700 1 -- 172.21.15.32:0/325845674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc678004160 con 0x7fc688109080 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc685d9b700 1 -- 172.21.15.32:0/325845674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc678004300 con 0x7fc688109080 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.322+0000 7fc685d9b700 1 -- 172.21.15.32:0/325845674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc678011b20 con 0x7fc688109080 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.323+0000 7fc68d123700 1 -- 172.21.15.32:0/325845674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 msgr2=0x7fc68810b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.323+0000 7fc68d123700 1 --2- 172.21.15.32:0/325845674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68810b470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6780098a0 tx=0x7fc678009bb0 comp rx=0 tx=0).stop 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.323+0000 7fc68d123700 1 -- 172.21.15.32:0/325845674 shutdown_connections 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.323+0000 7fc68d123700 1 --2- 172.21.15.32:0/325845674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68810b470 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.323+0000 7fc68d123700 1 -- 172.21.15.32:0/325845674 >> 172.21.15.32:0/325845674 conn(0x7fc688102a70 msgr2=0x7fc688104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.323+0000 7fc68d123700 1 -- 172.21.15.32:0/325845674 shutdown_connections 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.323+0000 7fc68d123700 1 -- 172.21.15.32:0/325845674 wait complete. 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.324+0000 7fc68d123700 1 Processor -- start 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.324+0000 7fc68d123700 1 -- start start 2024-04-16T05:25:42.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.324+0000 7fc68d123700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.324+0000 7fc68d123700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc68810c9c0 con 0x7fc688109080 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.324+0000 7fc686d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.324+0000 7fc686d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42456/0 (socket says 172.21.15.32:42456) 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.324+0000 7fc686d9d700 1 -- 172.21.15.32:0/1954539229 learned_addr learned my addr 172.21.15.32:0/1954539229 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.325+0000 7fc686d9d700 1 -- 172.21.15.32:0/1954539229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68819ec00 con 0x7fc688109080 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.325+0000 7fc686d9d700 1 --2- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68819e6c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6780117e0 tx=0x7fc678011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.325+0000 7fc66ffff700 1 -- 172.21.15.32:0/1954539229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc678003bf0 con 0x7fc688109080 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.325+0000 7fc66ffff700 1 -- 172.21.15.32:0/1954539229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67801a910 con 0x7fc688109080 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.325+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68819ee90 con 0x7fc688109080 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.325+0000 7fc66ffff700 1 -- 172.21.15.32:0/1954539229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc678024970 con 0x7fc688109080 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.325+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68819f2f0 con 0x7fc688109080 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.326+0000 7fc66ffff700 1 -- 172.21.15.32:0/1954539229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc678022020 con 0x7fc688109080 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.326+0000 7fc66ffff700 1 --2- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc670039650 0x7fc67003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:42.294 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.326+0000 7fc66ffff700 1 -- 172.21.15.32:0/1954539229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc67801aac0 con 0x7fc688109080 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.326+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc688064150 con 0x7fc688109080 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.326+0000 7fc68659c700 1 --2- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc670039650 0x7fc67003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.327+0000 7fc68659c700 1 --2- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc670039650 0x7fc67003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc67c006fd0 tx=0x7fc67c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.331+0000 7fc66ffff700 1 -- 172.21.15.32:0/1954539229 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc678011e20 con 0x7fc688109080 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.655+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc6881a1cc0 con 0x7fc688109080 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.657+0000 7fc66ffff700 1 -- 172.21.15.32:0/1954539229 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc678032030 con 0x7fc688109080 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc670039650 msgr2=0x7fc67003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 --2- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc670039650 0x7fc67003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc67c006fd0 tx=0x7fc67c006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 msgr2=0x7fc68819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 --2- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68819e6c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6780117e0 tx=0x7fc678011c10 comp rx=0 tx=0).stop 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 shutdown_connections 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 --2- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc670039650 0x7fc67003bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:42.295 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 --2- 172.21.15.32:0/1954539229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc688109080 0x7fc68819e6c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 >> 172.21.15.32:0/1954539229 conn(0x7fc688102a70 msgr2=0x7fc6881034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 shutdown_connections 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-04-16T05:25:41.659+0000 7fc68d123700 1 -- 172.21.15.32:0/1954539229 wait complete. 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:Or, if you are only running a single cluster on this host: 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-16T05:25:42.296 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout: ceph telemetry on 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout:For more information see: 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:42.297 INFO:teuthology.orchestra.run.smithi032.stdout:Bootstrap complete. 2024-04-16T05:25:42.332 INFO:tasks.cephadm:Fetching config... 2024-04-16T05:25:42.332 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:25:42.332 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-16T05:25:42.348 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-16T05:25:42.349 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:25:42.349 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-16T05:25:42.405 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-16T05:25:42.405 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:25:42.405 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-16T05:25:42.473 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-16T05:25:42.474 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:25:42.474 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-16T05:25:42.529 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-16T05:25:42.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDECWHLnrRq/LOdaxvkAzZWFljhZMIY9fFpWhy0n7MmYyaMZA3/4KJWSTNkgNKgCZh+9M0uiqNRWQBrBSJIs+6qfy8F2sHMq4tC42a4S8lf44MoJ9iKqXgcvuj0RugziJg0r4HjJt1sGCynXk8lzOepQCGDQRDooZjg6NziAyUlov56ec/jN2ZjH593ioakbUtxEPyt/E6g5YXLWoVUf94h2QX+iPb7Ba5xFHstno/g38vVR+nPmjVGkHnQ9fvkplDSaPohDxXUIXXdhhOktavjpvoxYBgBhscpJ/wgnqdJpKz/HGk0krmkenZNtqNmxGGUI5hfiifoXss5FCcqqH7syDk0gpLKAcm5LL+1TauZt6gYXLmPFFKeKnvN+R4tWQuG7EGEp3hwwyOJJZsgflQQxYTepZxdKYrVVqNXNj1ydr3J4DSDj13SNtUCQ+EA0GJAPgxqOCeNd1z/eIqZSH8TtGAZGaqnzgJgG9yxBlmmRQ/EK4YBr5cm8RH0W9hrZY0= ceph-578203a8-fbb1-11ee-bc8f-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T05:25:42.624 INFO:teuthology.orchestra.run.smithi032.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDECWHLnrRq/LOdaxvkAzZWFljhZMIY9fFpWhy0n7MmYyaMZA3/4KJWSTNkgNKgCZh+9M0uiqNRWQBrBSJIs+6qfy8F2sHMq4tC42a4S8lf44MoJ9iKqXgcvuj0RugziJg0r4HjJt1sGCynXk8lzOepQCGDQRDooZjg6NziAyUlov56ec/jN2ZjH593ioakbUtxEPyt/E6g5YXLWoVUf94h2QX+iPb7Ba5xFHstno/g38vVR+nPmjVGkHnQ9fvkplDSaPohDxXUIXXdhhOktavjpvoxYBgBhscpJ/wgnqdJpKz/HGk0krmkenZNtqNmxGGUI5hfiifoXss5FCcqqH7syDk0gpLKAcm5LL+1TauZt6gYXLmPFFKeKnvN+R4tWQuG7EGEp3hwwyOJJZsgflQQxYTepZxdKYrVVqNXNj1ydr3J4DSDj13SNtUCQ+EA0GJAPgxqOCeNd1z/eIqZSH8TtGAZGaqnzgJgG9yxBlmmRQ/EK4YBr5cm8RH0W9hrZY0= ceph-578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:42.640 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDECWHLnrRq/LOdaxvkAzZWFljhZMIY9fFpWhy0n7MmYyaMZA3/4KJWSTNkgNKgCZh+9M0uiqNRWQBrBSJIs+6qfy8F2sHMq4tC42a4S8lf44MoJ9iKqXgcvuj0RugziJg0r4HjJt1sGCynXk8lzOepQCGDQRDooZjg6NziAyUlov56ec/jN2ZjH593ioakbUtxEPyt/E6g5YXLWoVUf94h2QX+iPb7Ba5xFHstno/g38vVR+nPmjVGkHnQ9fvkplDSaPohDxXUIXXdhhOktavjpvoxYBgBhscpJ/wgnqdJpKz/HGk0krmkenZNtqNmxGGUI5hfiifoXss5FCcqqH7syDk0gpLKAcm5LL+1TauZt6gYXLmPFFKeKnvN+R4tWQuG7EGEp3hwwyOJJZsgflQQxYTepZxdKYrVVqNXNj1ydr3J4DSDj13SNtUCQ+EA0GJAPgxqOCeNd1z/eIqZSH8TtGAZGaqnzgJgG9yxBlmmRQ/EK4YBr5cm8RH0W9hrZY0= ceph-578203a8-fbb1-11ee-bc8f-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T05:25:42.687 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDECWHLnrRq/LOdaxvkAzZWFljhZMIY9fFpWhy0n7MmYyaMZA3/4KJWSTNkgNKgCZh+9M0uiqNRWQBrBSJIs+6qfy8F2sHMq4tC42a4S8lf44MoJ9iKqXgcvuj0RugziJg0r4HjJt1sGCynXk8lzOepQCGDQRDooZjg6NziAyUlov56ec/jN2ZjH593ioakbUtxEPyt/E6g5YXLWoVUf94h2QX+iPb7Ba5xFHstno/g38vVR+nPmjVGkHnQ9fvkplDSaPohDxXUIXXdhhOktavjpvoxYBgBhscpJ/wgnqdJpKz/HGk0krmkenZNtqNmxGGUI5hfiifoXss5FCcqqH7syDk0gpLKAcm5LL+1TauZt6gYXLmPFFKeKnvN+R4tWQuG7EGEp3hwwyOJJZsgflQQxYTepZxdKYrVVqNXNj1ydr3J4DSDj13SNtUCQ+EA0GJAPgxqOCeNd1z/eIqZSH8TtGAZGaqnzgJgG9yxBlmmRQ/EK4YBr5cm8RH0W9hrZY0= ceph-578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:42.702 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDECWHLnrRq/LOdaxvkAzZWFljhZMIY9fFpWhy0n7MmYyaMZA3/4KJWSTNkgNKgCZh+9M0uiqNRWQBrBSJIs+6qfy8F2sHMq4tC42a4S8lf44MoJ9iKqXgcvuj0RugziJg0r4HjJt1sGCynXk8lzOepQCGDQRDooZjg6NziAyUlov56ec/jN2ZjH593ioakbUtxEPyt/E6g5YXLWoVUf94h2QX+iPb7Ba5xFHstno/g38vVR+nPmjVGkHnQ9fvkplDSaPohDxXUIXXdhhOktavjpvoxYBgBhscpJ/wgnqdJpKz/HGk0krmkenZNtqNmxGGUI5hfiifoXss5FCcqqH7syDk0gpLKAcm5LL+1TauZt6gYXLmPFFKeKnvN+R4tWQuG7EGEp3hwwyOJJZsgflQQxYTepZxdKYrVVqNXNj1ydr3J4DSDj13SNtUCQ+EA0GJAPgxqOCeNd1z/eIqZSH8TtGAZGaqnzgJgG9yxBlmmRQ/EK4YBr5cm8RH0W9hrZY0= ceph-578203a8-fbb1-11ee-bc8f-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T05:25:42.750 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDECWHLnrRq/LOdaxvkAzZWFljhZMIY9fFpWhy0n7MmYyaMZA3/4KJWSTNkgNKgCZh+9M0uiqNRWQBrBSJIs+6qfy8F2sHMq4tC42a4S8lf44MoJ9iKqXgcvuj0RugziJg0r4HjJt1sGCynXk8lzOepQCGDQRDooZjg6NziAyUlov56ec/jN2ZjH593ioakbUtxEPyt/E6g5YXLWoVUf94h2QX+iPb7Ba5xFHstno/g38vVR+nPmjVGkHnQ9fvkplDSaPohDxXUIXXdhhOktavjpvoxYBgBhscpJ/wgnqdJpKz/HGk0krmkenZNtqNmxGGUI5hfiifoXss5FCcqqH7syDk0gpLKAcm5LL+1TauZt6gYXLmPFFKeKnvN+R4tWQuG7EGEp3hwwyOJJZsgflQQxYTepZxdKYrVVqNXNj1ydr3J4DSDj13SNtUCQ+EA0GJAPgxqOCeNd1z/eIqZSH8TtGAZGaqnzgJgG9yxBlmmRQ/EK4YBr5cm8RH0W9hrZY0= ceph-578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:25:42.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-16T05:25:42.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:42 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1954539229' entity='client.admin' 2024-04-16T05:25:43.009 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:25:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.734+0000 7f6e0a127700 1 -- 172.21.15.32:0/1605101375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 msgr2=0x7f6e04104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.734+0000 7f6e0a127700 1 --2- 172.21.15.32:0/1605101375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 0x7f6e04104080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009910 tx=0x7f6df4009c20 comp rx=0 tx=0).stop 2024-04-16T05:25:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 -- 172.21.15.32:0/1605101375 shutdown_connections 2024-04-16T05:25:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 --2- 172.21.15.32:0/1605101375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 0x7f6e04104080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 -- 172.21.15.32:0/1605101375 >> 172.21.15.32:0/1605101375 conn(0x7f6e040fdb40 msgr2=0x7f6e040fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 -- 172.21.15.32:0/1605101375 shutdown_connections 2024-04-16T05:25:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 -- 172.21.15.32:0/1605101375 wait complete. 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 Processor -- start 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 -- start start 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 0x7f6e04197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.735+0000 7f6e0a127700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04108720 con 0x7f6e04103c60 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 0x7f6e04197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 0x7f6e04197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42480/0 (socket says 172.21.15.32:42480) 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e037fe700 1 -- 172.21.15.32:0/121359956 learned_addr learned my addr 172.21.15.32:0/121359956 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e037fe700 1 -- 172.21.15.32:0/121359956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e04198320 con 0x7f6e04103c60 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e037fe700 1 --2- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 0x7f6e04197de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005850 tx=0x7f6df4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e00ff9700 1 -- 172.21.15.32:0/121359956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4026070 con 0x7f6e04103c60 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e00ff9700 1 -- 172.21.15.32:0/121359956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df40044a0 con 0x7f6e04103c60 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e041985b0 con 0x7f6e04103c60 2024-04-16T05:25:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e00ff9700 1 -- 172.21.15.32:0/121359956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4003ab0 con 0x7f6e04103c60 2024-04-16T05:25:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.736+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0419b210 con 0x7f6e04103c60 2024-04-16T05:25:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.737+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e04108ea0 con 0x7f6e04103c60 2024-04-16T05:25:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.737+0000 7f6e00ff9700 1 -- 172.21.15.32:0/121359956 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6df4029070 con 0x7f6e04103c60 2024-04-16T05:25:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.738+0000 7f6e00ff9700 1 --2- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6dec0396b0 0x7f6dec03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.738+0000 7f6e00ff9700 1 -- 172.21.15.32:0/121359956 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6df401e070 con 0x7f6e04103c60 2024-04-16T05:25:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.738+0000 7f6e02ffd700 1 --2- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6dec0396b0 0x7f6dec03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.738+0000 7f6e02ffd700 1 --2- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6dec0396b0 0x7f6dec03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006fd0 tx=0x7f6df8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:44.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.743+0000 7f6e00ff9700 1 -- 172.21.15.32:0/121359956 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6df4013070 con 0x7f6e04103c60 2024-04-16T05:25:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.950+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f6e0402dfe0 con 0x7f6e04103c60 2024-04-16T05:25:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.952+0000 7f6e00ff9700 1 -- 172.21.15.32:0/121359956 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f6df40300c0 con 0x7f6e04103c60 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.954+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6dec0396b0 msgr2=0x7f6dec03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.954+0000 7f6e0a127700 1 --2- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6dec0396b0 0x7f6dec03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006fd0 tx=0x7f6df8006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.954+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 msgr2=0x7f6e04197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.954+0000 7f6e0a127700 1 --2- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 0x7f6e04197de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005850 tx=0x7f6df4005030 comp rx=0 tx=0).stop 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.954+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 shutdown_connections 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.954+0000 7f6e0a127700 1 --2- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6dec0396b0 0x7f6dec03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.954+0000 7f6e0a127700 1 --2- 172.21.15.32:0/121359956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103c60 0x7f6e04197de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.954+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 >> 172.21.15.32:0/121359956 conn(0x7f6e040fdb40 msgr2=0x7f6e040fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.955+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 shutdown_connections 2024-04-16T05:25:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:44.955+0000 7f6e0a127700 1 -- 172.21.15.32:0/121359956 wait complete. 2024-04-16T05:25:45.529 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-16T05:25:45.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-16T05:25:45.963 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:25:46.182 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:45 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/121359956' entity='client.admin' 2024-04-16T05:25:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.511+0000 7fe48d06c700 1 -- 172.21.15.32:0/3628004014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 msgr2=0x7fe488101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.511+0000 7fe48d06c700 1 --2- 172.21.15.32:0/3628004014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 0x7fe488101870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b0a0 tx=0x7fe47800b3b0 comp rx=0 tx=0).stop 2024-04-16T05:25:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.511+0000 7fe48d06c700 1 -- 172.21.15.32:0/3628004014 shutdown_connections 2024-04-16T05:25:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.511+0000 7fe48d06c700 1 --2- 172.21.15.32:0/3628004014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 0x7fe488101870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.511+0000 7fe48d06c700 1 -- 172.21.15.32:0/3628004014 >> 172.21.15.32:0/3628004014 conn(0x7fe4880fc9f0 msgr2=0x7fe4880fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.511+0000 7fe48d06c700 1 -- 172.21.15.32:0/3628004014 shutdown_connections 2024-04-16T05:25:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.511+0000 7fe48d06c700 1 -- 172.21.15.32:0/3628004014 wait complete. 2024-04-16T05:25:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.511+0000 7fe48d06c700 1 Processor -- start 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.512+0000 7fe48d06c700 1 -- start start 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.512+0000 7fe48d06c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 0x7fe488199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.512+0000 7fe48d06c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881074e0 con 0x7fe488101450 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.512+0000 7fe486d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 0x7fe488199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.512+0000 7fe486d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 0x7fe488199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39252/0 (socket says 172.21.15.32:39252) 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.512+0000 7fe486d9d700 1 -- 172.21.15.32:0/3985901391 learned_addr learned my addr 172.21.15.32:0/3985901391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.512+0000 7fe486d9d700 1 -- 172.21.15.32:0/3985901391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48819a320 con 0x7fe488101450 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.512+0000 7fe486d9d700 1 --2- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 0x7fe488199de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe478007890 tx=0x7fe47800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.513+0000 7fe47ffff700 1 -- 172.21.15.32:0/3985901391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478017040 con 0x7fe488101450 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.513+0000 7fe47ffff700 1 -- 172.21.15.32:0/3985901391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe478022070 con 0x7fe488101450 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.513+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48819a5b0 con 0x7fe488101450 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.513+0000 7fe47ffff700 1 -- 172.21.15.32:0/3985901391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe47801d5b0 con 0x7fe488101450 2024-04-16T05:25:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.513+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48819d2a0 con 0x7fe488101450 2024-04-16T05:25:48.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.513+0000 7fe47ffff700 1 -- 172.21.15.32:0/3985901391 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe47801d8b0 con 0x7fe488101450 2024-04-16T05:25:48.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.514+0000 7fe47ffff700 1 --2- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe474039650 0x7fe47403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:48.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.514+0000 7fe47ffff700 1 -- 172.21.15.32:0/3985901391 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe47800f050 con 0x7fe488101450 2024-04-16T05:25:48.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.514+0000 7fe48659c700 1 --2- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe474039650 0x7fe47403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:48.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.514+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe488107c60 con 0x7fe488101450 2024-04-16T05:25:48.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.514+0000 7fe48659c700 1 --2- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe474039650 0x7fe47403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe470006fd0 tx=0x7fe470006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:48.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.519+0000 7fe47ffff700 1 -- 172.21.15.32:0/3985901391 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe478007e20 con 0x7fe488101450 2024-04-16T05:25:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.729+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe48802e050 con 0x7fe474039650 2024-04-16T05:25:48.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.732+0000 7fe47ffff700 1 -- 172.21.15.32:0/3985901391 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe48802e050 con 0x7fe474039650 2024-04-16T05:25:48.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe474039650 msgr2=0x7fe47403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:48.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 --2- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe474039650 0x7fe47403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe470006fd0 tx=0x7fe470006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:48.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 msgr2=0x7fe488199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:48.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 --2- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 0x7fe488199de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe478007890 tx=0x7fe47800bd40 comp rx=0 tx=0).stop 2024-04-16T05:25:48.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 shutdown_connections 2024-04-16T05:25:48.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 --2- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe474039650 0x7fe47403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:48.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 --2- 172.21.15.32:0/3985901391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe488101450 0x7fe488199de0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:48.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 >> 172.21.15.32:0/3985901391 conn(0x7fe4880fc9f0 msgr2=0x7fe488190f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:48.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 shutdown_connections 2024-04-16T05:25:48.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:48.735+0000 7fe48d06c700 1 -- 172.21.15.32:0/3985901391 wait complete. 2024-04-16T05:25:49.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:25:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:25:49.562 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi082 2024-04-16T05:25:49.563 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:25:49.563 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.conf 2024-04-16T05:25:49.581 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:25:49.581 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:25:49.638 INFO:tasks.cephadm:Adding host smithi082 to orchestrator... 2024-04-16T05:25:49.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch host add smithi082 2024-04-16T05:25:49.879 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:25:50.966 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:50 smithi032 ceph-mon[32351]: Updating smithi032:/etc/ceph/ceph.conf 2024-04-16T05:25:50.966 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:50 smithi032 ceph-mon[32351]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:25:50.966 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:50 smithi032 ceph-mon[32351]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:25:50.966 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:50 smithi032 ceph-mon[32351]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:25:50.966 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:50 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:50.966 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:50 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:50.967 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:50 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.479+0000 7f74df8c4700 1 -- 172.21.15.32:0/1216260115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 msgr2=0x7f74d81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.479+0000 7f74df8c4700 1 --2- 172.21.15.32:0/1216260115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 0x7f74d81012b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009910 tx=0x7f74d4009c20 comp rx=0 tx=0).stop 2024-04-16T05:25:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.479+0000 7f74df8c4700 1 -- 172.21.15.32:0/1216260115 shutdown_connections 2024-04-16T05:25:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.479+0000 7f74df8c4700 1 --2- 172.21.15.32:0/1216260115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 0x7f74d81012b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.479+0000 7f74df8c4700 1 -- 172.21.15.32:0/1216260115 >> 172.21.15.32:0/1216260115 conn(0x7f74d80fc660 msgr2=0x7f74d80fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.479+0000 7f74df8c4700 1 -- 172.21.15.32:0/1216260115 shutdown_connections 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.479+0000 7f74df8c4700 1 -- 172.21.15.32:0/1216260115 wait complete. 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.480+0000 7f74df8c4700 1 Processor -- start 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.480+0000 7f74df8c4700 1 -- start start 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.480+0000 7f74df8c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 0x7f74d81039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.480+0000 7f74df8c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f74d810b2b0 con 0x7f74d8100ed0 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.480+0000 7f74dd660700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 0x7f74d81039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.480+0000 7f74dd660700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 0x7f74d81039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39258/0 (socket says 172.21.15.32:39258) 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.480+0000 7f74dd660700 1 -- 172.21.15.32:0/1146565646 learned_addr learned my addr 172.21.15.32:0/1146565646 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.481+0000 7f74dd660700 1 -- 172.21.15.32:0/1146565646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d8105890 con 0x7f74d8100ed0 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.481+0000 7f74dd660700 1 --2- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 0x7f74d81039a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74d4006010 tx=0x7f74d4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.481+0000 7f74ce7fc700 1 -- 172.21.15.32:0/1146565646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74d4026070 con 0x7f74d8100ed0 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.481+0000 7f74ce7fc700 1 -- 172.21.15.32:0/1146565646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74d400bc30 con 0x7f74d8100ed0 2024-04-16T05:25:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.481+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d8103f40 con 0x7f74d8100ed0 2024-04-16T05:25:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.481+0000 7f74ce7fc700 1 -- 172.21.15.32:0/1146565646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74d40039d0 con 0x7f74d8100ed0 2024-04-16T05:25:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.481+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d8104400 con 0x7f74d8100ed0 2024-04-16T05:25:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.482+0000 7f74ce7fc700 1 -- 172.21.15.32:0/1146565646 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f74d4029070 con 0x7f74d8100ed0 2024-04-16T05:25:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.482+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d81022d0 con 0x7f74d8100ed0 2024-04-16T05:25:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.482+0000 7f74ce7fc700 1 --2- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c40396b0 0x7f74c403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.482+0000 7f74ce7fc700 1 -- 172.21.15.32:0/1146565646 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f74d401e070 con 0x7f74d8100ed0 2024-04-16T05:25:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.482+0000 7f74dce5f700 1 --2- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c40396b0 0x7f74c403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.483+0000 7f74dce5f700 1 --2- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c40396b0 0x7f74c403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74c8006fd0 tx=0x7f74c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.487+0000 7f74ce7fc700 1 -- 172.21.15.32:0/1146565646 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74d4013070 con 0x7f74d8100ed0 2024-04-16T05:25:51.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:51.697+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}) -- 0x7f74d8104ed0 con 0x7f74c40396b0 2024-04-16T05:25:51.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:51 smithi032 ceph-mon[32351]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:25:53.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:52 smithi032 ceph-mon[32351]: Deploying cephadm binary to smithi082 2024-04-16T05:25:54.157 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:53 smithi032 ceph-mon[32351]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:25:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.058+0000 7f74ce7fc700 1 -- 172.21.15.32:0/1146565646 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f74d8104ed0 con 0x7f74c40396b0 2024-04-16T05:25:56.057 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi082' with addr '172.21.15.82' 2024-04-16T05:25:56.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.061+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c40396b0 msgr2=0x7f74c403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:56.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.061+0000 7f74df8c4700 1 --2- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c40396b0 0x7f74c403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74c8006fd0 tx=0x7f74c8006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:56.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.061+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 msgr2=0x7f74d81039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:56.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.061+0000 7f74df8c4700 1 --2- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 0x7f74d81039a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74d4006010 tx=0x7f74d4005030 comp rx=0 tx=0).stop 2024-04-16T05:25:56.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.061+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 shutdown_connections 2024-04-16T05:25:56.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.061+0000 7f74df8c4700 1 --2- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c40396b0 0x7f74c403bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:56.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.061+0000 7f74df8c4700 1 --2- 172.21.15.32:0/1146565646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d8100ed0 0x7f74d81039a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:56.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.061+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 >> 172.21.15.32:0/1146565646 conn(0x7f74d80fc660 msgr2=0x7f74d80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:56.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.062+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 shutdown_connections 2024-04-16T05:25:56.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:56.062+0000 7f74df8c4700 1 -- 172.21.15.32:0/1146565646 wait complete. 2024-04-16T05:25:56.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:55 smithi032 ceph-mon[32351]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:25:56.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch host ls --format=json 2024-04-16T05:25:56.892 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:25:57.324 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:57.324 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:57 smithi032 ceph-mon[32351]: Added host smithi082 2024-04-16T05:25:57.324 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:25:57.324 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:58.245 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:58 smithi032 ceph-mon[32351]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:25:58.245 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:58.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.463+0000 7f3b46723700 1 -- 172.21.15.32:0/3387737561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 msgr2=0x7f3b401089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:58.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.463+0000 7f3b46723700 1 --2- 172.21.15.32:0/3387737561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 0x7f3b401089b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009910 tx=0x7f3b30009c20 comp rx=0 tx=0).stop 2024-04-16T05:25:58.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.463+0000 7f3b46723700 1 -- 172.21.15.32:0/3387737561 shutdown_connections 2024-04-16T05:25:58.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.463+0000 7f3b46723700 1 --2- 172.21.15.32:0/3387737561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 0x7f3b401089b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:58.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.463+0000 7f3b46723700 1 -- 172.21.15.32:0/3387737561 >> 172.21.15.32:0/3387737561 conn(0x7f3b400fe030 msgr2=0x7f3b40100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:58.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.463+0000 7f3b46723700 1 -- 172.21.15.32:0/3387737561 shutdown_connections 2024-04-16T05:25:58.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.463+0000 7f3b46723700 1 -- 172.21.15.32:0/3387737561 wait complete. 2024-04-16T05:25:58.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.464+0000 7f3b46723700 1 Processor -- start 2024-04-16T05:25:58.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.464+0000 7f3b46723700 1 -- start start 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.464+0000 7f3b46723700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 0x7f3b40074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.464+0000 7f3b46723700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010cb90 con 0x7f3b401085d0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.464+0000 7f3b3ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 0x7f3b40074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.464+0000 7f3b3ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 0x7f3b40074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37826/0 (socket says 172.21.15.32:37826) 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.464+0000 7f3b3ffff700 1 -- 172.21.15.32:0/828661612 learned_addr learned my addr 172.21.15.32:0/828661612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.464+0000 7f3b3ffff700 1 -- 172.21.15.32:0/828661612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b40076ce0 con 0x7f3b401085d0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.465+0000 7f3b3ffff700 1 --2- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 0x7f3b40074df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005850 tx=0x7f3b30005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.465+0000 7f3b3dffb700 1 -- 172.21.15.32:0/828661612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b30026070 con 0x7f3b401085d0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.465+0000 7f3b3dffb700 1 -- 172.21.15.32:0/828661612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3000bc30 con 0x7f3b401085d0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.465+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b40075330 con 0x7f3b401085d0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.465+0000 7f3b3dffb700 1 -- 172.21.15.32:0/828661612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b30003a20 con 0x7f3b401085d0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.465+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b400757f0 con 0x7f3b401085d0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.466+0000 7f3b3dffb700 1 -- 172.21.15.32:0/828661612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3b30029070 con 0x7f3b401085d0 2024-04-16T05:25:58.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.466+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b40103c40 con 0x7f3b401085d0 2024-04-16T05:25:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.466+0000 7f3b3dffb700 1 --2- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3b2c0396b0 0x7f3b2c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:25:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.466+0000 7f3b3dffb700 1 -- 172.21.15.32:0/828661612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3b3001e070 con 0x7f3b401085d0 2024-04-16T05:25:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.466+0000 7f3b37fff700 1 --2- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3b2c0396b0 0x7f3b2c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:25:58.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.466+0000 7f3b37fff700 1 --2- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3b2c0396b0 0x7f3b2c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b28006fd0 tx=0x7f3b28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:25:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.471+0000 7f3b3dffb700 1 -- 172.21.15.32:0/828661612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b30013070 con 0x7f3b401085d0 2024-04-16T05:25:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.687+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b40076390 con 0x7f3b2c0396b0 2024-04-16T05:25:58.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.688+0000 7f3b3dffb700 1 -- 172.21.15.32:0/828661612 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f3b40076390 con 0x7f3b2c0396b0 2024-04-16T05:25:58.686 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:25:58.686 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}] 2024-04-16T05:25:58.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3b2c0396b0 msgr2=0x7f3b2c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:58.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 --2- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3b2c0396b0 0x7f3b2c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b28006fd0 tx=0x7f3b28006e40 comp rx=0 tx=0).stop 2024-04-16T05:25:58.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 msgr2=0x7f3b40074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:25:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 --2- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 0x7f3b40074df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005850 tx=0x7f3b30005030 comp rx=0 tx=0).stop 2024-04-16T05:25:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 shutdown_connections 2024-04-16T05:25:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 --2- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3b2c0396b0 0x7f3b2c03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 --2- 172.21.15.32:0/828661612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b401085d0 0x7f3b40074df0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:25:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 >> 172.21.15.32:0/828661612 conn(0x7f3b400fe030 msgr2=0x7f3b400ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:25:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 shutdown_connections 2024-04-16T05:25:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:25:58.690+0000 7f3b46723700 1 -- 172.21.15.32:0/828661612 wait complete. 2024-04-16T05:25:59.307 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-04-16T05:25:59.307 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:25:59.307 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-04-16T05:25:59.325 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:25:59.325 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:25:59.382 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-04-16T05:25:59.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch host add smithi134 2024-04-16T05:25:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:25:59.885 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:25:59.885 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:25:59 smithi032 ceph-mon[32351]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:26:00.880 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:00 smithi032 ceph-mon[32351]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:01.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 -- 172.21.15.32:0/2528176604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 msgr2=0x7f9a181089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:01.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 --2- 172.21.15.32:0/2528176604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 0x7f9a181089b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009910 tx=0x7f9a08009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:01.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 -- 172.21.15.32:0/2528176604 shutdown_connections 2024-04-16T05:26:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 --2- 172.21.15.32:0/2528176604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 0x7f9a181089b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 -- 172.21.15.32:0/2528176604 >> 172.21.15.32:0/2528176604 conn(0x7f9a180fe030 msgr2=0x7f9a18100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 -- 172.21.15.32:0/2528176604 shutdown_connections 2024-04-16T05:26:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 -- 172.21.15.32:0/2528176604 wait complete. 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 Processor -- start 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 -- start start 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.102+0000 7f9a1e649700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 0x7f9a1819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a1e649700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18076590 con 0x7f9a181085d0 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a17fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 0x7f9a1819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a17fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 0x7f9a1819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37856/0 (socket says 172.21.15.32:37856) 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a17fff700 1 -- 172.21.15.32:0/2977534094 learned_addr learned my addr 172.21.15.32:0/2977534094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a17fff700 1 -- 172.21.15.32:0/2977534094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1819a740 con 0x7f9a181085d0 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a17fff700 1 --2- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 0x7f9a1819a200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a08005850 tx=0x7f9a08005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a15ffb700 1 -- 172.21.15.32:0/2977534094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a08026070 con 0x7f9a181085d0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a15ffb700 1 -- 172.21.15.32:0/2977534094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a0800bc30 con 0x7f9a181085d0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1819a9d0 con 0x7f9a181085d0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a15ffb700 1 -- 172.21.15.32:0/2977534094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a08003a20 con 0x7f9a181085d0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.103+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1819ead0 con 0x7f9a181085d0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.104+0000 7f9a15ffb700 1 -- 172.21.15.32:0/2977534094 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9a08029070 con 0x7f9a181085d0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.104+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a18103c40 con 0x7f9a181085d0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.104+0000 7f9a15ffb700 1 --2- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9a04039650 0x7f9a0403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.104+0000 7f9a15ffb700 1 -- 172.21.15.32:0/2977534094 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a0801e070 con 0x7f9a181085d0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.104+0000 7f9a0ffff700 1 --2- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9a04039650 0x7f9a0403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.105+0000 7f9a0ffff700 1 --2- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9a04039650 0x7f9a0403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9a00006fd0 tx=0x7f9a00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.109+0000 7f9a15ffb700 1 -- 172.21.15.32:0/2977534094 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a08013070 con 0x7f9a181085d0 2024-04-16T05:26:01.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:01.323+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) -- 0x7f9a1819b560 con 0x7f9a04039650 2024-04-16T05:26:02.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:26:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:03.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:03 smithi032 ceph-mon[32351]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-16T05:26:03.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:03 smithi032 ceph-mon[32351]: Deploying cephadm binary to smithi134 2024-04-16T05:26:03.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:03 smithi032 ceph-mon[32351]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:03.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:03 smithi032 ceph-mon[32351]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:04.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:04 smithi032 ceph-mon[32351]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.660+0000 7f9a15ffb700 1 -- 172.21.15.32:0/2977534094 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9a1819b560 con 0x7f9a04039650 2024-04-16T05:26:05.659 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-04-16T05:26:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.662+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9a04039650 msgr2=0x7f9a0403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.662+0000 7f9a1e649700 1 --2- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9a04039650 0x7f9a0403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9a00006fd0 tx=0x7f9a00006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.662+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 msgr2=0x7f9a1819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.662+0000 7f9a1e649700 1 --2- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 0x7f9a1819a200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a08005850 tx=0x7f9a08005030 comp rx=0 tx=0).stop 2024-04-16T05:26:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.662+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 shutdown_connections 2024-04-16T05:26:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.662+0000 7f9a1e649700 1 --2- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9a04039650 0x7f9a0403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.662+0000 7f9a1e649700 1 --2- 172.21.15.32:0/2977534094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a181085d0 0x7f9a1819a200 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.662+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 >> 172.21.15.32:0/2977534094 conn(0x7f9a180fe030 msgr2=0x7f9a180fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.663+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 shutdown_connections 2024-04-16T05:26:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:05.663+0000 7f9a1e649700 1 -- 172.21.15.32:0/2977534094 wait complete. 2024-04-16T05:26:06.289 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:06.289 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:26:06.289 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:06.289 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:06.289 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:06.290 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:06.290 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: Added host smithi134 2024-04-16T05:26:06.290 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:06.290 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:06.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch host ls --format=json 2024-04-16T05:26:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:07.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:07 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:08.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.030+0000 7fda68f1d700 1 -- 172.21.15.32:0/3037947739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 msgr2=0x7fda641089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:08.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.030+0000 7fda68f1d700 1 --2- 172.21.15.32:0/3037947739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 0x7fda641089b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fda54009910 tx=0x7fda54009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:08.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.030+0000 7fda68f1d700 1 -- 172.21.15.32:0/3037947739 shutdown_connections 2024-04-16T05:26:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.030+0000 7fda68f1d700 1 --2- 172.21.15.32:0/3037947739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 0x7fda641089b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.030+0000 7fda68f1d700 1 -- 172.21.15.32:0/3037947739 >> 172.21.15.32:0/3037947739 conn(0x7fda640fe030 msgr2=0x7fda64100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.030+0000 7fda68f1d700 1 -- 172.21.15.32:0/3037947739 shutdown_connections 2024-04-16T05:26:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.030+0000 7fda68f1d700 1 -- 172.21.15.32:0/3037947739 wait complete. 2024-04-16T05:26:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda68f1d700 1 Processor -- start 2024-04-16T05:26:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda68f1d700 1 -- start start 2024-04-16T05:26:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda68f1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 0x7fda64197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda68f1d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda6410cb90 con 0x7fda641085d0 2024-04-16T05:26:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda6259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 0x7fda64197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda6259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 0x7fda64197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37876/0 (socket says 172.21.15.32:37876) 2024-04-16T05:26:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda6259c700 1 -- 172.21.15.32:0/2108621339 learned_addr learned my addr 172.21.15.32:0/2108621339 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:26:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda6259c700 1 -- 172.21.15.32:0/2108621339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda641984d0 con 0x7fda641085d0 2024-04-16T05:26:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.031+0000 7fda6259c700 1 --2- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 0x7fda64197f90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda54005850 tx=0x7fda54005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.032+0000 7fda537fe700 1 -- 172.21.15.32:0/2108621339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda54026070 con 0x7fda641085d0 2024-04-16T05:26:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.032+0000 7fda537fe700 1 -- 172.21.15.32:0/2108621339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda5400bc30 con 0x7fda641085d0 2024-04-16T05:26:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.032+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda64198760 con 0x7fda641085d0 2024-04-16T05:26:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.032+0000 7fda537fe700 1 -- 172.21.15.32:0/2108621339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda54003a20 con 0x7fda641085d0 2024-04-16T05:26:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.032+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6419c860 con 0x7fda641085d0 2024-04-16T05:26:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.033+0000 7fda537fe700 1 -- 172.21.15.32:0/2108621339 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fda54029070 con 0x7fda641085d0 2024-04-16T05:26:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.033+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda64103c40 con 0x7fda641085d0 2024-04-16T05:26:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.033+0000 7fda537fe700 1 --2- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda4c039650 0x7fda4c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.033+0000 7fda537fe700 1 -- 172.21.15.32:0/2108621339 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fda5401e070 con 0x7fda641085d0 2024-04-16T05:26:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.033+0000 7fda61d9b700 1 --2- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda4c039650 0x7fda4c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.033+0000 7fda61d9b700 1 --2- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda4c039650 0x7fda4c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fda58006fd0 tx=0x7fda58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:08.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.038+0000 7fda537fe700 1 -- 172.21.15.32:0/2108621339 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda54013070 con 0x7fda641085d0 2024-04-16T05:26:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.253+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda641992f0 con 0x7fda4c039650 2024-04-16T05:26:08.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.254+0000 7fda537fe700 1 -- 172.21.15.32:0/2108621339 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fda641992f0 con 0x7fda4c039650 2024-04-16T05:26:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:26:08.253 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-04-16T05:26:08.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda4c039650 msgr2=0x7fda4c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:08.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 --2- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda4c039650 0x7fda4c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fda58006fd0 tx=0x7fda58006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:08.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 msgr2=0x7fda64197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:08.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 --2- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 0x7fda64197f90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda54005850 tx=0x7fda54005030 comp rx=0 tx=0).stop 2024-04-16T05:26:08.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 shutdown_connections 2024-04-16T05:26:08.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 --2- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda4c039650 0x7fda4c03bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:08.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 --2- 172.21.15.32:0/2108621339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda641085d0 0x7fda64197f90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:08.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 >> 172.21.15.32:0/2108621339 conn(0x7fda640fe030 msgr2=0x7fda640ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:08.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 shutdown_connections 2024-04-16T05:26:08.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:08.257+0000 7fda68f1d700 1 -- 172.21.15.32:0/2108621339 wait complete. 2024-04-16T05:26:08.835 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:08 smithi032 ceph-mon[32351]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:08.835 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:08.863 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-16T05:26:08.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd crush tunables default 2024-04-16T05:26:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:09.841 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:09 smithi032 ceph-mon[32351]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:26:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.601+0000 7fe212722700 1 -- 172.21.15.32:0/922448249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 msgr2=0x7fe20c103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.601+0000 7fe212722700 1 --2- 172.21.15.32:0/922448249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 0x7fe20c103a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009910 tx=0x7fe1fc009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.601+0000 7fe212722700 1 -- 172.21.15.32:0/922448249 shutdown_connections 2024-04-16T05:26:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.601+0000 7fe212722700 1 --2- 172.21.15.32:0/922448249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 0x7fe20c103a80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:10.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.602+0000 7fe212722700 1 -- 172.21.15.32:0/922448249 >> 172.21.15.32:0/922448249 conn(0x7fe20c0fcef0 msgr2=0x7fe20c0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:10.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.602+0000 7fe212722700 1 -- 172.21.15.32:0/922448249 shutdown_connections 2024-04-16T05:26:10.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.602+0000 7fe212722700 1 -- 172.21.15.32:0/922448249 wait complete. 2024-04-16T05:26:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.602+0000 7fe212722700 1 Processor -- start 2024-04-16T05:26:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.602+0000 7fe212722700 1 -- start start 2024-04-16T05:26:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe212722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 0x7fe20c101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe212722700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c10cb90 con 0x7fe20c1036a0 2024-04-16T05:26:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe20bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 0x7fe20c101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe20bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 0x7fe20c101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52996/0 (socket says 172.21.15.32:52996) 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe20bfff700 1 -- 172.21.15.32:0/2897919031 learned_addr learned my addr 172.21.15.32:0/2897919031 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe20bfff700 1 -- 172.21.15.32:0/2897919031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c1031f0 con 0x7fe20c1036a0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe20bfff700 1 --2- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 0x7fe20c101300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006010 tx=0x7fe1fc005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe2097fa700 1 -- 172.21.15.32:0/2897919031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1fc026070 con 0x7fe20c1036a0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20c101840 con 0x7fe20c1036a0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe2097fa700 1 -- 172.21.15.32:0/2897919031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1fc00bc30 con 0x7fe20c1036a0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.603+0000 7fe2097fa700 1 -- 172.21.15.32:0/2897919031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1fc003ab0 con 0x7fe20c1036a0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.604+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20c101d00 con 0x7fe20c1036a0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.604+0000 7fe2097fa700 1 -- 172.21.15.32:0/2897919031 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe1fc029070 con 0x7fe20c1036a0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.604+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20c06b080 con 0x7fe20c1036a0 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.605+0000 7fe2097fa700 1 --2- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1f80396b0 0x7fe1f803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.605+0000 7fe2097fa700 1 -- 172.21.15.32:0/2897919031 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe1fc01e070 con 0x7fe20c1036a0 2024-04-16T05:26:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.605+0000 7fe20b7fe700 1 --2- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1f80396b0 0x7fe1f803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.605+0000 7fe20b7fe700 1 --2- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1f80396b0 0x7fe1f803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4006fd0 tx=0x7fe1f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:10.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.610+0000 7fe2097fa700 1 -- 172.21.15.32:0/2897919031 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1fc013070 con 0x7fe20c1036a0 2024-04-16T05:26:10.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:10.828+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe20c1028d0 con 0x7fe20c1036a0 2024-04-16T05:26:10.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:10 smithi032 ceph-mon[32351]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:11.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.662+0000 7fe2097fa700 1 -- 172.21.15.32:0/2897919031 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fe1fc00e3e0 con 0x7fe20c1036a0 2024-04-16T05:26:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.664+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1f80396b0 msgr2=0x7fe1f803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 --2- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1f80396b0 0x7fe1f803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4006fd0 tx=0x7fe1f4006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 msgr2=0x7fe20c101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 --2- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 0x7fe20c101300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006010 tx=0x7fe1fc005050 comp rx=0 tx=0).stop 2024-04-16T05:26:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 shutdown_connections 2024-04-16T05:26:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 --2- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1f80396b0 0x7fe1f803bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 --2- 172.21.15.32:0/2897919031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20c1036a0 0x7fe20c101300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 >> 172.21.15.32:0/2897919031 conn(0x7fe20c0fcef0 msgr2=0x7fe20c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 shutdown_connections 2024-04-16T05:26:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:11.665+0000 7fe212722700 1 -- 172.21.15.32:0/2897919031 wait complete. 2024-04-16T05:26:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-04-16T05:26:11.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:11 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2897919031' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T05:26:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:26:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:12.333 INFO:tasks.cephadm:Adding mon.a on smithi032 2024-04-16T05:26:12.334 INFO:tasks.cephadm:Adding mon.b on smithi082 2024-04-16T05:26:12.334 INFO:tasks.cephadm:Adding mon.c on smithi134 2024-04-16T05:26:12.334 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch apply mon '3;smithi032:172.21.15.32=a;smithi082:172.21.15.82=b;smithi134:172.21.15.134=c' 2024-04-16T05:26:12.557 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:12.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:12 smithi032 ceph-mon[32351]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:12.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:12 smithi032 ceph-mon[32351]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-16T05:26:12.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:12 smithi032 ceph-mon[32351]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:12.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:12 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2897919031' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T05:26:12.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:12 smithi032 ceph-mon[32351]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:12.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:12.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:12.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:13.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:13 smithi032 ceph-mon[32351]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:13.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:13 smithi032 ceph-mon[32351]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.006+0000 7f1b519ea700 1 -- 172.21.15.134:0/1116829678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 msgr2=0x7f1b4c103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.006+0000 7f1b519ea700 1 --2- 172.21.15.134:0/1116829678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 0x7f1b4c103aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009910 tx=0x7f1b3c009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.006+0000 7f1b519ea700 1 -- 172.21.15.134:0/1116829678 shutdown_connections 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.006+0000 7f1b519ea700 1 --2- 172.21.15.134:0/1116829678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 0x7f1b4c103aa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.006+0000 7f1b519ea700 1 -- 172.21.15.134:0/1116829678 >> 172.21.15.134:0/1116829678 conn(0x7f1b4c0fcf10 msgr2=0x7f1b4c0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.006+0000 7f1b519ea700 1 -- 172.21.15.134:0/1116829678 shutdown_connections 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.006+0000 7f1b519ea700 1 -- 172.21.15.134:0/1116829678 wait complete. 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.007+0000 7f1b519ea700 1 Processor -- start 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.007+0000 7f1b519ea700 1 -- start start 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.007+0000 7f1b519ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 0x7f1b4c101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:14.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.007+0000 7f1b519ea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c10cbb0 con 0x7f1b4c1036c0 2024-04-16T05:26:14.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.007+0000 7f1b4affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 0x7f1b4c101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:14.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.007+0000 7f1b4affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 0x7f1b4c101320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.134:44556/0 (socket says 172.21.15.134:44556) 2024-04-16T05:26:14.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.007+0000 7f1b4affd700 1 -- 172.21.15.134:0/1890451619 learned_addr learned my addr 172.21.15.134:0/1890451619 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:26:14.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.008+0000 7f1b4affd700 1 -- 172.21.15.134:0/1890451619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c103210 con 0x7f1b4c1036c0 2024-04-16T05:26:14.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.008+0000 7f1b4affd700 1 --2- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 0x7f1b4c101320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006010 tx=0x7f1b3c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:14.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.008+0000 7f1b509e8700 1 -- 172.21.15.134:0/1890451619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3c026070 con 0x7f1b4c1036c0 2024-04-16T05:26:14.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.008+0000 7f1b509e8700 1 -- 172.21.15.134:0/1890451619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c00bc30 con 0x7f1b4c1036c0 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.008+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c101860 con 0x7f1b4c1036c0 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.008+0000 7f1b509e8700 1 -- 172.21.15.134:0/1890451619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3c003ab0 con 0x7f1b4c1036c0 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.008+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c101d20 con 0x7f1b4c1036c0 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.009+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c06b280 con 0x7f1b4c1036c0 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.009+0000 7f1b509e8700 1 -- 172.21.15.134:0/1890451619 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1b3c029070 con 0x7f1b4c1036c0 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.009+0000 7f1b509e8700 1 --2- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1b380396b0 0x7f1b3803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.009+0000 7f1b509e8700 1 -- 172.21.15.134:0/1890451619 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1b3c01e070 con 0x7f1b4c1036c0 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.009+0000 7f1b427fc700 1 --2- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1b380396b0 0x7f1b3803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:14.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.010+0000 7f1b427fc700 1 --2- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1b380396b0 0x7f1b3803bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1b34006fd0 tx=0x7f1b34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:14.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.015+0000 7f1b509e8700 1 -- 172.21.15.134:0/1890451619 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b3c013070 con 0x7f1b4c1036c0 2024-04-16T05:26:14.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.227+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi082:172.21.15.82=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}) -- 0x7f1b4c101f50 con 0x7f1b380396b0 2024-04-16T05:26:14.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.234+0000 7f1b509e8700 1 -- 172.21.15.134:0/1890451619 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1b4c101f50 con 0x7f1b380396b0 2024-04-16T05:26:14.233 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mon update... 2024-04-16T05:26:14.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1b380396b0 msgr2=0x7f1b3803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:14.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 --2- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1b380396b0 0x7f1b3803bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1b34006fd0 tx=0x7f1b34006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:14.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 msgr2=0x7f1b4c101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:14.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 --2- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 0x7f1b4c101320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006010 tx=0x7f1b3c005050 comp rx=0 tx=0).stop 2024-04-16T05:26:14.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 shutdown_connections 2024-04-16T05:26:14.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 --2- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1b380396b0 0x7f1b3803bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:14.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 --2- 172.21.15.134:0/1890451619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c1036c0 0x7f1b4c101320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:14.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 >> 172.21.15.134:0/1890451619 conn(0x7f1b4c0fcf10 msgr2=0x7f1b4c104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:14.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 shutdown_connections 2024-04-16T05:26:14.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:14.236+0000 7f1b519ea700 1 -- 172.21.15.134:0/1890451619 wait complete. 2024-04-16T05:26:14.827 DEBUG:teuthology.orchestra.run.smithi082:mon.b> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.b.service 2024-04-16T05:26:14.830 DEBUG:teuthology.orchestra.run.smithi134:mon.c> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.c.service 2024-04-16T05:26:14.832 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T05:26:14.832 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph mon dump -f json 2024-04-16T05:26:14.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:14 smithi032 ceph-mon[32351]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:15.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:15 smithi032 ceph-mon[32351]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi082:172.21.15.82=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:15.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:15 smithi032 ceph-mon[32351]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi082:172.21.15.82=b;smithi134:172.21.15.134=c;count:3 2024-04-16T05:26:15.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:15 smithi032 ceph-mon[32351]: Deploying daemon mon.c on smithi134 2024-04-16T05:26:16.673 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:26:16.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:16 smithi032 ceph-mon[32351]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:18.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.476+0000 7fad4b3d7700 1 -- 172.21.15.134:0/834831384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 msgr2=0x7fad441079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:18.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.476+0000 7fad4b3d7700 1 --2- 172.21.15.134:0/834831384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 0x7fad441079a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad40009910 tx=0x7fad40009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:18.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.476+0000 7fad4b3d7700 1 -- 172.21.15.134:0/834831384 shutdown_connections 2024-04-16T05:26:18.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.476+0000 7fad4b3d7700 1 --2- 172.21.15.134:0/834831384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 0x7fad441079a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:18.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.476+0000 7fad4b3d7700 1 -- 172.21.15.134:0/834831384 >> 172.21.15.134:0/834831384 conn(0x7fad440fd020 msgr2=0x7fad440ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:18.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.476+0000 7fad4b3d7700 1 -- 172.21.15.134:0/834831384 shutdown_connections 2024-04-16T05:26:18.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.476+0000 7fad4b3d7700 1 -- 172.21.15.134:0/834831384 wait complete. 2024-04-16T05:26:18.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.477+0000 7fad4b3d7700 1 Processor -- start 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.477+0000 7fad4b3d7700 1 -- start start 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.477+0000 7fad4b3d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 0x7fad44198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.477+0000 7fad4b3d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fad4410bb80 con 0x7fad441075c0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.477+0000 7fad49173700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 0x7fad44198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.477+0000 7fad49173700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 0x7fad44198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.134:53162/0 (socket says 172.21.15.134:53162) 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.477+0000 7fad49173700 1 -- 172.21.15.134:0/1329355830 learned_addr learned my addr 172.21.15.134:0/1329355830 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.478+0000 7fad49173700 1 -- 172.21.15.134:0/1329355830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad44199410 con 0x7fad441075c0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.478+0000 7fad49173700 1 --2- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 0x7fad44198ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad40005850 tx=0x7fad40005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.478+0000 7fad3a7fc700 1 -- 172.21.15.134:0/1329355830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad40026070 con 0x7fad441075c0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.478+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad441996a0 con 0x7fad441075c0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.478+0000 7fad3a7fc700 1 -- 172.21.15.134:0/1329355830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4000bc30 con 0x7fad441075c0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.478+0000 7fad3a7fc700 1 -- 172.21.15.134:0/1329355830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad40003a20 con 0x7fad441075c0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.478+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad44199b00 con 0x7fad441075c0 2024-04-16T05:26:18.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.479+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad44102c30 con 0x7fad441075c0 2024-04-16T05:26:18.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.479+0000 7fad3a7fc700 1 -- 172.21.15.134:0/1329355830 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fad40029070 con 0x7fad441075c0 2024-04-16T05:26:18.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.479+0000 7fad3a7fc700 1 --2- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fad30039650 0x7fad3003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:18.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.479+0000 7fad3a7fc700 1 -- 172.21.15.134:0/1329355830 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fad4001e070 con 0x7fad441075c0 2024-04-16T05:26:18.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.479+0000 7fad48972700 1 --2- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fad30039650 0x7fad3003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:18.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.480+0000 7fad48972700 1 --2- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fad30039650 0x7fad3003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad34006fd0 tx=0x7fad34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:18.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.485+0000 7fad3a7fc700 1 -- 172.21.15.134:0/1329355830 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad40013070 con 0x7fad441075c0 2024-04-16T05:26:18.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.775+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fad4406b350 con 0x7fad441075c0 2024-04-16T05:26:18.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.775+0000 7fad3a7fc700 1 -- 172.21.15.134:0/1329355830 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fad40030090 con 0x7fad441075c0 2024-04-16T05:26:18.778 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:26:18.778 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":1,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","modified":"2024-04-16T05:24:34.788250Z","created":"2024-04-16T05:24:34.788250Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-16T05:26:18.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fad30039650 msgr2=0x7fad3003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:18.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 --2- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fad30039650 0x7fad3003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad34006fd0 tx=0x7fad34006e40 comp rx=0 tx=0).stop 2024-04-16T05:26:18.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 msgr2=0x7fad44198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:18.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 --2- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 0x7fad44198ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad40005850 tx=0x7fad40005030 comp rx=0 tx=0).stop 2024-04-16T05:26:18.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 shutdown_connections 2024-04-16T05:26:18.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 --2- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fad30039650 0x7fad3003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:18.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 --2- 172.21.15.134:0/1329355830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad441075c0 0x7fad44198ed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:18.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 >> 172.21.15.134:0/1329355830 conn(0x7fad440fd020 msgr2=0x7fad440fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:18.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 shutdown_connections 2024-04-16T05:26:18.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:18.778+0000 7fad4b3d7700 1 -- 172.21.15.134:0/1329355830 wait complete. 2024-04-16T05:26:18.780 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 1 2024-04-16T05:26:18.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:18 smithi032 ceph-mon[32351]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:20.434 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T05:26:20.435 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph mon dump -f json 2024-04-16T05:26:20.682 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:26:24.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:23 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: mon.a calling monitor election 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: mon.c calling monitor election 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: monmap epoch 2 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: last_changed 2024-04-16T05:26:18.812401+0000 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: created 2024-04-16T05:24:34.788250+0000 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: min_mon_release 19 (squid) 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: election_strategy: 1 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: fsmap 2024-04-16T05:26:24.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:24.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: mgrmap e13: x(active, since 51s) 2024-04-16T05:26:24.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: overall HEALTH_OK 2024-04-16T05:26:24.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:24.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:24.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:24.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:24.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:25.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.106+0000 7fd2c26b0700 1 -- 172.21.15.134:0/519839275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a40035c0 msgr2=0x7fd2a4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:25.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.106+0000 7fd2c26b0700 1 --2- 172.21.15.134:0/519839275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a40035c0 0x7fd2a4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc101430 tx=0x7fd2ac00feb0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.106+0000 7fd2c26b0700 1 -- 172.21.15.134:0/519839275 shutdown_connections 2024-04-16T05:26:25.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.106+0000 7fd2c26b0700 1 --2- 172.21.15.134:0/519839275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a40035c0 0x7fd2a4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.106+0000 7fd2c26b0700 1 -- 172.21.15.134:0/519839275 >> 172.21.15.134:0/519839275 conn(0x7fd2bc0fd030 msgr2=0x7fd2bc0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:25.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.106+0000 7fd2c26b0700 1 -- 172.21.15.134:0/519839275 shutdown_connections 2024-04-16T05:26:25.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.107+0000 7fd2c26b0700 1 -- 172.21.15.134:0/519839275 wait complete. 2024-04-16T05:26:25.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.107+0000 7fd2c26b0700 1 Processor -- start 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2c26b0700 1 -- start start 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2c26b0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 0x7fd2bc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2c26b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2bc19a8c0 0x7fd2bc19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2c26b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc1095a0 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2c26b0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc109420 con 0x7fd2a40035c0 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 0x7fd2bc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 0x7fd2bc19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:49934/0 (socket says 172.21.15.134:49934) 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2bbfff700 1 -- 172.21.15.134:0/687096546 learned_addr learned my addr 172.21.15.134:0/687096546 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2bb7fe700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2bc19a8c0 0x7fd2bc19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2bbfff700 1 -- 172.21.15.134:0/687096546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 msgr2=0x7fd2bc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-16T05:26:25.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2bbfff700 1 -- 172.21.15.134:0/687096546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 msgr2=0x7fd2bc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2bbfff700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 0x7fd2bc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.108+0000 7fd2bbfff700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 0x7fd2bc19a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2bb7fe700 1 -- 172.21.15.134:0/687096546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 msgr2=0x7fd2bc19a380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2bb7fe700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 0x7fd2bc19a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2bb7fe700 1 -- 172.21.15.134:0/687096546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc19af00 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2bb7fe700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2bc19a8c0 0x7fd2bc19ec50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000b4d0 tx=0x7fd2b000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2b97fa700 1 -- 172.21.15.134:0/687096546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd2b0011630 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2b97fa700 1 -- 172.21.15.134:0/687096546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2b0011cb0 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2bc19f190 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2b97fa700 1 -- 172.21.15.134:0/687096546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd2b0010e60 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.109+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2bc19f770 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.110+0000 7fd2b97fa700 1 -- 172.21.15.134:0/687096546 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd2b0018020 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.110+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2bc19fa80 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.111+0000 7fd2b97fa700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd2a80393f0 0x7fd2a803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:25.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.111+0000 7fd2b97fa700 1 -- 172.21.15.134:0/687096546 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd2b004c200 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.111+0000 7fd2bbfff700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd2a80393f0 0x7fd2a803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:25.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.112+0000 7fd2bbfff700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd2a80393f0 0x7fd2a803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00fa60 tx=0x7fd2ac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:25.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.117+0000 7fd2b97fa700 1 -- 172.21.15.134:0/687096546 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2bc19fa80 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:24 smithi032 ceph-mon[32351]: Deploying daemon mon.b on smithi082 2024-04-16T05:26:25.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:25.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:24 smithi134 ceph-mon[36153]: Deploying daemon mon.b on smithi082 2024-04-16T05:26:25.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:25.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.397+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd2bc19b770 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.398+0000 7fd2b97fa700 1 -- 172.21.15.134:0/687096546 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd2bc19b770 con 0x7fd2bc19a8c0 2024-04-16T05:26:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:26:25.397 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","modified":"2024-04-16T05:26:18.812401Z","created":"2024-04-16T05:24:34.788250Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd2a80393f0 msgr2=0x7fd2a803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd2a80393f0 0x7fd2a803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00fa60 tx=0x7fd2ac005960 comp rx=0 tx=0).stop 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2bc19a8c0 msgr2=0x7fd2bc19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2bc19a8c0 0x7fd2bc19ec50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000b4d0 tx=0x7fd2b000b890 comp rx=0 tx=0).stop 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 shutdown_connections 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd2a80393f0 0x7fd2a803b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40035c0 0x7fd2bc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 --2- 172.21.15.134:0/687096546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2bc19a8c0 0x7fd2bc19ec50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 >> 172.21.15.134:0/687096546 conn(0x7fd2bc0fd030 msgr2=0x7fd2bc06b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 shutdown_connections 2024-04-16T05:26:25.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:25.400+0000 7fd2c26b0700 1 -- 172.21.15.134:0/687096546 wait complete. 2024-04-16T05:26:25.400 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-04-16T05:26:25.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:25 smithi134 ceph-mon[36153]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:25 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/687096546' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:25 smithi032 ceph-mon[32351]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:25 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/687096546' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:26.954 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T05:26:26.954 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph mon dump -f json 2024-04-16T05:26:27.196 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:26:28.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:27 smithi082 systemd[1]: Started Ceph mon.b for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:26:32.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.956+0000 7efc29312700 1 -- 172.21.15.134:0/3666036636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc24103890 msgr2=0x7efc10005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:32.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.956+0000 7efc29312700 1 --2- 172.21.15.134:0/3666036636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc24103890 0x7efc10005610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc14005fa0 tx=0x7efc1400ff70 comp rx=0 tx=0).stop 2024-04-16T05:26:32.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.956+0000 7efc29312700 1 -- 172.21.15.134:0/3666036636 shutdown_connections 2024-04-16T05:26:32.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.956+0000 7efc29312700 1 --2- 172.21.15.134:0/3666036636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc24103890 0x7efc10005610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:32.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.956+0000 7efc29312700 1 -- 172.21.15.134:0/3666036636 >> 172.21.15.134:0/3666036636 conn(0x7efc240fd040 msgr2=0x7efc240ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:32.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.956+0000 7efc29312700 1 -- 172.21.15.134:0/3666036636 shutdown_connections 2024-04-16T05:26:32.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.956+0000 7efc29312700 1 -- 172.21.15.134:0/3666036636 wait complete. 2024-04-16T05:26:32.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.957+0000 7efc29312700 1 Processor -- start 2024-04-16T05:26:32.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.957+0000 7efc29312700 1 -- start start 2024-04-16T05:26:32.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc29312700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 0x7efc241981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:32.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc29312700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 0x7efc2419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:32.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc29312700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc2419cfd0 0x7efc2419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc29312700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410a680 con 0x7efc2419cfd0 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc29312700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410a500 con 0x7efc24198700 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc29312700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410a800 con 0x7efc24103890 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc22ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 0x7efc241981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc2419cfd0 0x7efc2419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc22ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 0x7efc241981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:34334/0 (socket says 172.21.15.134:34334) 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc22ffd700 1 -- 172.21.15.134:0/1076221019 learned_addr learned my addr 172.21.15.134:0/1076221019 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc22ffd700 1 -- 172.21.15.134:0/1076221019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 msgr2=0x7efc241981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc22ffd700 1 -- 172.21.15.134:0/1076221019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 msgr2=0x7efc241981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc22ffd700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 0x7efc241981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc22ffd700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 0x7efc241981c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc1a7fc700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 0x7efc2419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:32.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc237fe700 1 -- 172.21.15.134:0/1076221019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 msgr2=0x7efc241981c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:26:32.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc237fe700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 0x7efc241981c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:32.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc237fe700 1 -- 172.21.15.134:0/1076221019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 msgr2=0x7efc2419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:32.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc1a7fc700 1 -- 172.21.15.134:0/1076221019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 msgr2=0x7efc2419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-16T05:26:32.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc1a7fc700 1 -- 172.21.15.134:0/1076221019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 msgr2=0x7efc2419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T05:26:32.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc1a7fc700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 0x7efc2419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T05:26:32.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc1a7fc700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 0x7efc2419ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:26:32.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc237fe700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 0x7efc2419ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:32.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.958+0000 7efc237fe700 1 -- 172.21.15.134:0/1076221019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc241a4d40 con 0x7efc2419cfd0 2024-04-16T05:26:32.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.959+0000 7efc237fe700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc2419cfd0 0x7efc2419d480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00bd50 tx=0x7efc1c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:32.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.959+0000 7efc20ff9700 1 -- 172.21.15.134:0/1076221019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c019070 con 0x7efc2419cfd0 2024-04-16T05:26:32.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.959+0000 7efc20ff9700 1 -- 172.21.15.134:0/1076221019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c00cd10 con 0x7efc2419cfd0 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.959+0000 7efc20ff9700 1 -- 172.21.15.134:0/1076221019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0127e0 con 0x7efc2419cfd0 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.959+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241a4f70 con 0x7efc2419cfd0 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.959+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241a5400 con 0x7efc2419cfd0 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.960+0000 7efc20ff9700 1 -- 172.21.15.134:0/1076221019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efc1c01c070 con 0x7efc2419cfd0 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.960+0000 7efc20ff9700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efc04039ae0 0x7efc0403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.961+0000 7efc20ff9700 1 -- 172.21.15.134:0/1076221019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efc1c04c120 con 0x7efc2419cfd0 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.961+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc24064150 con 0x7efc2419cfd0 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.961+0000 7efc22ffd700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efc04039ae0 0x7efc0403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.961+0000 7efc22ffd700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efc04039ae0 0x7efc0403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efc1400fad0 tx=0x7efc14005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:32.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:32.966+0000 7efc20ff9700 1 -- 172.21.15.134:0/1076221019 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc1c013c30 con 0x7efc2419cfd0 2024-04-16T05:26:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: mon.a calling monitor election 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: mon.c calling monitor election 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: mon.b calling monitor election 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T05:26:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: monmap epoch 3 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: last_changed 2024-04-16T05:26:27.770640+0000 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: created 2024-04-16T05:24:34.788250+0000 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: min_mon_release 19 (squid) 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: election_strategy: 1 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: fsmap 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: mgrmap e13: x(active, since 59s) 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: overall HEALTH_OK 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.137 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: mon.a calling monitor election 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: mon.c calling monitor election 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: mon.b calling monitor election 2024-04-16T05:26:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: monmap epoch 3 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: last_changed 2024-04-16T05:26:27.770640+0000 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: created 2024-04-16T05:24:34.788250+0000 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: min_mon_release 19 (squid) 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: election_strategy: 1 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-04-16T05:26:33.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: fsmap 2024-04-16T05:26:33.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:33.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: mgrmap e13: x(active, since 59s) 2024-04-16T05:26:33.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: overall HEALTH_OK 2024-04-16T05:26:33.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: mon.a calling monitor election 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: mon.c calling monitor election 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: mon.b calling monitor election 2024-04-16T05:26:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: monmap epoch 3 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: last_changed 2024-04-16T05:26:27.770640+0000 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: created 2024-04-16T05:24:34.788250+0000 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: min_mon_release 19 (squid) 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: election_strategy: 1 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-04-16T05:26:33.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: fsmap 2024-04-16T05:26:33.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T05:26:33.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: mgrmap e13: x(active, since 59s) 2024-04-16T05:26:33.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: overall HEALTH_OK 2024-04-16T05:26:33.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:33.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.265+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efc2402dfe0 con 0x7efc2419cfd0 2024-04-16T05:26:33.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.266+0000 7efc20ff9700 1 -- 172.21.15.134:0/1076221019 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7efc1c01d730 con 0x7efc2419cfd0 2024-04-16T05:26:33.265 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-16T05:26:33.265 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","modified":"2024-04-16T05:26:27.770640Z","created":"2024-04-16T05:24:34.788250Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-16T05:26:33.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efc04039ae0 msgr2=0x7efc0403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:33.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efc04039ae0 0x7efc0403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efc1400fad0 tx=0x7efc14005c00 comp rx=0 tx=0).stop 2024-04-16T05:26:33.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc2419cfd0 msgr2=0x7efc2419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:33.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc2419cfd0 0x7efc2419d480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00bd50 tx=0x7efc1c00be30 comp rx=0 tx=0).stop 2024-04-16T05:26:33.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 shutdown_connections 2024-04-16T05:26:33.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efc04039ae0 0x7efc0403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:33.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc2419cfd0 0x7efc2419d480 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:33.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc24103890 0x7efc241981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:33.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 --2- 172.21.15.134:0/1076221019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc24198700 0x7efc2419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:33.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 >> 172.21.15.134:0/1076221019 conn(0x7efc240fd040 msgr2=0x7efc240fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:33.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 shutdown_connections 2024-04-16T05:26:33.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:33.270+0000 7efc29312700 1 -- 172.21.15.134:0/1076221019 wait complete. 2024-04-16T05:26:33.270 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 3 2024-04-16T05:26:33.890 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-16T05:26:33.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph config generate-minimal-conf 2024-04-16T05:26:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:33 smithi134 ceph-mon[36153]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:33 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/1076221019' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:33 smithi082 ceph-mon[36195]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:33 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/1076221019' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:34.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:33 smithi032 ceph-mon[32351]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:34.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:33 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/1076221019' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T05:26:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: Updating smithi032:/etc/ceph/ceph.conf 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-16T05:26:35.449 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:35 smithi032 ceph-mon[32351]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-16T05:26:35.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:35.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:35.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: Updating smithi032:/etc/ceph/ceph.conf 2024-04-16T05:26:35.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-16T05:26:35.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:35 smithi134 ceph-mon[36153]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: Updating smithi032:/etc/ceph/ceph.conf 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-16T05:26:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:35 smithi082 ceph-mon[36195]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-16T05:26:35.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.666+0000 7fb705268700 1 -- 172.21.15.32:0/3469449202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 msgr2=0x7fb700108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:35.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.666+0000 7fb705268700 1 --2- 172.21.15.32:0/3469449202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 0x7fb700108960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009910 tx=0x7fb6f0009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:35.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.666+0000 7fb705268700 1 -- 172.21.15.32:0/3469449202 shutdown_connections 2024-04-16T05:26:35.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.666+0000 7fb705268700 1 --2- 172.21.15.32:0/3469449202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 0x7fb700108960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:35.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.666+0000 7fb705268700 1 -- 172.21.15.32:0/3469449202 >> 172.21.15.32:0/3469449202 conn(0x7fb700076f50 msgr2=0x7fb700077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:35.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.666+0000 7fb705268700 1 -- 172.21.15.32:0/3469449202 shutdown_connections 2024-04-16T05:26:35.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.666+0000 7fb705268700 1 -- 172.21.15.32:0/3469449202 wait complete. 2024-04-16T05:26:35.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.667+0000 7fb705268700 1 Processor -- start 2024-04-16T05:26:35.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.667+0000 7fb705268700 1 -- start start 2024-04-16T05:26:35.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.667+0000 7fb705268700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 0x7fb700198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:35.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.667+0000 7fb705268700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb700198760 0x7fb70019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:35.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.667+0000 7fb705268700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb70019d030 0x7fb70019d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:35.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.667+0000 7fb705268700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb70010ca70 con 0x7fb700108580 2024-04-16T05:26:35.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.667+0000 7fb705268700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb70010c8f0 con 0x7fb70019d030 2024-04-16T05:26:35.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.667+0000 7fb705268700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb70010cbf0 con 0x7fb700198760 2024-04-16T05:26:35.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 0x7fb700198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:35.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 0x7fb700198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54272/0 (socket says 172.21.15.32:54272) 2024-04-16T05:26:35.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 -- 172.21.15.32:0/1502994794 learned_addr learned my addr 172.21.15.32:0/1502994794 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:26:35.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6ff7fe700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb70019d030 0x7fb70019d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:35.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6f67fc700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb700198760 0x7fb70019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:35.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 -- 172.21.15.32:0/1502994794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb700198760 msgr2=0x7fb70019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:35.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb700198760 0x7fb70019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 -- 172.21.15.32:0/1502994794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb70019d030 msgr2=0x7fb70019d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb70019d030 0x7fb70019d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 -- 172.21.15.32:0/1502994794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7001a4da0 con 0x7fb700108580 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6f67fc700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb700198760 0x7fb70019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6feffd700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 0x7fb700198220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000bdc0 tx=0x7fb6f000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6fcff9700 1 -- 172.21.15.32:0/1502994794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0027070 con 0x7fb700108580 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7001a4fd0 con 0x7fb700108580 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6fcff9700 1 -- 172.21.15.32:0/1502994794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f00050d0 con 0x7fb700108580 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb6fcff9700 1 -- 172.21.15.32:0/1502994794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0022dc0 con 0x7fb700108580 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.668+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7001a5430 con 0x7fb700108580 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.669+0000 7fb6fcff9700 1 -- 172.21.15.32:0/1502994794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb6f002a070 con 0x7fb700108580 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.669+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb700064150 con 0x7fb700108580 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.669+0000 7fb6fcff9700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6ec039ae0 0x7fb6ec03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.670+0000 7fb6fcff9700 1 -- 172.21.15.32:0/1502994794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb6f004e7d0 con 0x7fb700108580 2024-04-16T05:26:35.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.670+0000 7fb6f67fc700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6ec039ae0 0x7fb6ec03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:35.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.670+0000 7fb6f67fc700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6ec039ae0 0x7fb6ec03bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009910 tx=0x7fb6e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:35.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.675+0000 7fb6fcff9700 1 -- 172.21.15.32:0/1502994794 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6f0013070 con 0x7fb700108580 2024-04-16T05:26:35.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.888+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb7001a5750 con 0x7fb700108580 2024-04-16T05:26:35.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.888+0000 7fb6fcff9700 1 -- 172.21.15.32:0/1502994794 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fb6f0031090 con 0x7fb700108580 2024-04-16T05:26:35.887 INFO:teuthology.orchestra.run.smithi032.stdout:# minimal ceph.conf for 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:35.888 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-16T05:26:35.888 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:26:35.888 INFO:teuthology.orchestra.run.smithi032.stdout: mon_host = [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-04-16T05:26:35.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6ec039ae0 msgr2=0x7fb6ec03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6ec039ae0 0x7fb6ec03bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009910 tx=0x7fb6e8008040 comp rx=0 tx=0).stop 2024-04-16T05:26:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 msgr2=0x7fb700198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 0x7fb700198220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000bdc0 tx=0x7fb6f000bdf0 comp rx=0 tx=0).stop 2024-04-16T05:26:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 shutdown_connections 2024-04-16T05:26:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6ec039ae0 0x7fb6ec03bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb70019d030 0x7fb70019d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb700108580 0x7fb700198220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 --2- 172.21.15.32:0/1502994794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb700198760 0x7fb70019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 >> 172.21.15.32:0/1502994794 conn(0x7fb700076f50 msgr2=0x7fb7000fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 shutdown_connections 2024-04-16T05:26:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:35.891+0000 7fb705268700 1 -- 172.21.15.32:0/1502994794 wait complete. 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: Reconfiguring daemon mon.a on smithi032 2024-04-16T05:26:36.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:36 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1502994794' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T05:26:36.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:36.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:36.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: Reconfiguring daemon mon.a on smithi032 2024-04-16T05:26:36.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:36 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1502994794' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:36.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:26:36.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: Reconfiguring daemon mon.a on smithi032 2024-04-16T05:26:36.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:36 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1502994794' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:36.701 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-16T05:26:36.702 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:26:36.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T05:26:36.734 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:26:36.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:36.804 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:26:36.804 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T05:26:36.839 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:26:36.839 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:36.909 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:26:36.909 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T05:26:36.943 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:26:36.943 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:26:37.014 INFO:tasks.cephadm:Adding mgr.x on smithi032 2024-04-16T05:26:37.015 INFO:tasks.cephadm:Adding mgr.y on smithi082 2024-04-16T05:26:37.015 INFO:tasks.cephadm:Adding mgr.z on smithi134 2024-04-16T05:26:37.015 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch apply mgr '3;smithi032=x;smithi082=y;smithi134=z' 2024-04-16T05:26:37.289 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:26:38.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:38 smithi134 ceph-mon[36153]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:38.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:38 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:38.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:38 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:38.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:38 smithi134 ceph-mon[36153]: Reconfiguring mon.b (monmap changed)... 2024-04-16T05:26:38.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:38 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:38.711 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:38 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:38.711 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:38 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:38.711 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:38 smithi134 ceph-mon[36153]: Reconfiguring daemon mon.b on smithi082 2024-04-16T05:26:38.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.730+0000 7f6a251d3700 1 -- 172.21.15.134:0/3078756112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a20103c60 msgr2=0x7f6a20104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:38.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.730+0000 7f6a251d3700 1 --2- 172.21.15.134:0/3078756112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a20103c60 0x7f6a20104080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009910 tx=0x7f6a10009c20 comp rx=0 tx=0).stop 2024-04-16T05:26:38.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.730+0000 7f6a251d3700 1 -- 172.21.15.134:0/3078756112 shutdown_connections 2024-04-16T05:26:38.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.730+0000 7f6a251d3700 1 --2- 172.21.15.134:0/3078756112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a20103c60 0x7f6a20104080 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:38.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.730+0000 7f6a251d3700 1 -- 172.21.15.134:0/3078756112 >> 172.21.15.134:0/3078756112 conn(0x7f6a200fc2a0 msgr2=0x7f6a200fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:38.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.730+0000 7f6a251d3700 1 -- 172.21.15.134:0/3078756112 shutdown_connections 2024-04-16T05:26:38.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.730+0000 7f6a251d3700 1 -- 172.21.15.134:0/3078756112 wait complete. 2024-04-16T05:26:38.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.730+0000 7f6a251d3700 1 Processor -- start 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a251d3700 1 -- start start 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a251d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a20103c60 0x7f6a20197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a251d3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a20198380 0x7f6a2019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a251d3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20198800 0x7f6a2019f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a251d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20108660 con 0x7f6a20103c60 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a251d3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6a201084e0 con 0x7f6a20198800 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a251d3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a201087e0 con 0x7f6a20198380 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a20198380 0x7f6a2019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a20198380 0x7f6a2019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:45310/0 (socket says 172.21.15.134:45310) 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 -- 172.21.15.134:0/2983312846 learned_addr learned my addr 172.21.15.134:0/2983312846 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1f59e700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20198800 0x7f6a2019f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1ed9d700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a20103c60 0x7f6a20197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:38.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 -- 172.21.15.134:0/2983312846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20198800 msgr2=0x7f6a2019f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20198800 0x7f6a2019f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 -- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a20103c60 msgr2=0x7f6a20197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a20103c60 0x7f6a20197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 -- 172.21.15.134:0/2983312846 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a201a0250 con 0x7f6a20198380 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.731+0000 7f6a1e59c700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a20198380 0x7f6a2019d3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400b780 tx=0x7f6a1400bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.732+0000 7f6a07fff700 1 -- 172.21.15.134:0/2983312846 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14004230 con 0x7f6a20198380 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.732+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a201a0540 con 0x7f6a20198380 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.732+0000 7f6a07fff700 1 -- 172.21.15.134:0/2983312846 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a140043d0 con 0x7f6a20198380 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.732+0000 7f6a07fff700 1 -- 172.21.15.134:0/2983312846 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14011c60 con 0x7f6a20198380 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.732+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a201a0b50 con 0x7f6a20198380 2024-04-16T05:26:38.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.733+0000 7f6a07fff700 1 -- 172.21.15.134:0/2983312846 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6a1401a400 con 0x7f6a20198380 2024-04-16T05:26:38.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.733+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a20064150 con 0x7f6a20198380 2024-04-16T05:26:38.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.733+0000 7f6a07fff700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6a08039ae0 0x7f6a0803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:38.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.733+0000 7f6a07fff700 1 -- 172.21.15.134:0/2983312846 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6a1404c4c0 con 0x7f6a20198380 2024-04-16T05:26:38.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.733+0000 7f6a1ed9d700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6a08039ae0 0x7f6a0803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:38.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.733+0000 7f6a1ed9d700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6a08039ae0 0x7f6a0803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000b5c0 tx=0x7f6a1001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:38.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.738+0000 7f6a07fff700 1 -- 172.21.15.134:0/2983312846 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a14020500 con 0x7f6a20198380 2024-04-16T05:26:38.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:38 smithi082 ceph-mon[36195]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:38.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:38 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:38.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:38 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:38.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:38 smithi082 ceph-mon[36195]: Reconfiguring mon.b (monmap changed)... 2024-04-16T05:26:38.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:38 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:38.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:38 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:38.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:38 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:38.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:38 smithi082 ceph-mon[36195]: Reconfiguring daemon mon.b on smithi082 2024-04-16T05:26:38.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.946+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi082=y;smithi134=z", "target": ["mon-mgr", ""]}) -- 0x7f6a20062fc0 con 0x7f6a08039ae0 2024-04-16T05:26:38.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.952+0000 7f6a07fff700 1 -- 172.21.15.134:0/2983312846 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6a20062fc0 con 0x7f6a08039ae0 2024-04-16T05:26:38.951 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mgr update... 2024-04-16T05:26:38.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6a08039ae0 msgr2=0x7f6a0803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6a08039ae0 0x7f6a0803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000b5c0 tx=0x7f6a1001a040 comp rx=0 tx=0).stop 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a20198380 msgr2=0x7f6a2019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a20198380 0x7f6a2019d3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400b780 tx=0x7f6a1400bb40 comp rx=0 tx=0).stop 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 shutdown_connections 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6a08039ae0 0x7f6a0803bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20198800 0x7f6a2019f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a20103c60 0x7f6a20197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 --2- 172.21.15.134:0/2983312846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a20198380 0x7f6a2019d3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 >> 172.21.15.134:0/2983312846 conn(0x7f6a200fc2a0 msgr2=0x7f6a200fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 shutdown_connections 2024-04-16T05:26:38.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:26:38.955+0000 7f6a251d3700 1 -- 172.21.15.134:0/2983312846 wait complete. 2024-04-16T05:26:38.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:38 smithi032 ceph-mon[32351]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:38.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:38 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:38.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:38 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:38.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:38 smithi032 ceph-mon[32351]: Reconfiguring mon.b (monmap changed)... 2024-04-16T05:26:38.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:38 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:38.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:38 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:38.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:38 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:38.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:38 smithi032 ceph-mon[32351]: Reconfiguring daemon mon.b on smithi082 2024-04-16T05:26:39.576 DEBUG:teuthology.orchestra.run.smithi082:mgr.y> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.y.service 2024-04-16T05:26:39.578 DEBUG:teuthology.orchestra.run.smithi134:mgr.z> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.z.service 2024-04-16T05:26:39.580 INFO:tasks.cephadm:Deploying OSDs... 2024-04-16T05:26:39.580 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:26:39.580 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T05:26:39.601 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T05:26:39.601 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-04-16T05:26:39.660 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T05:26:39.660 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-16T05:26:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-16 05:25:47.016758392 +0000 2024-04-16T05:26:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-16 05:18:17.979987662 +0000 2024-04-16T05:26:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-16 05:18:17.979987662 +0000 2024-04-16T05:26:39.661 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-16 05:18:17.979987662 +0000 2024-04-16T05:26:39.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T05:26:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-16T05:26:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-16T05:26:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.00012595 s, 4.1 MB/s 2024-04-16T05:26:39.734 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T05:26:39.792 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-04-16T05:26:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T05:26:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:39.850 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-16T05:26:39.850 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:39.850 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:39.850 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-16 05:25:47.051757671 +0000 2024-04-16T05:26:39.850 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-16 05:18:18.323980699 +0000 2024-04-16T05:26:39.850 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-16 05:18:18.323980699 +0000 2024-04-16T05:26:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-16 05:18:18.323980699 +0000 2024-04-16T05:26:39.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T05:26:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-16T05:26:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-16T05:26:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000112965 s, 4.5 MB/s 2024-04-16T05:26:39.920 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T05:26:39.978 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi082=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: Saving service mgr spec with placement smithi032=x;smithi082=y;smithi134=z;count:3 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: Reconfiguring mon.c (monmap changed)... 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:39.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:39 smithi032 ceph-mon[32351]: Reconfiguring daemon mon.c on smithi134 2024-04-16T05:26:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T05:26:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:39.998 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-16T05:26:39.998 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:39.998 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:39.999 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-16 05:25:47.089756888 +0000 2024-04-16T05:26:39.999 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-16 05:18:18.671973655 +0000 2024-04-16T05:26:39.999 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-16 05:18:18.671973655 +0000 2024-04-16T05:26:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-16 05:18:18.671973655 +0000 2024-04-16T05:26:39.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T05:26:40.066 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-16T05:26:40.066 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-16T05:26:40.066 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000109375 s, 4.7 MB/s 2024-04-16T05:26:40.067 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T05:26:40.124 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-04-16T05:26:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi082=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: Saving service mgr spec with placement smithi032=x;smithi082=y;smithi134=z;count:3 2024-04-16T05:26:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: Reconfiguring mon.c (monmap changed)... 2024-04-16T05:26:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:39 smithi134 ceph-mon[36153]: Reconfiguring daemon mon.c on smithi134 2024-04-16T05:26:40.179 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T05:26:40.180 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:40.180 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-04-16T05:26:40.180 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:40.180 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:40.180 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-16 05:25:47.123756188 +0000 2024-04-16T05:26:40.180 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-16 05:18:19.015966692 +0000 2024-04-16T05:26:40.180 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-16 05:18:19.015966692 +0000 2024-04-16T05:26:40.180 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-16 05:18:19.015966692 +0000 2024-04-16T05:26:40.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T05:26:40.247 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-16T05:26:40.247 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-16T05:26:40.247 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.2919e-05 s, 5.5 MB/s 2024-04-16T05:26:40.248 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T05:26:40.304 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-16T05:26:40.305 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T05:26:40.323 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T05:26:40.323 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-16 05:26:27.062306865 +0000 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-16 05:18:31.832129919 +0000 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-16 05:18:31.832129919 +0000 2024-04-16T05:26:40.380 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-16 05:18:31.832129919 +0000 2024-04-16T05:26:40.380 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi082=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: Saving service mgr spec with placement smithi032=x;smithi082=y;smithi134=z;count:3 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: Reconfiguring mon.c (monmap changed)... 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:39 smithi082 ceph-mon[36195]: Reconfiguring daemon mon.c on smithi134 2024-04-16T05:26:40.428 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-16T05:26:40.428 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-16T05:26:40.428 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000286286 s, 1.8 MB/s 2024-04-16T05:26:40.429 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T05:26:40.486 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-04-16T05:26:40.542 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T05:26:40.542 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:40.542 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-16T05:26:40.542 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:40.543 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:40.543 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-16 05:26:27.063306846 +0000 2024-04-16T05:26:40.543 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-16 05:18:32.174122867 +0000 2024-04-16T05:26:40.543 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-16 05:18:32.174122867 +0000 2024-04-16T05:26:40.543 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-16 05:18:32.174122867 +0000 2024-04-16T05:26:40.543 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T05:26:40.609 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-16T05:26:40.609 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-16T05:26:40.609 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.00020425 s, 2.5 MB/s 2024-04-16T05:26:40.610 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T05:26:40.667 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-04-16T05:26:40.721 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T05:26:40.722 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:40.722 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-04-16T05:26:40.722 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:40.722 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:40.722 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-16 05:26:27.063306846 +0000 2024-04-16T05:26:40.722 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-16 05:18:32.515115836 +0000 2024-04-16T05:26:40.722 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-16 05:18:32.515115836 +0000 2024-04-16T05:26:40.722 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-16 05:18:32.515115836 +0000 2024-04-16T05:26:40.722 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T05:26:40.787 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-16T05:26:40.787 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-16T05:26:40.788 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000193524 s, 2.6 MB/s 2024-04-16T05:26:40.789 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T05:26:40.845 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-04-16T05:26:40.900 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T05:26:40.900 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:40.900 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-16T05:26:40.901 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:40.901 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:40.901 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-16 05:26:27.063306846 +0000 2024-04-16T05:26:40.901 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-16 05:18:32.850108927 +0000 2024-04-16T05:26:40.901 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-16 05:18:32.850108927 +0000 2024-04-16T05:26:40.901 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-16 05:18:32.850108927 +0000 2024-04-16T05:26:40.901 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T05:26:40.966 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-16T05:26:40.966 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-16T05:26:40.966 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000203262 s, 2.5 MB/s 2024-04-16T05:26:40.967 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T05:26:41.023 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-16T05:26:41.023 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T05:26:41.039 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T05:26:41.039 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-04-16T05:26:41.093 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T05:26:41.094 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:41.094 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-04-16T05:26:41.094 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:41.094 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:41.094 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-16 05:26:17.748860310 +0000 2024-04-16T05:26:41.094 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-16 05:19:07.618528717 +0000 2024-04-16T05:26:41.094 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-16 05:19:07.618528717 +0000 2024-04-16T05:26:41.094 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-16 05:19:07.618528717 +0000 2024-04-16T05:26:41.094 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T05:26:41.160 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-16T05:26:41.160 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-16T05:26:41.160 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000273924 s, 1.9 MB/s 2024-04-16T05:26:41.161 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T05:26:41.217 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-04-16T05:26:41.271 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T05:26:41.271 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:41.271 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-04-16T05:26:41.271 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:41.271 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:41.271 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-16 05:26:17.748860310 +0000 2024-04-16T05:26:41.271 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-16 05:19:07.973520383 +0000 2024-04-16T05:26:41.272 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-16 05:19:07.973520383 +0000 2024-04-16T05:26:41.272 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-16 05:19:07.973520383 +0000 2024-04-16T05:26:41.272 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T05:26:41.337 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-16T05:26:41.337 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-16T05:26:41.337 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000206929 s, 2.5 MB/s 2024-04-16T05:26:41.338 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T05:26:41.394 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-16 05:26:17.749860287 +0000 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-16 05:19:08.324512143 +0000 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-16 05:19:08.324512143 +0000 2024-04-16T05:26:41.449 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-16 05:19:08.324512143 +0000 2024-04-16T05:26:41.449 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T05:26:41.514 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-16T05:26:41.514 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-16T05:26:41.514 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000221166 s, 2.3 MB/s 2024-04-16T05:26:41.515 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T05:26:41.572 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-04-16T05:26:41.627 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T05:26:41.627 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T05:26:41.627 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-16T05:26:41.628 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:26:41.628 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T05:26:41.628 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-16 05:26:17.749860287 +0000 2024-04-16T05:26:41.628 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-16 05:19:08.650504490 +0000 2024-04-16T05:26:41.628 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-16 05:19:08.650504490 +0000 2024-04-16T05:26:41.628 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-16 05:19:08.650504490 +0000 2024-04-16T05:26:41.628 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T05:26:41.693 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-16T05:26:41.693 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-16T05:26:41.693 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000195235 s, 2.6 MB/s 2024-04-16T05:26:41.694 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T05:26:41.750 INFO:tasks.cephadm:Deploying osd.0 on smithi032 with /dev/vg_nvme/lv_4... 2024-04-16T05:26:41.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T05:26:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:42.112 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:42.112 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.112 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.112 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:42.112 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:42.113 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:42.113 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.113 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:42.113 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:42.113 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:42.113 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:42.113 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:41 smithi032 ceph-mon[32351]: Deploying daemon mgr.z on smithi134 2024-04-16T05:26:42.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:42.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:42.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:42.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:42.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:42.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:42.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:42.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:42.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:41 smithi134 ceph-mon[36153]: Deploying daemon mgr.z on smithi134 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:42.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:42.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:41 smithi082 ceph-mon[36195]: Deploying daemon mgr.z on smithi134 2024-04-16T05:26:43.597 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:43 smithi134 systemd[1]: Starting Ceph mgr.z for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:26:43.857 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:43 smithi134 podman[38025]: 2024-04-16 05:26:43.598229196 +0000 UTC m=+0.113054813 container create cea3fb9e5da54b24b661f7a6e960dd290f72e8128343e40595d42cdfa7de1913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-16T05:26:43.857 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:43 smithi134 podman[38025]: 2024-04-16 05:26:43.502982547 +0000 UTC m=+0.017808158 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:26:43.857 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:43 smithi134 podman[38025]: 2024-04-16 05:26:43.817284383 +0000 UTC m=+0.332109971 container init cea3fb9e5da54b24b661f7a6e960dd290f72e8128343e40595d42cdfa7de1913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-04-16T05:26:43.857 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:43 smithi134 podman[38025]: 2024-04-16 05:26:43.820111703 +0000 UTC m=+0.334937302 container start cea3fb9e5da54b24b661f7a6e960dd290f72e8128343e40595d42cdfa7de1913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-16T05:26:43.857 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:43 smithi134 ceph-mgr[38044]: -- 172.21.15.134:0/1679578875 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625b6fca5a0 con 0x5625b6f9ec00 2024-04-16T05:26:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T05:26:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T05:26:43.917 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-04-16T05:26:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-04-16T05:26:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00806669 s, 1.3 GB/s 2024-04-16T05:26:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-04-16T05:26:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:43 smithi134 ceph-mon[36153]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:44.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:43 smithi134 bash[38025]: cea3fb9e5da54b24b661f7a6e960dd290f72e8128343e40595d42cdfa7de1913 2024-04-16T05:26:44.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:43 smithi134 systemd[1]: Started Ceph mgr.z for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:26:44.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:43 smithi082 ceph-mon[36195]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:43 smithi032 ceph-mon[32351]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:44.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_4 2024-04-16T05:26:44.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:44.357+0000 7f498924c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:26:44.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:44.455+0000 7f498924c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:26:44.823 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:26:45.196 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.197 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.197 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.197 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:45.197 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:45.197 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:45.197 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:45.197 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:44 smithi032 ceph-mon[32351]: Deploying daemon mgr.y on smithi082 2024-04-16T05:26:45.321 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:45.094+0000 7f498924c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:26:45.321 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:45.184+0000 7f498924c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:45.322+0000 7f498924c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:45.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:44 smithi134 ceph-mon[36153]: Deploying daemon mgr.y on smithi082 2024-04-16T05:26:45.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:45.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:45.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T05:26:45.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:45.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:45.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:44 smithi082 ceph-mon[36195]: Deploying daemon mgr.y on smithi082 2024-04-16T05:26:45.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:45.415+0000 7f498924c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:26:45.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:45.496+0000 7f498924c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:26:46.031 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:45.754+0000 7f498924c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:26:46.032 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:45.861+0000 7f498924c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:26:46.032 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:45.944+0000 7f498924c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:26:46.046 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:46 smithi032 ceph-mon[32351]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:46.275 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:46 smithi082 ceph-mon[36195]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:46.275 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:46 smithi082 podman[37558]: 2024-04-16 05:26:46.277308463 +0000 UTC m=+0.145487656 container create d58fb8e1390f6ebf13c0c7544e86290b52b5b454048c459f3ea775a6dbcaace8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:26:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 -- 172.21.15.32:0/1942539864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 msgr2=0x7f27501041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 --2- 172.21.15.32:0/1942539864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 0x7f27501041c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27440099c0 tx=0x7f2744009cd0 comp rx=0 tx=0).stop 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 -- 172.21.15.32:0/1942539864 shutdown_connections 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 --2- 172.21.15.32:0/1942539864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2750104700 0x7f275010afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 --2- 172.21.15.32:0/1942539864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2750102b60 0x7f2750102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 --2- 172.21.15.32:0/1942539864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 0x7f27501041c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 -- 172.21.15.32:0/1942539864 >> 172.21.15.32:0/1942539864 conn(0x7f27500fe100 msgr2=0x7f2750100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 -- 172.21.15.32:0/1942539864 shutdown_connections 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 -- 172.21.15.32:0/1942539864 wait complete. 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.280+0000 7f275574c700 1 Processor -- start 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f275574c700 1 -- start start 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f275574c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2750102b60 0x7f2750076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f275574c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 0x7f2750074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f275574c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2750104700 0x7f2750075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f275574c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f275010e7a0 con 0x7f2750103d60 2024-04-16T05:26:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f275574c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f275010e620 con 0x7f2750104700 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f275574c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f275010e920 con 0x7f2750102b60 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 0x7f2750074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 0x7f2750074ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41738/0 (socket says 172.21.15.32:41738) 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 -- 172.21.15.32:0/2654164201 learned_addr learned my addr 172.21.15.32:0/2654164201 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274effd700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2750102b60 0x7f2750076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274f7fe700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2750104700 0x7f2750075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 -- 172.21.15.32:0/2654164201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2750102b60 msgr2=0x7f2750076890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2750102b60 0x7f2750076890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 -- 172.21.15.32:0/2654164201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2750104700 msgr2=0x7f2750075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2750104700 0x7f2750075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 -- 172.21.15.32:0/2654164201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2750075cd0 con 0x7f2750103d60 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274f7fe700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2750104700 0x7f2750075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274effd700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2750102b60 0x7f2750076890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:26:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.281+0000 7f274e7fc700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 0x7f2750074ee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f274400fa00 tx=0x7f274400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.282+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744027070 con 0x7f2750103d60 2024-04-16T05:26:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.282+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27501a2e90 con 0x7f2750103d60 2024-04-16T05:26:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.282+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2744022950 con 0x7f2750103d60 2024-04-16T05:26:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.282+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744017650 con 0x7f2750103d60 2024-04-16T05:26:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.282+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27501a3230 con 0x7f2750103d60 2024-04-16T05:26:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.283+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f274402a070 con 0x7f2750103d60 2024-04-16T05:26:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.283+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27500697e0 con 0x7f2750103d60 2024-04-16T05:26:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.283+0000 7f2737fff700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2738039890 0x7f273803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:26:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.283+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f274404d790 con 0x7f2750103d60 2024-04-16T05:26:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.283+0000 7f274effd700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2738039890 0x7f273803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:26:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.283+0000 7f274effd700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2738039890 0x7f273803bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2750103bc0 tx=0x7f2740007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:26:46.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.288+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2744013070 con 0x7f2750103d60 2024-04-16T05:26:46.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:46 smithi134 ceph-mon[36153]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:46.384 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:46 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:46.037+0000 7f498924c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:26:46.384 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:46 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:46.211+0000 7f498924c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:26:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:46.531+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2750062f30 con 0x7f2738039890 2024-04-16T05:26:46.561 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:46 smithi082 podman[37558]: 2024-04-16 05:26:46.479668681 +0000 UTC m=+0.347847885 container init d58fb8e1390f6ebf13c0c7544e86290b52b5b454048c459f3ea775a6dbcaace8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-16T05:26:46.561 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:46 smithi082 podman[37558]: 2024-04-16 05:26:46.482966536 +0000 UTC m=+0.351145740 container start d58fb8e1390f6ebf13c0c7544e86290b52b5b454048c459f3ea775a6dbcaace8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:26:46.561 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:46 smithi082 ceph-mgr[37585]: -- 172.21.15.82:0/1632480105 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55939ca085a0 con 0x55939c9dd000 2024-04-16T05:26:46.900 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:46 smithi082 bash[37558]: d58fb8e1390f6ebf13c0c7544e86290b52b5b454048c459f3ea775a6dbcaace8 2024-04-16T05:26:46.900 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:46 smithi082 systemd[1]: Started Ceph mgr.y for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:26:47.334 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:47.195+0000 7f498924c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:47.335+0000 7f498924c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.335 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:47.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:47.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:26:47.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:26:47.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:47.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.337 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.337 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.337 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.337 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:47.399 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:47.054+0000 7f454163e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:47.151+0000 7f454163e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:47.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:47.988 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:47.699+0000 7f498924c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:26:47.988 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:47.789+0000 7f498924c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:26:47.988 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:47.988+0000 7f498924c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:26:48.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:47.830+0000 7f454163e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:26:48.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:47 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:47.923+0000 7f454163e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:26:48.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:48.068+0000 7f454163e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:26:48.384 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:48.180+0000 7f498924c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:26:48.503 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:48.165+0000 7f454163e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:26:48.503 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:48.249+0000 7f454163e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:26:48.504 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:48.504+0000 7f454163e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:26:48.725 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:48.466+0000 7f498924c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:26:48.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:48.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:48.784 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:48.610+0000 7f454163e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:26:48.784 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:48.694+0000 7f454163e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:48.785+0000 7f454163e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:48.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:48.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T05:26:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:49.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:48.726+0000 7f498924c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:26:49.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:48.960+0000 7f498924c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:26:49.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:48.957+0000 7f454163e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:26:49.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:49 smithi032 ceph-mon[32351]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T05:26:49.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:49 smithi032 ceph-mon[32351]: Reconfiguring daemon mgr.x on smithi032 2024-04-16T05:26:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:49 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3659466044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3646a416-b8ca-4d84-9f20-cb3b2cb2273d"}]: dispatch 2024-04-16T05:26:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:49 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3659466044' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3646a416-b8ca-4d84-9f20-cb3b2cb2273d"}]': finished 2024-04-16T05:26:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:49 smithi032 ceph-mon[32351]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T05:26:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:26:50.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:49 smithi082 ceph-mon[36195]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T05:26:50.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:49 smithi082 ceph-mon[36195]: Reconfiguring daemon mgr.x on smithi032 2024-04-16T05:26:50.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:49 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3659466044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3646a416-b8ca-4d84-9f20-cb3b2cb2273d"}]: dispatch 2024-04-16T05:26:50.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:49 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3659466044' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3646a416-b8ca-4d84-9f20-cb3b2cb2273d"}]': finished 2024-04-16T05:26:50.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:49 smithi082 ceph-mon[36195]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T05:26:50.040 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:49 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:26:50.040 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:49 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:49.910+0000 7f454163e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:26:50.040 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:50.040+0000 7f454163e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:26:50.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:49 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:49.754+0000 7f498924c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:26:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:49 smithi134 ceph-mon[36153]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T05:26:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:49 smithi134 ceph-mon[36153]: Reconfiguring daemon mgr.x on smithi032 2024-04-16T05:26:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:49 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3659466044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3646a416-b8ca-4d84-9f20-cb3b2cb2273d"}]: dispatch 2024-04-16T05:26:50.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:49 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3659466044' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3646a416-b8ca-4d84-9f20-cb3b2cb2273d"}]': finished 2024-04-16T05:26:50.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:49 smithi134 ceph-mon[36153]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T05:26:50.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:49 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:26:50.629 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:50.320+0000 7f498924c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:26:50.630 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:50.416+0000 7f498924c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:26:50.630 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:26:50.630+0000 7f498924c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:26:50.649 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:50.389+0000 7f454163e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:26:50.650 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:50.477+0000 7f454163e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:26:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:50.668+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f2744017d10 con 0x7f2750103d60 2024-04-16T05:26:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-mon[36153]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1073751163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:26:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-mon[36153]: Standby manager daemon z started 2024-04-16T05:26:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:26:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:26:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:26:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:50 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:26:50.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:50 smithi032 ceph-mon[32351]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:50.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:50 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1073751163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:26:50.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:50 smithi032 ceph-mon[32351]: Standby manager daemon z started 2024-04-16T05:26:50.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:50 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:26:50.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:50 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:26:50.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:50 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:26:50.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:50 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:26:51.131 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:50.670+0000 7f454163e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:26:51.131 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:50.852+0000 7f454163e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:26:51.131 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-mon[36195]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:51.131 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1073751163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:26:51.131 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-mon[36195]: Standby manager daemon z started 2024-04-16T05:26:51.131 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:26:51.131 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:26:51.132 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:26:51.132 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:50 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.134:0/945522488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:26:51.385 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:51 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:51.132+0000 7f454163e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:26:51.385 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:51 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:51.386+0000 7f454163e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:26:51.649 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:51 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:51.620+0000 7f454163e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:26:51.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:51 smithi032 ceph-mon[32351]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-16T05:26:51.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:51 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:26:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:51 smithi134 ceph-mon[36153]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-16T05:26:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:51 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:26:52.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:51 smithi082 ceph-mon[36195]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-16T05:26:52.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:51 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:26:52.677 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:52.414+0000 7f454163e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:26:52.973 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:52 smithi082 ceph-mon[36195]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:52.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:52 smithi032 ceph-mon[32351]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:52 smithi134 ceph-mon[36153]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:53.282 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:52.974+0000 7f454163e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:26:53.282 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:53.070+0000 7f454163e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:26:53.282 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:26:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:26:53.283+0000 7f454163e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:26:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:26:53.688+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f27440177f0 con 0x7f2750103d60 2024-04-16T05:26:53.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:53 smithi032 ceph-mon[32351]: Standby manager daemon y started 2024-04-16T05:26:53.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:53 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:26:53.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:53 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:26:53.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:53 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:26:53.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:53 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:26:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:53 smithi134 ceph-mon[36153]: Standby manager daemon y started 2024-04-16T05:26:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:53 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:26:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:53 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:26:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:53 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:26:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:53 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:26:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:53 smithi082 ceph-mon[36195]: Standby manager daemon y started 2024-04-16T05:26:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:53 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:26:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:53 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:26:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:53 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:26:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:53 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.82:0/2022515738' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:26:54.902 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:54 smithi032 ceph-mon[32351]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:54.902 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:54 smithi032 ceph-mon[32351]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-16T05:26:54.903 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:54 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:26:54.903 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:54 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:54.903 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:54 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:54 smithi134 ceph-mon[36153]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:54 smithi134 ceph-mon[36153]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-16T05:26:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:54 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:26:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:54 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:54 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:54 smithi082 ceph-mon[36195]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:54 smithi082 ceph-mon[36195]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-16T05:26:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:54 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:26:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:54 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:54 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:55.920 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:55 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:55.920 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:55 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:55.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:55 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:55.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:55 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:55 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:55 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:55 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:55 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:56.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:55 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:26:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:55 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:55 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:26:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:55 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:26:56.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:56 smithi032 ceph-mon[32351]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:57.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:56 smithi134 ceph-mon[36153]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:57.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:56 smithi082 ceph-mon[36195]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:57.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T05:26:57.972 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T05:26:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:58.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T05:26:58.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:26:58.892 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:58 smithi032 ceph-mon[32351]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:58.892 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:26:58 smithi032 ceph-mon[32351]: Deploying daemon osd.0 on smithi032 2024-04-16T05:26:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:58 smithi134 ceph-mon[36153]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:26:58 smithi134 ceph-mon[36153]: Deploying daemon osd.0 on smithi032 2024-04-16T05:26:59.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:58 smithi082 ceph-mon[36195]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:26:59.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:26:58 smithi082 ceph-mon[36195]: Deploying daemon osd.0 on smithi032 2024-04-16T05:27:00.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:00 smithi032 ceph-mon[32351]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:01.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:00 smithi134 ceph-mon[36153]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:01.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:00 smithi082 ceph-mon[36195]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:03.002 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:02 smithi032 ceph-mon[32351]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:03.003 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:03.003 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:03.003 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:02 smithi134 ceph-mon[36153]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:02 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:02 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:02 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:03.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:02 smithi082 ceph-mon[36195]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:02 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:02 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:02 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:04.900 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:04 smithi032 ceph-mon[32351]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:04.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:04.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:04.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:04.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:04.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:05.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:04 smithi134 ceph-mon[36153]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:05.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:05.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:05.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:05.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:05.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:05.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:04 smithi082 ceph-mon[36195]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:05.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:05.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:05.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:05.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:05.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:06.135 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 0 on host 'smithi032' 2024-04-16T05:27:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.136+0000 7f2737fff700 1 -- 172.21.15.32:0/2654164201 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2750062f30 con 0x7f2738039890 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2738039890 msgr2=0x7f273803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2738039890 0x7f273803bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2750103bc0 tx=0x7f2740007400 comp rx=0 tx=0).stop 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 msgr2=0x7f2750074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 0x7f2750074ee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f274400fa00 tx=0x7f274400fa30 comp rx=0 tx=0).stop 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 shutdown_connections 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2738039890 0x7f273803bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2750104700 0x7f2750075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2750102b60 0x7f2750076890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 --2- 172.21.15.32:0/2654164201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2750103d60 0x7f2750074ee0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 >> 172.21.15.32:0/2654164201 conn(0x7f27500fe100 msgr2=0x7f2750107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:06.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 shutdown_connections 2024-04-16T05:27:06.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:06.138+0000 7f275574c700 1 -- 172.21.15.32:0/2654164201 wait complete. 2024-04-16T05:27:06.692 DEBUG:teuthology.orchestra.run.smithi032:osd.0> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.0.service 2024-04-16T05:27:06.694 INFO:tasks.cephadm:Deploying osd.1 on smithi082 with /dev/vg_nvme/lv_4... 2024-04-16T05:27:06.694 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T05:27:06.939 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:27:06.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:06 smithi032 ceph-mon[32351]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:06.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:06.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:06.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:06.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:06 smithi032 ceph-mon[32351]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T05:27:07.060 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:06 smithi082 ceph-mon[36195]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:07.060 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:06 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:07.060 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:06 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:07.060 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:06 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:07.060 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:06 smithi082 ceph-mon[36195]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T05:27:07.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:06 smithi134 ceph-mon[36153]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:07.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:06 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:07.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:06 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:07.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:06 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:07.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:06 smithi134 ceph-mon[36153]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T05:27:08.179 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:08 smithi032 ceph-mon[32351]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:08.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:08 smithi032 ceph-mon[32351]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T05:27:08.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:08 smithi032 ceph-mon[32351]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T05:27:08.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:08 smithi032 ceph-mon[32351]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:27:08.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:08.180 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:08 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0[47110]: 2024-04-16T05:27:08.138+0000 7f4ab61d9700 -1 osd.0 0 waiting for initial osdmap 2024-04-16T05:27:08.180 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:08 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0[47110]: 2024-04-16T05:27:08.141+0000 7f4ab1fcc700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:27:08.328 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:08 smithi082 ceph-mon[36195]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:08.328 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:08 smithi082 ceph-mon[36195]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T05:27:08.329 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:08 smithi082 ceph-mon[36195]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T05:27:08.329 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:08 smithi082 ceph-mon[36195]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:27:08.329 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:08 smithi134 ceph-mon[36153]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:08 smithi134 ceph-mon[36153]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T05:27:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:08 smithi134 ceph-mon[36153]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T05:27:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:08 smithi134 ceph-mon[36153]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:27:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:08.865 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T05:27:08.865 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T05:27:08.865 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-16T05:27:08.865 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-16T05:27:08.865 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127945 s, 820 MB/s 2024-04-16T05:27:08.865 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-16T05:27:09.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:09 smithi032 ceph-mon[32351]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:27:09.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:09 smithi032 ceph-mon[32351]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T05:27:09.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:09.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:09.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:09 smithi082 ceph-mon[36195]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:27:09.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:09 smithi082 ceph-mon[36195]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T05:27:09.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:09.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:09.616 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2024-04-16T05:27:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:09 smithi134 ceph-mon[36153]: from='osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:27:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:09 smithi134 ceph-mon[36153]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T05:27:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:09.862 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] boot 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:10.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:10.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:10.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:27:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:27:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] boot 2024-04-16T05:27:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T05:27:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:10.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:10.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:10.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:27:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:27:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T05:27:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: osd.0 [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] boot 2024-04-16T05:27:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T05:27:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:27:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.334+0000 7fd334484700 1 -- 172.21.15.82:0/4071497220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 msgr2=0x7fd32c10b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.334+0000 7fd334484700 1 --2- 172.21.15.82:0/4071497220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 0x7fd32c10b150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd32400b0d0 tx=0x7fd32400b3e0 comp rx=0 tx=0).stop 2024-04-16T05:27:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 -- 172.21.15.82:0/4071497220 shutdown_connections 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 --2- 172.21.15.82:0/4071497220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 0x7fd32c10b150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 --2- 172.21.15.82:0/4071497220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c100e80 0x7fd32c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 --2- 172.21.15.82:0/4071497220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c103f00 0x7fd32c104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 -- 172.21.15.82:0/4071497220 >> 172.21.15.82:0/4071497220 conn(0x7fd32c0fc7c0 msgr2=0x7fd32c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 -- 172.21.15.82:0/4071497220 shutdown_connections 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 -- 172.21.15.82:0/4071497220 wait complete. 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 Processor -- start 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 -- start start 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c100e80 0x7fd32c103860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c103f00 0x7fd32c101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 0x7fd32c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10e770 con 0x7fd32c103f00 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10e5f0 con 0x7fd32c1048a0 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.335+0000 7fd334484700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10e8f0 con 0x7fd32c100e80 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 0x7fd32c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 0x7fd32c1023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53140/0 (socket says 172.21.15.82:53140) 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 -- 172.21.15.82:0/365689340 learned_addr learned my addr 172.21.15.82:0/365689340 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd331a1f700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c103f00 0x7fd32c101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332220700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c100e80 0x7fd32c103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 -- 172.21.15.82:0/365689340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c100e80 msgr2=0x7fd32c103860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c100e80 0x7fd32c103860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 -- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c103f00 msgr2=0x7fd32c101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c103f00 0x7fd32c101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 -- 172.21.15.82:0/365689340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c102ca0 con 0x7fd32c1048a0 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd332a21700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 0x7fd32c1023f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd3240062a0 tx=0x7fd324007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd3237fe700 1 -- 172.21.15.82:0/365689340 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32401f070 con 0x7fd32c1048a0 2024-04-16T05:27:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c1a2df0 con 0x7fd32c1048a0 2024-04-16T05:27:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd3237fe700 1 -- 172.21.15.82:0/365689340 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd324004020 con 0x7fd32c1048a0 2024-04-16T05:27:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd3237fe700 1 -- 172.21.15.82:0/365689340 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324004e60 con 0x7fd32c1048a0 2024-04-16T05:27:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.336+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c1a3250 con 0x7fd32c1048a0 2024-04-16T05:27:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.337+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c0697e0 con 0x7fd32c1048a0 2024-04-16T05:27:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.338+0000 7fd3237fe700 1 -- 172.21.15.82:0/365689340 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd3240041d0 con 0x7fd32c1048a0 2024-04-16T05:27:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.339+0000 7fd3237fe700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd3180a3bb0 0x7fd3180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:11.340 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:11 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi032 2024-04-16T05:27:11.340 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:11 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi032 to 17093M 2024-04-16T05:27:11.340 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:11 smithi082 ceph-mon[36195]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T05:27:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.339+0000 7fd3237fe700 1 -- 172.21.15.82:0/365689340 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fd32400f020 con 0x7fd32c1048a0 2024-04-16T05:27:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.339+0000 7fd332220700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd3180a3bb0 0x7fd3180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.340+0000 7fd332220700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd3180a3bb0 0x7fd3180a6070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd328005fd0 tx=0x7fd328005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:11.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.343+0000 7fd3237fe700 1 -- 172.21.15.82:0/365689340 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd324028020 con 0x7fd32c1048a0 2024-04-16T05:27:11.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:11 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi032 2024-04-16T05:27:11.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:11 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi032 to 17093M 2024-04-16T05:27:11.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:11 smithi032 ceph-mon[32351]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T05:27:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:11.547+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd32c062f30 con 0x7fd3180a3bb0 2024-04-16T05:27:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:11 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi032 2024-04-16T05:27:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:11 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi032 to 17093M 2024-04-16T05:27:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:11 smithi134 ceph-mon[36153]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T05:27:12.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:12 smithi082 ceph-mon[36195]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:12.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:12 smithi082 ceph-mon[36195]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:12.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:12.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:12.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:12.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:12 smithi032 ceph-mon[32351]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:12.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:12 smithi032 ceph-mon[32351]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:12.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:12.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:12 smithi134 ceph-mon[36153]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:12 smithi134 ceph-mon[36153]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:14.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:14 smithi032 ceph-mon[32351]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:14.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:14 smithi032 ceph-mon[32351]: from='client.? 172.21.15.82:0/4001048507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]: dispatch 2024-04-16T05:27:14.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:14 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]: dispatch 2024-04-16T05:27:14.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:14 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]': finished 2024-04-16T05:27:14.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:14 smithi032 ceph-mon[32351]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T05:27:14.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:14.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:14 smithi134 ceph-mon[36153]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:14.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:14 smithi134 ceph-mon[36153]: from='client.? 172.21.15.82:0/4001048507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]: dispatch 2024-04-16T05:27:14.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:14 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]: dispatch 2024-04-16T05:27:14.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:14 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]': finished 2024-04-16T05:27:14.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:14 smithi134 ceph-mon[36153]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T05:27:14.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:14 smithi082 ceph-mon[36195]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:14 smithi082 ceph-mon[36195]: from='client.? 172.21.15.82:0/4001048507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]: dispatch 2024-04-16T05:27:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:14 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]: dispatch 2024-04-16T05:27:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:14 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bc55145-c8d6-445c-aa03-7617bf25e77e"}]': finished 2024-04-16T05:27:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:14 smithi082 ceph-mon[36195]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T05:27:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:15.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:15 smithi032 ceph-mon[32351]: from='client.? 172.21.15.82:0/1306830206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:15.551 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:15 smithi082 ceph-mon[36195]: from='client.? 172.21.15.82:0/1306830206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:15.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:15 smithi134 ceph-mon[36153]: from='client.? 172.21.15.82:0/1306830206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:16.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:16 smithi032 ceph-mon[32351]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:16.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:16 smithi134 ceph-mon[36153]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:16.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:16 smithi082 ceph-mon[36195]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:18.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:18 smithi082 ceph-mon[36195]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:18.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:18 smithi032 ceph-mon[32351]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:18 smithi134 ceph-mon[36153]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:20.391 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:20 smithi082 ceph-mon[36195]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:20.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:20 smithi032 ceph-mon[32351]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:20.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:20 smithi134 ceph-mon[36153]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:21.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:21 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T05:27:21.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:21 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:21.612 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:21 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T05:27:21.612 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:21 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:21.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:21 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T05:27:21.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:21 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:22.395 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:22 smithi082 ceph-mon[36195]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:22.395 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:22 smithi082 ceph-mon[36195]: Deploying daemon osd.1 on smithi082 2024-04-16T05:27:22.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:22 smithi032 ceph-mon[32351]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:22.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:22 smithi032 ceph-mon[32351]: Deploying daemon osd.1 on smithi082 2024-04-16T05:27:22.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:22 smithi134 ceph-mon[36153]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:22.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:22 smithi134 ceph-mon[36153]: Deploying daemon osd.1 on smithi082 2024-04-16T05:27:24.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:24 smithi032 ceph-mon[32351]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:24.497 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:24 smithi082 ceph-mon[36195]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:24.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:24 smithi134 ceph-mon[36153]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:26.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:26 smithi032 ceph-mon[32351]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:26.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:26 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:26.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:26 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:26.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:26 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:26.532 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:26 smithi082 ceph-mon[36195]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:26.532 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:26 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:26.532 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:26 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:26.532 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:26 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:26.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:26 smithi134 ceph-mon[36153]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:26.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:26 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:26.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:26 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:26.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:26 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:28.449 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:28 smithi082 ceph-mon[36195]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:28.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:28 smithi032 ceph-mon[32351]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:28.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:28 smithi134 ceph-mon[36153]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.417+0000 7fd3237fe700 1 -- 172.21.15.82:0/365689340 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd32c062f30 con 0x7fd3180a3bb0 2024-04-16T05:27:29.415 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 1 on host 'smithi082' 2024-04-16T05:27:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd3180a3bb0 msgr2=0x7fd3180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd3180a3bb0 0x7fd3180a6070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd328005fd0 tx=0x7fd328005dd0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 msgr2=0x7fd32c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 0x7fd32c1023f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd3240062a0 tx=0x7fd324007d90 comp rx=0 tx=0).stop 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 shutdown_connections 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd3180a3bb0 0x7fd3180a6070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c1048a0 0x7fd32c1023f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c100e80 0x7fd32c103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 --2- 172.21.15.82:0/365689340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c103f00 0x7fd32c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.418+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 >> 172.21.15.82:0/365689340 conn(0x7fd32c0fc7c0 msgr2=0x7fd32c107130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.419+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 shutdown_connections 2024-04-16T05:27:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:27:29.419+0000 7fd334484700 1 -- 172.21.15.82:0/365689340 wait complete. 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:29.981 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:29 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:30.006 DEBUG:teuthology.orchestra.run.smithi082:osd.1> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.1.service 2024-04-16T05:27:30.008 INFO:tasks.cephadm:Deploying osd.2 on smithi134 with /dev/vg_nvme/lv_4... 2024-04-16T05:27:30.009 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:29 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:30.251 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:27:30.373 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:30.373 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.373 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.373 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:30.373 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:30.373 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.373 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.373 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:30.374 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:29 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:31.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:30 smithi134 ceph-mon[36153]: from='osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:31.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:30 smithi134 ceph-mon[36153]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:31.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:30 smithi032 ceph-mon[32351]: from='osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:31.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:30 smithi032 ceph-mon[32351]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:31.329 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:30 smithi082 ceph-mon[36195]: from='osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:31.330 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:30 smithi082 ceph-mon[36195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T05:27:32.037 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T05:27:32.037 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T05:27:32.037 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-16T05:27:32.037 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-16T05:27:32.037 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111259 s, 942 MB/s 2024-04-16T05:27:32.037 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-16T05:27:32.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:31 smithi032 ceph-mon[32351]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:32.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:31 smithi032 ceph-mon[32351]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T05:27:32.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:31 smithi032 ceph-mon[32351]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T05:27:32.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:32.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:31 smithi032 ceph-mon[32351]: from='osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:27:32.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:31 smithi032 ceph-mon[32351]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:27:32.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:31 smithi134 ceph-mon[36153]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:32.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:31 smithi134 ceph-mon[36153]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T05:27:32.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:31 smithi134 ceph-mon[36153]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T05:27:32.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:32.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:31 smithi134 ceph-mon[36153]: from='osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:27:32.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:31 smithi134 ceph-mon[36153]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:27:32.360 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:31 smithi082 ceph-mon[36195]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:32.360 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:31 smithi082 ceph-mon[36195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T05:27:32.361 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:31 smithi082 ceph-mon[36195]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T05:27:32.361 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:32.361 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:31 smithi082 ceph-mon[36195]: from='osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:27:32.361 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:31 smithi082 ceph-mon[36195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:27:32.650 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1[40796]: 2024-04-16T05:27:32.454+0000 7fc62894b700 -1 osd.1 0 waiting for initial osdmap 2024-04-16T05:27:32.650 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1[40796]: 2024-04-16T05:27:32.457+0000 7fc620736700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:27:32.740 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-04-16T05:27:32.984 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:27:33.035 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:32 smithi082 ceph-mon[36195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:27:33.035 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:32 smithi082 ceph-mon[36195]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T05:27:33.035 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:33.035 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:33.036 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:32 smithi082 ceph-mon[36195]: from='osd.1 ' entity='osd.1' 2024-04-16T05:27:33.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:32 smithi134 ceph-mon[36153]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:27:33.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:32 smithi134 ceph-mon[36153]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T05:27:33.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:33.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:33.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:32 smithi134 ceph-mon[36153]: from='osd.1 ' entity='osd.1' 2024-04-16T05:27:33.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:32 smithi032 ceph-mon[32351]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:27:33.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:32 smithi032 ceph-mon[32351]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T05:27:33.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:32 smithi032 ceph-mon[32351]: from='osd.1 ' entity='osd.1' 2024-04-16T05:27:34.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi082 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi082 to 17080M 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:34.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] boot 2024-04-16T05:27:34.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T05:27:34.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:34.270 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi082 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi082 to 17080M 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] boot 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T05:27:34.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:34.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:27:34.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:27:34.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T05:27:34.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:34.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi082 2024-04-16T05:27:34.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi082 to 17080M 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: osd.1 [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] boot 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T05:27:34.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:27:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.480+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/3612524628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34102b50 msgr2=0x7fdd34102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.480+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/3612524628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34102b50 0x7fdd34102f70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009990 tx=0x7fdd28009ca0 comp rx=0 tx=0).stop 2024-04-16T05:27:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/3612524628 shutdown_connections 2024-04-16T05:27:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/3612524628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd341046f0 0x7fdd3410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/3612524628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34102b50 0x7fdd34102f70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/3612524628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd34103d50 0x7fdd341041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/3612524628 >> 172.21.15.134:0/3612524628 conn(0x7fdd340fe0f0 msgr2=0x7fdd34100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/3612524628 shutdown_connections 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/3612524628 wait complete. 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 Processor -- start 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.481+0000 7fdd3aa3f700 1 -- start start 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd3aa3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd34102b50 0x7fdd3419b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd3aa3f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34103d50 0x7fdd3419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd3aa3f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd341046f0 0x7fdd341a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd3aa3f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410e790 con 0x7fdd34102b50 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd3aa3f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410e610 con 0x7fdd341046f0 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd3aa3f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410e910 con 0x7fdd34103d50 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34103d50 0x7fdd3419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:34.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34103d50 0x7fdd3419b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:59112/0 (socket says 172.21.15.134:59112) 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 -- 172.21.15.134:0/2173071662 learned_addr learned my addr 172.21.15.134:0/2173071662 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd38fdc700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd341046f0 0x7fdd341a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd33fff700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd34102b50 0x7fdd3419b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 -- 172.21.15.134:0/2173071662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd341046f0 msgr2=0x7fdd341a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd341046f0 0x7fdd341a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 -- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd34102b50 msgr2=0x7fdd3419b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd34102b50 0x7fdd3419b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 -- 172.21.15.134:0/2173071662 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd341a2c80 con 0x7fdd34103d50 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd33fff700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd34102b50 0x7fdd3419b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd337fe700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34103d50 0x7fdd3419b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400b560 tx=0x7fdd2400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd317fa700 1 -- 172.21.15.134:0/2173071662 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24015020 con 0x7fdd34103d50 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd317fa700 1 -- 172.21.15.134:0/2173071662 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd24011d10 con 0x7fdd34103d50 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd317fa700 1 -- 172.21.15.134:0/2173071662 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400f2e0 con 0x7fdd34103d50 2024-04-16T05:27:34.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.482+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd341a2f70 con 0x7fdd34103d50 2024-04-16T05:27:34.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.483+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd341a34f0 con 0x7fdd34103d50 2024-04-16T05:27:34.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.484+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd34069810 con 0x7fdd34103d50 2024-04-16T05:27:34.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.485+0000 7fdd317fa700 1 -- 172.21.15.134:0/2173071662 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fdd24018020 con 0x7fdd34103d50 2024-04-16T05:27:34.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.485+0000 7fdd317fa700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdd1c0a3c30 0x7fdd1c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:34.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.486+0000 7fdd317fa700 1 -- 172.21.15.134:0/2173071662 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fdd240ce780 con 0x7fdd34103d50 2024-04-16T05:27:34.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.486+0000 7fdd33fff700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdd1c0a3c30 0x7fdd1c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:34.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.486+0000 7fdd33fff700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdd1c0a3c30 0x7fdd1c0a60f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800b5c0 tx=0x7fdd28003a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:34.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.490+0000 7fdd317fa700 1 -- 172.21.15.134:0/2173071662 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd34069810 con 0x7fdd34103d50 2024-04-16T05:27:34.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:34.698+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fdd3410ac40 con 0x7fdd1c0a3c30 2024-04-16T05:27:35.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:34 smithi032 ceph-mon[32351]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:35.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:35.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:35.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:35.296 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:34 smithi134 ceph-mon[36153]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:35.296 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:35.296 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:35.296 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:35.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:34 smithi082 ceph-mon[36195]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:35.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:35.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:35.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:36.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:35 smithi134 ceph-mon[36153]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:36.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:35 smithi134 ceph-mon[36153]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T05:27:36.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:35 smithi082 ceph-mon[36195]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:35 smithi082 ceph-mon[36195]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T05:27:36.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:35 smithi032 ceph-mon[32351]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:36.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:35 smithi032 ceph-mon[32351]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T05:27:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:37 smithi134 ceph-mon[36153]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:37 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/1242857559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]: dispatch 2024-04-16T05:27:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:37 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]: dispatch 2024-04-16T05:27:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:37 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]': finished 2024-04-16T05:27:38.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:37 smithi134 ceph-mon[36153]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T05:27:38.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:37 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:38.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:37 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/3613153354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:37 smithi082 ceph-mon[36195]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:37 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/1242857559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]: dispatch 2024-04-16T05:27:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:37 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]: dispatch 2024-04-16T05:27:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:37 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]': finished 2024-04-16T05:27:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:37 smithi082 ceph-mon[36195]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T05:27:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:37 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:37 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/3613153354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:38.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:37 smithi032 ceph-mon[32351]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:38.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:37 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/1242857559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]: dispatch 2024-04-16T05:27:38.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:37 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]: dispatch 2024-04-16T05:27:38.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:37 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6916f0e-af2f-4e04-a6e1-656f11cc3423"}]': finished 2024-04-16T05:27:38.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:37 smithi032 ceph-mon[32351]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T05:27:38.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:38.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:37 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/3613153354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:27:40.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:39 smithi134 ceph-mon[36153]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:40.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:39 smithi082 ceph-mon[36195]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:40.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:39 smithi032 ceph-mon[32351]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:42.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:41 smithi134 ceph-mon[36153]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:42.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:41 smithi082 ceph-mon[36195]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:42.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:41 smithi032 ceph-mon[32351]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:43 smithi032 ceph-mon[32351]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:44.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:43 smithi134 ceph-mon[36153]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:44.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:43 smithi082 ceph-mon[36195]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:45.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:45 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T05:27:45.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:45 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:45.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:45 smithi134 ceph-mon[36153]: Deploying daemon osd.2 on smithi134 2024-04-16T05:27:45.301 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:45 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T05:27:45.301 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:45 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:45.301 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:45 smithi082 ceph-mon[36195]: Deploying daemon osd.2 on smithi134 2024-04-16T05:27:45.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:45 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T05:27:45.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:45 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:45.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:45 smithi032 ceph-mon[32351]: Deploying daemon osd.2 on smithi134 2024-04-16T05:27:46.360 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:46 smithi134 ceph-mon[36153]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:46.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:46 smithi082 ceph-mon[36195]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:46.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:46 smithi032 ceph-mon[32351]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:48.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:48 smithi134 ceph-mon[36153]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:48.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:48 smithi082 ceph-mon[36195]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:48.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:48 smithi032 ceph-mon[32351]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:50.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:50 smithi134 ceph-mon[36153]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:50.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:50 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:50.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:50 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:50.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:50 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:50.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:50 smithi082 ceph-mon[36195]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:50.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:50 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:50.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:50 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:50.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:50 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:50.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:50 smithi032 ceph-mon[32351]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:50.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:50 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:50.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:50 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:50.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:50 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:52.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.565+0000 7fdd317fa700 1 -- 172.21.15.134:0/2173071662 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdd3410ac40 con 0x7fdd1c0a3c30 2024-04-16T05:27:52.565 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 2 on host 'smithi134' 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdd1c0a3c30 msgr2=0x7fdd1c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdd1c0a3c30 0x7fdd1c0a60f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800b5c0 tx=0x7fdd28003a00 comp rx=0 tx=0).stop 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34103d50 msgr2=0x7fdd3419b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34103d50 0x7fdd3419b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400b560 tx=0x7fdd2400b920 comp rx=0 tx=0).stop 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 shutdown_connections 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdd1c0a3c30 0x7fdd1c0a60f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd341046f0 0x7fdd341a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd34102b50 0x7fdd3419b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 --2- 172.21.15.134:0/2173071662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34103d50 0x7fdd3419b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 >> 172.21.15.134:0/2173071662 conn(0x7fdd340fe0f0 msgr2=0x7fdd34109520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:52.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 shutdown_connections 2024-04-16T05:27:52.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:27:52.568+0000 7fdd3aa3f700 1 -- 172.21.15.134:0/2173071662 wait complete. 2024-04-16T05:27:52.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:52 smithi134 ceph-mon[36153]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:52.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:52 smithi082 ceph-mon[36195]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:52.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:52 smithi032 ceph-mon[32351]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:53.132 DEBUG:teuthology.orchestra.run.smithi134:osd.2> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.2.service 2024-04-16T05:27:53.134 INFO:tasks.cephadm:Deploying osd.3 on smithi032 with /dev/vg_nvme/lv_3... 2024-04-16T05:27:53.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T05:27:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:27:53.838 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:53 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.838 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:53 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.839 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:53 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.839 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:53 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.839 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:53 smithi032 ceph-mon[32351]: from='osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:27:53.839 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:53 smithi032 ceph-mon[32351]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:27:53.841 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:53 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:53 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:53 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:53 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:53 smithi134 ceph-mon[36153]: from='osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:27:53.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:53 smithi134 ceph-mon[36153]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:27:53.842 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2[41312]: 2024-04-16T05:27:53.461+0000 7fd01017c7c0 -1 osd.2 0 log_to_monitors true 2024-04-16T05:27:53.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:53 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:53 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:53 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:53 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:53.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:53 smithi082 ceph-mon[36195]: from='osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:27:53.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:53 smithi082 ceph-mon[36195]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T05:27:54.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:54 smithi032 ceph-mon[32351]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:54 smithi032 ceph-mon[32351]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T05:27:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:54 smithi032 ceph-mon[32351]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T05:27:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:54 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:54 smithi032 ceph-mon[32351]: from='osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:27:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:54 smithi032 ceph-mon[32351]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:27:54.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:54 smithi134 ceph-mon[36153]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:54.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:54 smithi134 ceph-mon[36153]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T05:27:54.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:54 smithi134 ceph-mon[36153]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T05:27:54.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:54 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:54.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:54 smithi134 ceph-mon[36153]: from='osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:27:54.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:54 smithi134 ceph-mon[36153]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:27:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:54 smithi082 ceph-mon[36195]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:54 smithi082 ceph-mon[36195]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T05:27:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:54 smithi082 ceph-mon[36195]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T05:27:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:54 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:54 smithi082 ceph-mon[36195]: from='osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:27:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:54 smithi082 ceph-mon[36195]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:27:55.494 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T05:27:55.494 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T05:27:55.494 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-04-16T05:27:55.494 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-04-16T05:27:55.494 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00894898 s, 1.2 GB/s 2024-04-16T05:27:55.494 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-04-16T05:27:55.990 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2[41312]: 2024-04-16T05:27:55.669+0000 7fd0057e1700 -1 osd.2 0 waiting for initial osdmap 2024-04-16T05:27:55.991 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2[41312]: 2024-04-16T05:27:55.674+0000 7fcffd5cc700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:27:56.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_3 2024-04-16T05:27:56.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:56 smithi134 ceph-mon[36153]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:56.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:56 smithi134 ceph-mon[36153]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:27:56.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:56 smithi134 ceph-mon[36153]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T05:27:56.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:56 smithi134 ceph-mon[36153]: from='osd.2 ' entity='osd.2' 2024-04-16T05:27:56.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:56 smithi082 ceph-mon[36195]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:56.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:56 smithi082 ceph-mon[36195]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:27:56.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:56 smithi082 ceph-mon[36195]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T05:27:56.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:56 smithi082 ceph-mon[36195]: from='osd.2 ' entity='osd.2' 2024-04-16T05:27:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:27:56.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:56 smithi032 ceph-mon[32351]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:56.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:56 smithi032 ceph-mon[32351]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:27:56.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:56 smithi032 ceph-mon[32351]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T05:27:56.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:56.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:56 smithi032 ceph-mon[32351]: from='osd.2 ' entity='osd.2' 2024-04-16T05:27:57.532 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:27:57.532 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi134 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi134 to 17080M 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] boot 2024-04-16T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T05:27:57.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:57.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:57.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:57.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:57.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:57 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:27:57.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:27:57.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:27:57.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:57.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi134 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi134 to 17080M 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] boot 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:57.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:57.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:57 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:27:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:27:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:27:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi134 2024-04-16T05:27:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:27:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi134 to 17080M 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: osd.2 [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] boot 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:57.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:57 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:27:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 -- 172.21.15.32:0/1961908789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860100be0 msgr2=0x7f6860101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 --2- 172.21.15.32:0/1961908789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860100be0 0x7f6860101040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6854009990 tx=0x7f6854009ca0 comp rx=0 tx=0).stop 2024-04-16T05:27:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 -- 172.21.15.32:0/1961908789 shutdown_connections 2024-04-16T05:27:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 --2- 172.21.15.32:0/1961908789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6860101580 0x7f686010b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 --2- 172.21.15.32:0/1961908789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6860104160 0x7f6860104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 --2- 172.21.15.32:0/1961908789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860100be0 0x7f6860101040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 -- 172.21.15.32:0/1961908789 >> 172.21.15.32:0/1961908789 conn(0x7f68600fc780 msgr2=0x7f68600feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 -- 172.21.15.32:0/1961908789 shutdown_connections 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.069+0000 7f68668a4700 1 -- 172.21.15.32:0/1961908789 wait complete. 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f68668a4700 1 Processor -- start 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f68668a4700 1 -- start start 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f68668a4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6860100be0 0x7f6860103860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f68668a4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6860101580 0x7f6860101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f68668a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860104160 0x7f68601023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f68668a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f686010e700 con 0x7f6860104160 2024-04-16T05:27:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f68668a4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f686010e580 con 0x7f6860100be0 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f68668a4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f686010e880 con 0x7f6860101580 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f6864e41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860104160 0x7f68601023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f6864e41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860104160 0x7f68601023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38220/0 (socket says 172.21.15.32:38220) 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.070+0000 7f685ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6860100be0 0x7f6860103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f6864e41700 1 -- 172.21.15.32:0/206116618 learned_addr learned my addr 172.21.15.32:0/206116618 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f685f7fe700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6860101580 0x7f6860101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f6864e41700 1 -- 172.21.15.32:0/206116618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6860101580 msgr2=0x7f6860101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f6864e41700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6860101580 0x7f6860101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f6864e41700 1 -- 172.21.15.32:0/206116618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6860100be0 msgr2=0x7f6860103860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f6864e41700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6860100be0 0x7f6860103860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f6864e41700 1 -- 172.21.15.32:0/206116618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6860102ca0 con 0x7f6860104160 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f6864e41700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860104160 0x7f68601023f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6858007ee0 tx=0x7f685800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f685d7fa700 1 -- 172.21.15.32:0/206116618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858004bb0 con 0x7f6860104160 2024-04-16T05:27:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f685d7fa700 1 -- 172.21.15.32:0/206116618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6858004d50 con 0x7f6860104160 2024-04-16T05:27:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68601a2e70 con 0x7f6860104160 2024-04-16T05:27:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f685d7fa700 1 -- 172.21.15.32:0/206116618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858005960 con 0x7f6860104160 2024-04-16T05:27:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.071+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68601a32d0 con 0x7f6860104160 2024-04-16T05:27:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.072+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68600697e0 con 0x7f6860104160 2024-04-16T05:27:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.074+0000 7f685d7fa700 1 -- 172.21.15.32:0/206116618 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f685801a020 con 0x7f6860104160 2024-04-16T05:27:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.074+0000 7f685d7fa700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68480a3bb0 0x7f68480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:27:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.074+0000 7f685ffff700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68480a3bb0 0x7f68480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:27:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.074+0000 7f685d7fa700 1 -- 172.21.15.32:0/206116618 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f68580cec90 con 0x7f6860104160 2024-04-16T05:27:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.075+0000 7f685ffff700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68480a3bb0 0x7f68480a6070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f686006a4b0 tx=0x7f6850007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:27:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.079+0000 7f685d7fa700 1 -- 172.21.15.32:0/206116618 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6858010070 con 0x7f6860104160 2024-04-16T05:27:58.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:27:58.295+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6860062f30 con 0x7f68480a3bb0 2024-04-16T05:27:58.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:58 smithi032 ceph-mon[32351]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:58.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T05:27:58.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:58 smithi032 ceph-mon[32351]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T05:27:58.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:27:58.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:58.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:58.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:59.042 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:58 smithi082 ceph-mon[36195]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:59.042 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:58 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T05:27:59.043 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:58 smithi082 ceph-mon[36195]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T05:27:59.043 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:58 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:27:59.043 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:58 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:59.043 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:58 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:59.043 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:58 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:59.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:58 smithi134 ceph-mon[36153]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-16T05:27:59.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:58 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T05:27:59.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:58 smithi134 ceph-mon[36153]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T05:27:59.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:58 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:27:59.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:58 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:27:59.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:58 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:27:59.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:58 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:27:59.191 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51933]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:27:59.191 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51933]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.192 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51933]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.192 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51933]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.192 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:58 smithi032 sudo[51922]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:27:59.192 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:58 smithi032 sudo[51922]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.192 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:58 smithi032 sudo[51922]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.192 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:58 smithi032 sudo[51922]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.192 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51925]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:27:59.192 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51925]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.192 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51925]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.192 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51925]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45399]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:27:59.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45399]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45399]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45399]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45402]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:27:59.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45402]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45402]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45402]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.385 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:27:59.385 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.385 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45393]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.385 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45393]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.385 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45396]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:27:59.385 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45396]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.385 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45396]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.385 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:27:59 smithi134 sudo[45396]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:27:59.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44825]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44825]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44828]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:27:59.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44828]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44828]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44828]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.400 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44819]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T05:27:59.400 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44819]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.400 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44819]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.400 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44819]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.401 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:27:59.401 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.401 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44822]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.401 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:27:59 smithi082 sudo[44822]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51949]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T05:27:59.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T05:27:59.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51949]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T05:27:59.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 sudo[51949]: pam_unix(sudo:session): session closed for user root 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:27:59.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:27:59.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:27:59 smithi032 ceph-mon[32351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:00.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:00.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:00.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:27:59 smithi134 ceph-mon[36153]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:28:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T05:28:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:28:00.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:27:59 smithi082 ceph-mon[36195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T05:28:00.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:00 smithi032 ceph-mon[32351]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:00.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:00 smithi032 ceph-mon[32351]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:28:00.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:00 smithi032 ceph-mon[32351]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T05:28:01.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:00 smithi134 ceph-mon[36153]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:01.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:00 smithi134 ceph-mon[36153]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:28:01.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:00 smithi134 ceph-mon[36153]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T05:28:01.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:00 smithi082 ceph-mon[36195]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:00 smithi082 ceph-mon[36195]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:28:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:00 smithi082 ceph-mon[36195]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T05:28:01.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:01.698+0000 7f685d7fa700 1 -- 172.21.15.32:0/206116618 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6858098aa0 con 0x7f6860104160 2024-04-16T05:28:01.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1387554360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "847ede2b-81d0-4993-895f-2e712666ea5a"}]: dispatch 2024-04-16T05:28:01.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1387554360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "847ede2b-81d0-4993-895f-2e712666ea5a"}]': finished 2024-04-16T05:28:01.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:01 smithi032 ceph-mon[32351]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T05:28:01.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:01 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1387554360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "847ede2b-81d0-4993-895f-2e712666ea5a"}]: dispatch 2024-04-16T05:28:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1387554360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "847ede2b-81d0-4993-895f-2e712666ea5a"}]': finished 2024-04-16T05:28:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:01 smithi134 ceph-mon[36153]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T05:28:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:01 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:02.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1387554360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "847ede2b-81d0-4993-895f-2e712666ea5a"}]: dispatch 2024-04-16T05:28:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1387554360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "847ede2b-81d0-4993-895f-2e712666ea5a"}]': finished 2024-04-16T05:28:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:01 smithi082 ceph-mon[36195]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T05:28:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:01 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:02.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:02 smithi032 ceph-mon[32351]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:02 smithi032 ceph-mon[32351]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:28:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:02 smithi032 ceph-mon[32351]: Cluster is now healthy 2024-04-16T05:28:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:02 smithi032 ceph-mon[32351]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T05:28:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:02 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3358976639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:02 smithi134 ceph-mon[36153]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:02 smithi134 ceph-mon[36153]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:28:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:02 smithi134 ceph-mon[36153]: Cluster is now healthy 2024-04-16T05:28:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:02 smithi134 ceph-mon[36153]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T05:28:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:02 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3358976639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:02 smithi082 ceph-mon[36195]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:02 smithi082 ceph-mon[36195]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:28:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:02 smithi082 ceph-mon[36195]: Cluster is now healthy 2024-04-16T05:28:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:02 smithi082 ceph-mon[36195]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T05:28:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:02 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3358976639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:04.959 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:04 smithi134 ceph-mon[36153]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:04.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:04 smithi032 ceph-mon[32351]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:05.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:04 smithi082 ceph-mon[36195]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:06.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:06 smithi032 ceph-mon[32351]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 8.1 KiB/s rd, 368 KiB/s wr, 25 op/s 2024-04-16T05:28:07.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:06 smithi134 ceph-mon[36153]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 8.1 KiB/s rd, 368 KiB/s wr, 25 op/s 2024-04-16T05:28:07.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:06 smithi082 ceph-mon[36195]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 8.1 KiB/s rd, 368 KiB/s wr, 25 op/s 2024-04-16T05:28:08.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:08 smithi032 ceph-mon[32351]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 5.7 KiB/s rd, 263 KiB/s wr, 18 op/s 2024-04-16T05:28:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:08 smithi134 ceph-mon[36153]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 5.7 KiB/s rd, 263 KiB/s wr, 18 op/s 2024-04-16T05:28:09.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:08 smithi082 ceph-mon[36195]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 5.7 KiB/s rd, 263 KiB/s wr, 18 op/s 2024-04-16T05:28:09.910 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T05:28:09.910 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T05:28:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T05:28:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:10.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:10 smithi032 ceph-mon[32351]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 204 KiB/s wr, 14 op/s 2024-04-16T05:28:10.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:10 smithi032 ceph-mon[32351]: Deploying daemon osd.3 on smithi032 2024-04-16T05:28:11.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:10 smithi134 ceph-mon[36153]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 204 KiB/s wr, 14 op/s 2024-04-16T05:28:11.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:10 smithi134 ceph-mon[36153]: Deploying daemon osd.3 on smithi032 2024-04-16T05:28:11.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:10 smithi082 ceph-mon[36195]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 204 KiB/s wr, 14 op/s 2024-04-16T05:28:11.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:10 smithi082 ceph-mon[36195]: Deploying daemon osd.3 on smithi032 2024-04-16T05:28:12.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:12 smithi032 ceph-mon[32351]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-04-16T05:28:13.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:12 smithi134 ceph-mon[36153]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-04-16T05:28:13.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:12 smithi082 ceph-mon[36195]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-04-16T05:28:14.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:14 smithi032 ceph-mon[32351]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 156 KiB/s wr, 10 op/s 2024-04-16T05:28:14.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:14 smithi134 ceph-mon[36153]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 156 KiB/s wr, 10 op/s 2024-04-16T05:28:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:15.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:14 smithi082 ceph-mon[36195]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 156 KiB/s wr, 10 op/s 2024-04-16T05:28:15.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:15.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:15.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:15 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:16.051 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:15 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:16.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:15 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:16 smithi032 ceph-mon[32351]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-04-16T05:28:17.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:16 smithi134 ceph-mon[36153]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-04-16T05:28:17.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:16 smithi082 ceph-mon[36195]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-04-16T05:28:18.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.462+0000 7f685d7fa700 1 -- 172.21.15.32:0/206116618 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6860062f30 con 0x7f68480a3bb0 2024-04-16T05:28:18.460 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 3 on host 'smithi032' 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68480a3bb0 msgr2=0x7f68480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68480a3bb0 0x7f68480a6070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f686006a4b0 tx=0x7f6850007400 comp rx=0 tx=0).stop 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860104160 msgr2=0x7f68601023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860104160 0x7f68601023f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6858007ee0 tx=0x7f685800d3b0 comp rx=0 tx=0).stop 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 shutdown_connections 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68480a3bb0 0x7f68480a6070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6860104160 0x7f68601023f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6860100be0 0x7f6860103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 --2- 172.21.15.32:0/206116618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6860101580 0x7f6860101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:18.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 >> 172.21.15.32:0/206116618 conn(0x7f68600fc780 msgr2=0x7f6860107200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:28:18.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 shutdown_connections 2024-04-16T05:28:18.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:28:18.464+0000 7f68668a4700 1 -- 172.21.15.32:0/206116618 wait complete. 2024-04-16T05:28:18.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:18 smithi032 ceph-mon[32351]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T05:28:18.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:18.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:18.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:18.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:18.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:18 smithi134 ceph-mon[36153]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:18 smithi082 ceph-mon[36195]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T05:28:19.145 DEBUG:teuthology.orchestra.run.smithi032:osd.3> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.3.service 2024-04-16T05:28:19.147 INFO:tasks.cephadm:Deploying osd.4 on smithi082 with /dev/vg_nvme/lv_3... 2024-04-16T05:28:19.147 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T05:28:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:28:20.635 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:20 smithi082 ceph-mon[36195]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:20.636 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:20 smithi082 ceph-mon[36195]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T05:28:20.636 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:20 smithi082 ceph-mon[36195]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T05:28:20.636 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:20 smithi082 ceph-mon[36195]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:28:20.636 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:20.644 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:20 smithi032 ceph-mon[32351]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:20.645 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:20 smithi032 ceph-mon[32351]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T05:28:20.645 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:20 smithi032 ceph-mon[32351]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T05:28:20.645 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:20 smithi032 ceph-mon[32351]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:28:20.645 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:20.645 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:28:20 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3[54062]: 2024-04-16T05:28:20.437+0000 7f9056ab6700 -1 osd.3 0 waiting for initial osdmap 2024-04-16T05:28:20.645 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:28:20 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3[54062]: 2024-04-16T05:28:20.442+0000 7f90528a9700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:28:20.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:20 smithi134 ceph-mon[36153]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:20.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:20 smithi134 ceph-mon[36153]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T05:28:20.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:20 smithi134 ceph-mon[36153]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T05:28:20.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:20 smithi134 ceph-mon[36153]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:28:20.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:21.330 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T05:28:21.331 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T05:28:21.331 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-16T05:28:21.331 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-16T05:28:21.331 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112768 s, 930 MB/s 2024-04-16T05:28:21.331 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 2024-04-16T05:28:21.331 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-16T05:28:21.625 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:21 smithi082 ceph-mon[36195]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:28:21.625 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:21 smithi082 ceph-mon[36195]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T05:28:21.626 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:21 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:21.626 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:21 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:21.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:21 smithi032 ceph-mon[32351]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:28:21.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:21 smithi032 ceph-mon[32351]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T05:28:21.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:21 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:21.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:21 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:21.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:21 smithi134 ceph-mon[36153]: from='osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:28:21.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:21 smithi134 ceph-mon[36153]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T05:28:21.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:21 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:21.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:21 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:22.029 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2024-04-16T05:28:22.273 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:28:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:28:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:28:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] boot 2024-04-16T05:28:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T05:28:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:22 smithi082 ceph-mon[36195]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T05:28:22.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] boot 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:22 smithi032 ceph-mon[32351]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T05:28:22.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:28:22.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: osd.3 [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] boot 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:22 smithi134 ceph-mon[36153]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T05:28:23.566 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:23 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi032 2024-04-16T05:28:23.567 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:23 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi032 to 8546M 2024-04-16T05:28:23.567 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:23 smithi082 ceph-mon[36195]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T05:28:23.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:23 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi032 2024-04-16T05:28:23.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:23 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi032 to 8546M 2024-04-16T05:28:23.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:23 smithi032 ceph-mon[32351]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.787+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1023356121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c103600 msgr2=0x7f7a7c109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.787+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1023356121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c103600 0x7f7a7c109eb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400b0d0 tx=0x7f7a7400b3e0 comp rx=0 tx=0).stop 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1023356121 shutdown_connections 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1023356121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c103600 0x7f7a7c109eb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1023356121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a7c101a60 0x7f7a7c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1023356121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7c102c60 0x7f7a7c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1023356121 >> 172.21.15.82:0/1023356121 conn(0x7f7a7c0fd020 msgr2=0x7f7a7c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1023356121 shutdown_connections 2024-04-16T05:28:23.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1023356121 wait complete. 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 Processor -- start 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 -- start start 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c101a60 0x7f7a7c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7c102c60 0x7f7a7c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a7c103600 0x7f7a7c1a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10d6c0 con 0x7f7a7c102c60 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10d540 con 0x7f7a7c101a60 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a82a6b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10d840 con 0x7f7a7c103600 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a80807700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c101a60 0x7f7a7c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a80807700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c101a60 0x7f7a7c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34402/0 (socket says 172.21.15.82:34402) 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.788+0000 7f7a80807700 1 -- 172.21.15.82:0/1611552905 learned_addr learned my addr 172.21.15.82:0/1611552905 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a7bfff700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7c102c60 0x7f7a7c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a80807700 1 -- 172.21.15.82:0/1611552905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a7c103600 msgr2=0x7f7a7c1a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:23.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a80807700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a7c103600 0x7f7a7c1a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a80807700 1 -- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7c102c60 msgr2=0x7f7a7c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a80807700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7c102c60 0x7f7a7c199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a80807700 1 -- 172.21.15.82:0/1611552905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c1a0a20 con 0x7f7a7c101a60 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a80807700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c101a60 0x7f7a7c198dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000b500 tx=0x7f7a7000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a79ffb700 1 -- 172.21.15.82:0/1611552905 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70015020 con 0x7f7a7c101a60 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7c1a0d10 con 0x7f7a7c101a60 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a79ffb700 1 -- 172.21.15.82:0/1611552905 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a70011d10 con 0x7f7a7c101a60 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a79ffb700 1 -- 172.21.15.82:0/1611552905 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000f2e0 con 0x7f7a7c101a60 2024-04-16T05:28:23.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.789+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7c1a1220 con 0x7f7a7c101a60 2024-04-16T05:28:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.790+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7c1a1510 con 0x7f7a7c101a60 2024-04-16T05:28:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.791+0000 7f7a79ffb700 1 -- 172.21.15.82:0/1611552905 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7a70018020 con 0x7f7a7c101a60 2024-04-16T05:28:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.792+0000 7f7a79ffb700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.792+0000 7f7a79ffb700 1 -- 172.21.15.82:0/1611552905 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f7a700cea00 con 0x7f7a7c101a60 2024-04-16T05:28:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.792+0000 7f7a7bfff700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.793+0000 7f7a7bfff700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005fd0 tx=0x7f7a6800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:28:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:23.797+0000 7f7a79ffb700 1 -- 172.21.15.82:0/1611552905 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a7001f020 con 0x7f7a7c101a60 2024-04-16T05:28:23.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:23 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi032 2024-04-16T05:28:23.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:23 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi032 to 8546M 2024-04-16T05:28:23.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:23 smithi134 ceph-mon[36153]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T05:28:24.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:24.007+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7a7c1a16e0 con 0x7f7a6c0a3c40 2024-04-16T05:28:24.720 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:24 smithi082 ceph-mon[36195]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:24.720 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:24 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:24.720 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:24 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:24.721 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:24 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:24.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:24 smithi032 ceph-mon[32351]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:24.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:24.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:24.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:24.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:24 smithi134 ceph-mon[36153]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:24.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:24.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:24.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:25.615 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:25 smithi082 ceph-mon[36195]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:25.615 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:25 smithi082 ceph-mon[36195]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T05:28:25.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:25 smithi032 ceph-mon[32351]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:25.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:25 smithi032 ceph-mon[32351]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T05:28:25.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:25 smithi134 ceph-mon[36153]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:25.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:25 smithi134 ceph-mon[36153]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T05:28:26.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:26 smithi032 ceph-mon[32351]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:28:26.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:26 smithi032 ceph-mon[32351]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T05:28:26.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:26 smithi134 ceph-mon[36153]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:28:26.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:26 smithi134 ceph-mon[36153]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T05:28:26.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:26 smithi082 ceph-mon[36195]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:28:26.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:26 smithi082 ceph-mon[36195]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T05:28:27.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:27 smithi032 ceph-mon[32351]: from='client.? 172.21.15.82:0/4150903243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]: dispatch 2024-04-16T05:28:27.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:27 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]: dispatch 2024-04-16T05:28:27.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:27 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]': finished 2024-04-16T05:28:27.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:27 smithi032 ceph-mon[32351]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T05:28:27.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:27 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:27.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:27 smithi032 ceph-mon[32351]: from='client.? 172.21.15.82:0/520044361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:27.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:27 smithi134 ceph-mon[36153]: from='client.? 172.21.15.82:0/4150903243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]: dispatch 2024-04-16T05:28:27.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:27 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]: dispatch 2024-04-16T05:28:27.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:27 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]': finished 2024-04-16T05:28:27.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:27 smithi134 ceph-mon[36153]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T05:28:27.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:27 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:27.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:27 smithi134 ceph-mon[36153]: from='client.? 172.21.15.82:0/520044361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:27.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:27 smithi082 ceph-mon[36195]: from='client.? 172.21.15.82:0/4150903243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]: dispatch 2024-04-16T05:28:27.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:27 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]: dispatch 2024-04-16T05:28:27.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:27 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f816e73-9c2a-4392-80fd-ad98adc90aed"}]': finished 2024-04-16T05:28:27.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:27 smithi082 ceph-mon[36195]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T05:28:27.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:27 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:27.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:27 smithi082 ceph-mon[36195]: from='client.? 172.21.15.82:0/520044361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:29.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:28 smithi134 ceph-mon[36153]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:28:29.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:28 smithi082 ceph-mon[36195]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:28:29.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:28 smithi032 ceph-mon[32351]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:28:31.017 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:30 smithi082 ceph-mon[36195]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-16T05:28:31.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:30 smithi134 ceph-mon[36153]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-16T05:28:31.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:30 smithi032 ceph-mon[32351]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-16T05:28:32.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:32 smithi082 ceph-mon[36195]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:32 smithi134 ceph-mon[36153]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:33.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:32 smithi032 ceph-mon[32351]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:34.833 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:34 smithi082 ceph-mon[36195]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:34.833 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T05:28:34.833 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:35.035 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:34 smithi032 ceph-mon[32351]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:35.035 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T05:28:35.035 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:35.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:34 smithi134 ceph-mon[36153]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:35.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T05:28:35.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:35.771 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:35 smithi082 ceph-mon[36195]: Deploying daemon osd.4 on smithi082 2024-04-16T05:28:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:35 smithi134 ceph-mon[36153]: Deploying daemon osd.4 on smithi082 2024-04-16T05:28:36.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:35 smithi032 ceph-mon[32351]: Deploying daemon osd.4 on smithi082 2024-04-16T05:28:37.051 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:36 smithi082 ceph-mon[36195]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:37.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:36 smithi134 ceph-mon[36153]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:37.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:36 smithi032 ceph-mon[32351]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:38.993 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:38 smithi082 ceph-mon[36195]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:38 smithi134 ceph-mon[36153]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:39.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:38 smithi032 ceph-mon[32351]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:40.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:40.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:40.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:40.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:40.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:40.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:40.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:40 smithi082 ceph-mon[36195]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:40 smithi134 ceph-mon[36153]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:40 smithi032 ceph-mon[32351]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:43.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:42 smithi082 ceph-mon[36195]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:43.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:42 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:42 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:42 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:43.039 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:42 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:43.040 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:42 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:42 smithi134 ceph-mon[36153]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:43.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:42 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:42 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:42 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:43.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:42 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:43.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:42 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:42 smithi032 ceph-mon[32351]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:43.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:42 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:42 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:42 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:43.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:42 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:43.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:42 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.378+0000 7f7a79ffb700 1 -- 172.21.15.82:0/1611552905 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7a7c1a16e0 con 0x7f7a6c0a3c40 2024-04-16T05:28:43.377 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 4 on host 'smithi082' 2024-04-16T05:28:43.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.380+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a6c0a3c40 msgr2=0x7f7a6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:43.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.380+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005fd0 tx=0x7f7a6800a560 comp rx=0 tx=0).stop 2024-04-16T05:28:43.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.380+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c101a60 msgr2=0x7f7a7c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.380+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c101a60 0x7f7a7c198dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000b500 tx=0x7f7a7000b8c0 comp rx=0 tx=0).stop 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.380+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 shutdown_connections 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.380+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.381+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a7c103600 0x7f7a7c1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.381+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a7c101a60 0x7f7a7c198dc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.381+0000 7f7a82a6b700 1 --2- 172.21.15.82:0/1611552905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7c102c60 0x7f7a7c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.381+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 >> 172.21.15.82:0/1611552905 conn(0x7f7a7c0fd020 msgr2=0x7f7a7c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.381+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 shutdown_connections 2024-04-16T05:28:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:28:43.381+0000 7f7a82a6b700 1 -- 172.21.15.82:0/1611552905 wait complete. 2024-04-16T05:28:43.976 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:43 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:43.976 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:43 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.976 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:43 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:43.976 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:43 smithi082 ceph-mon[36195]: from='osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:28:43.976 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:43 smithi082 ceph-mon[36195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:28:44.004 DEBUG:teuthology.orchestra.run.smithi082:osd.4> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.4.service 2024-04-16T05:28:44.006 INFO:tasks.cephadm:Deploying osd.5 on smithi134 with /dev/vg_nvme/lv_3... 2024-04-16T05:28:44.007 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T05:28:44.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:43 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:44.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:43 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:44.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:43 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:44.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:43 smithi134 ceph-mon[36153]: from='osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:28:44.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:43 smithi134 ceph-mon[36153]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:28:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:43 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:28:44.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:43 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:44.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:43 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:44.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:43 smithi032 ceph-mon[32351]: from='osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:28:44.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:43 smithi032 ceph-mon[32351]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T05:28:44.252 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:28:45.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:44 smithi082 ceph-mon[36195]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:45.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:44 smithi082 ceph-mon[36195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T05:28:45.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:44 smithi082 ceph-mon[36195]: from='osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:28:45.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:44 smithi082 ceph-mon[36195]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T05:28:45.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:45.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:44 smithi082 ceph-mon[36195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:28:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:44 smithi134 ceph-mon[36153]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:44 smithi134 ceph-mon[36153]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T05:28:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:44 smithi134 ceph-mon[36153]: from='osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:28:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:44 smithi134 ceph-mon[36153]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T05:28:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:44 smithi134 ceph-mon[36153]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:28:45.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:44 smithi032 ceph-mon[32351]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:45.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:44 smithi032 ceph-mon[32351]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T05:28:45.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:44 smithi032 ceph-mon[32351]: from='osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:28:45.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:44 smithi032 ceph-mon[32351]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T05:28:45.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:45.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:44 smithi032 ceph-mon[32351]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:28:45.367 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:28:45 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4[47644]: 2024-04-16T05:28:45.361+0000 7f55e381c700 -1 osd.4 0 waiting for initial osdmap 2024-04-16T05:28:45.368 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:28:45 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4[47644]: 2024-04-16T05:28:45.368+0000 7f55df60f700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:28:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T05:28:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T05:28:46.167 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-16T05:28:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-16T05:28:46.167 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111882 s, 937 MB/s 2024-04-16T05:28:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-16T05:28:46.464 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:46 smithi134 ceph-mon[36153]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:46.464 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:46 smithi134 ceph-mon[36153]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:28:46.464 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:46 smithi134 ceph-mon[36153]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T05:28:46.464 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:46 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:46.464 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:46 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:46.515 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:46 smithi082 ceph-mon[36195]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:46.515 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:46 smithi082 ceph-mon[36195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:28:46.515 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:46 smithi082 ceph-mon[36195]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T05:28:46.515 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:46 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:46.515 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:46 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:46.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:46 smithi032 ceph-mon[32351]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T05:28:46.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:46 smithi032 ceph-mon[32351]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:28:46.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:46 smithi032 ceph-mon[32351]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T05:28:46.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:46 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:46.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:46 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:46.863 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-04-16T05:28:47.145 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:28:47.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:28:47.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:28:47.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] boot 2024-04-16T05:28:47.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:47.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:28:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:28:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] boot 2024-04-16T05:28:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T05:28:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:47.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:47.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:47.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:47.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: osd.4 [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] boot 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:47.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:28:47.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:28:48.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:48 smithi134 ceph-mon[36153]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:48.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:48 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi082 2024-04-16T05:28:48.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:48 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi082 to 8540M 2024-04-16T05:28:48.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:48 smithi134 ceph-mon[36153]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T05:28:48.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 -- 172.21.15.134:0/3629553208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc778102b40 msgr2=0x7fc778102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:48.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 --2- 172.21.15.134:0/3629553208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc778102b40 0x7fc778102f60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc774009990 tx=0x7fc774009ca0 comp rx=0 tx=0).stop 2024-04-16T05:28:48.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 -- 172.21.15.134:0/3629553208 shutdown_connections 2024-04-16T05:28:48.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 --2- 172.21.15.134:0/3629553208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7781046e0 0x7fc77810af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:48.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 --2- 172.21.15.134:0/3629553208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc778102b40 0x7fc778102f60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:48.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 --2- 172.21.15.134:0/3629553208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778103d40 0x7fc7781041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 -- 172.21.15.134:0/3629553208 >> 172.21.15.134:0/3629553208 conn(0x7fc7780fe0e0 msgr2=0x7fc778100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 -- 172.21.15.134:0/3629553208 shutdown_connections 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 -- 172.21.15.134:0/3629553208 wait complete. 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 Processor -- start 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 -- start start 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.640+0000 7fc78077d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778102b40 0x7fc778198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc78077d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778103d40 0x7fc778199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc78077d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7781046e0 0x7fc7781a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc78077d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc77810e720 con 0x7fc778102b40 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc78077d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc77810e5a0 con 0x7fc778103d40 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc78077d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc77810e8a0 con 0x7fc7781046e0 2024-04-16T05:28:48.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7781046e0 0x7fc7781a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77e519700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778102b40 0x7fc778198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7781046e0 0x7fc7781a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:40606/0 (socket says 172.21.15.134:40606) 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 -- 172.21.15.134:0/1009388170 learned_addr learned my addr 172.21.15.134:0/1009388170 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77dd18700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778103d40 0x7fc778199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 -- 172.21.15.134:0/1009388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778103d40 msgr2=0x7fc778199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778103d40 0x7fc778199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 -- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778102b40 msgr2=0x7fc778198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778102b40 0x7fc778198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 -- 172.21.15.134:0/1009388170 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7781a09b0 con 0x7fc7781046e0 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc77ed1a700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7781046e0 0x7fc7781a0310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc770007f10 tx=0x7fc77000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc76f7fe700 1 -- 172.21.15.134:0/1009388170 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770004bb0 con 0x7fc7781046e0 2024-04-16T05:28:48.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7781a0ca0 con 0x7fc7781046e0 2024-04-16T05:28:48.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc76f7fe700 1 -- 172.21.15.134:0/1009388170 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc770004d50 con 0x7fc7781046e0 2024-04-16T05:28:48.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc76f7fe700 1 -- 172.21.15.134:0/1009388170 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700059f0 con 0x7fc7781046e0 2024-04-16T05:28:48.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.641+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7781a1190 con 0x7fc7781046e0 2024-04-16T05:28:48.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.642+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7780697e0 con 0x7fc7781046e0 2024-04-16T05:28:48.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.643+0000 7fc76f7fe700 1 -- 172.21.15.134:0/1009388170 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc77001a020 con 0x7fc7781046e0 2024-04-16T05:28:48.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.644+0000 7fc76f7fe700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7640a3c40 0x7fc7640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:28:48.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.644+0000 7fc76f7fe700 1 -- 172.21.15.134:0/1009388170 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fc7700cea80 con 0x7fc7781046e0 2024-04-16T05:28:48.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.644+0000 7fc77e519700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7640a3c40 0x7fc7640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:28:48.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.645+0000 7fc77e519700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7640a3c40 0x7fc7640a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc774003890 tx=0x7fc774005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:28:48.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.649+0000 7fc76f7fe700 1 -- 172.21.15.134:0/1009388170 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc770010070 con 0x7fc7781046e0 2024-04-16T05:28:48.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:48 smithi082 ceph-mon[36195]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:48.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:48 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi082 2024-04-16T05:28:48.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:48 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi082 to 8540M 2024-04-16T05:28:48.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:48 smithi082 ceph-mon[36195]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T05:28:48.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:48 smithi032 ceph-mon[32351]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:48.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:48 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi082 2024-04-16T05:28:48.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:48 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi082 to 8540M 2024-04-16T05:28:48.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:48 smithi032 ceph-mon[32351]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T05:28:48.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:28:48.856+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc778062f30 con 0x7fc7640a3c40 2024-04-16T05:28:49.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:49 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:49.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:49 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:49.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:49 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:49 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:49 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:49 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:49.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:28:49.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:28:49.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:49 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:50.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:50 smithi134 ceph-mon[36153]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:50.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:50 smithi134 ceph-mon[36153]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:50 smithi082 ceph-mon[36195]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:50 smithi082 ceph-mon[36195]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:50.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:50 smithi032 ceph-mon[32351]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:28:50.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:50 smithi032 ceph-mon[32351]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:52.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:52 smithi134 ceph-mon[36153]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:52.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:52 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/3067062470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]: dispatch 2024-04-16T05:28:52.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:52 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]: dispatch 2024-04-16T05:28:52.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:52 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]': finished 2024-04-16T05:28:52.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:52 smithi134 ceph-mon[36153]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T05:28:52.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:28:52.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:52 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/3865793670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:52.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:52 smithi082 ceph-mon[36195]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:52.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:52 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/3067062470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]: dispatch 2024-04-16T05:28:52.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:52 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]: dispatch 2024-04-16T05:28:52.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:52 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]': finished 2024-04-16T05:28:52.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:52 smithi082 ceph-mon[36195]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T05:28:52.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:28:52.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:52 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/3865793670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:52.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:52 smithi032 ceph-mon[32351]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:52.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:52 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/3067062470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]: dispatch 2024-04-16T05:28:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:52 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]: dispatch 2024-04-16T05:28:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:52 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49194b43-0fad-47b3-ad59-e6f24c8865c7"}]': finished 2024-04-16T05:28:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:52 smithi032 ceph-mon[32351]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T05:28:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:28:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:52 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/3865793670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:28:54.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:54 smithi134 ceph-mon[36153]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:54.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:54 smithi082 ceph-mon[36195]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:54.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:54 smithi032 ceph-mon[32351]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:56.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:56 smithi134 ceph-mon[36153]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:56.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:56 smithi082 ceph-mon[36195]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:56.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:56 smithi032 ceph-mon[32351]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:58.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:58 smithi134 ceph-mon[36153]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:58.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:58 smithi082 ceph-mon[36195]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:58.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:58 smithi032 ceph-mon[32351]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:28:59.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T05:28:59.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:28:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:59.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T05:28:59.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:28:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:28:59.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T05:28:59.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:28:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:00.479 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:00 smithi134 ceph-mon[36153]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:00.479 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:00 smithi134 ceph-mon[36153]: Deploying daemon osd.5 on smithi134 2024-04-16T05:29:00.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:00 smithi082 ceph-mon[36195]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:00.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:00 smithi082 ceph-mon[36195]: Deploying daemon osd.5 on smithi134 2024-04-16T05:29:00.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:00 smithi032 ceph-mon[32351]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:00.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:00 smithi032 ceph-mon[32351]: Deploying daemon osd.5 on smithi134 2024-04-16T05:29:02.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:02 smithi134 ceph-mon[36153]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:02.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:02 smithi082 ceph-mon[36195]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:02.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:02 smithi032 ceph-mon[32351]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:04.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:04 smithi134 ceph-mon[36153]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:04.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:04 smithi082 ceph-mon[36195]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:04.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:04 smithi032 ceph-mon[32351]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:05.777 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:05.777 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:05.777 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:05.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:05.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:05.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:05.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:05.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:05.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:06.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:06 smithi134 ceph-mon[36153]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:06.885 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:06 smithi032 ceph-mon[32351]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:06.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:06 smithi082 ceph-mon[36195]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:08.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.280+0000 7fc76f7fe700 1 -- 172.21.15.134:0/1009388170 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc778062f30 con 0x7fc7640a3c40 2024-04-16T05:29:08.279 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 5 on host 'smithi134' 2024-04-16T05:29:08.280 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:07 smithi134 ceph-mon[36153]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:08.280 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:07 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.280 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:07 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.280 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:07 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:08.280 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:07 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:08.280 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:07 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7640a3c40 msgr2=0x7fc7640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7640a3c40 0x7fc7640a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc774003890 tx=0x7fc774005fd0 comp rx=0 tx=0).stop 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7781046e0 msgr2=0x7fc7781a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7781046e0 0x7fc7781a0310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc770007f10 tx=0x7fc77000d3b0 comp rx=0 tx=0).stop 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 shutdown_connections 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7640a3c40 0x7fc7640a6100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7781046e0 0x7fc7781a0310 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778102b40 0x7fc778198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 --2- 172.21.15.134:0/1009388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778103d40 0x7fc778199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 >> 172.21.15.134:0/1009388170 conn(0x7fc7780fe0e0 msgr2=0x7fc778107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 shutdown_connections 2024-04-16T05:29:08.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:29:08.283+0000 7fc78077d700 1 -- 172.21.15.134:0/1009388170 wait complete. 2024-04-16T05:29:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:07 smithi082 ceph-mon[36195]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:07 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:07 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:07 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:07 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:07 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:07 smithi032 ceph-mon[32351]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:08.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:07 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:07 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:07 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:07 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:07 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:08.919 DEBUG:teuthology.orchestra.run.smithi134:osd.5> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.5.service 2024-04-16T05:29:08.921 INFO:tasks.cephadm:Deploying osd.6 on smithi032 with /dev/vg_nvme/lv_2... 2024-04-16T05:29:08.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T05:29:09.179 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:29:09.516 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:09.516 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:09.516 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:09.516 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:09 smithi032 ceph-mon[32351]: from='osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:09.516 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:09 smithi032 ceph-mon[32351]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:09 smithi134 ceph-mon[36153]: from='osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:09 smithi134 ceph-mon[36153]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:09.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:09.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:09.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:09.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:09 smithi082 ceph-mon[36195]: from='osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:09.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:09 smithi082 ceph-mon[36195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T05:29:10.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:10 smithi032 ceph-mon[32351]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:10.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:10 smithi032 ceph-mon[32351]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T05:29:10.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:10 smithi032 ceph-mon[32351]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T05:29:10.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:10 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:10.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:10 smithi032 ceph-mon[32351]: from='osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:29:10.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:10 smithi032 ceph-mon[32351]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:29:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:10 smithi134 ceph-mon[36153]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:10 smithi134 ceph-mon[36153]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T05:29:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:10 smithi134 ceph-mon[36153]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T05:29:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:10 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:10 smithi134 ceph-mon[36153]: from='osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:29:10.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:10 smithi134 ceph-mon[36153]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:29:10.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:10 smithi082 ceph-mon[36195]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:10.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:10 smithi082 ceph-mon[36195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T05:29:10.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:10 smithi082 ceph-mon[36195]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T05:29:10.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:10 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:10.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:10 smithi082 ceph-mon[36195]: from='osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:29:10.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:10 smithi082 ceph-mon[36195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:29:11.056 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:29:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:29:10.734+0000 7f819055c700 -1 osd.5 0 waiting for initial osdmap 2024-04-16T05:29:11.056 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:29:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:29:10.742+0000 7f8188347700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:29:11.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:11 smithi134 ceph-mon[36153]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:29:11.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:11 smithi134 ceph-mon[36153]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T05:29:11.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:11.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:11.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:11 smithi134 ceph-mon[36153]: from='osd.5 ' entity='osd.5' 2024-04-16T05:29:11.475 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T05:29:11.475 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T05:29:11.476 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-04-16T05:29:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-04-16T05:29:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00892737 s, 1.2 GB/s 2024-04-16T05:29:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-04-16T05:29:11.500 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:11 smithi032 ceph-mon[32351]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:29:11.500 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:11 smithi032 ceph-mon[32351]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T05:29:11.500 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:11.500 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:11.500 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:11 smithi032 ceph-mon[32351]: from='osd.5 ' entity='osd.5' 2024-04-16T05:29:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:11 smithi082 ceph-mon[36195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:29:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:11 smithi082 ceph-mon[36195]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T05:29:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:11 smithi082 ceph-mon[36195]: from='osd.5 ' entity='osd.5' 2024-04-16T05:29:12.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_2 2024-04-16T05:29:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:29:12.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:29:12.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] boot 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:12.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:12 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:29:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:29:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] boot 2024-04-16T05:29:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:12 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:29:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:29:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T05:29:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: osd.5 [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] boot 2024-04-16T05:29:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T05:29:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:29:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:12 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:13.565 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:13 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi134 2024-04-16T05:29:13.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:13 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi134 to 8540M 2024-04-16T05:29:13.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:13 smithi032 ceph-mon[32351]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T05:29:13.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:13 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi134 2024-04-16T05:29:13.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:13 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi134 to 8540M 2024-04-16T05:29:13.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:13 smithi134 ceph-mon[36153]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T05:29:13.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:13 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi134 2024-04-16T05:29:13.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:13 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi134 to 8540M 2024-04-16T05:29:13.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:13 smithi082 ceph-mon[36195]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 -- 172.21.15.32:0/2166583766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4104730 msgr2=0x7f6bf410afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/2166583766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4104730 0x7f6bf410afe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0099c0 tx=0x7f6bec009cd0 comp rx=0 tx=0).stop 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 -- 172.21.15.32:0/2166583766 shutdown_connections 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/2166583766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4104730 0x7f6bf410afe0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/2166583766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf4102b90 0x7f6bf4102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/2166583766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf4103d90 0x7f6bf41041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 -- 172.21.15.32:0/2166583766 >> 172.21.15.32:0/2166583766 conn(0x7f6bf40fe130 msgr2=0x7f6bf4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 -- 172.21.15.32:0/2166583766 shutdown_connections 2024-04-16T05:29:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.190+0000 7f6bfac5e700 1 -- 172.21.15.32:0/2166583766 wait complete. 2024-04-16T05:29:14.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bfac5e700 1 Processor -- start 2024-04-16T05:29:14.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bfac5e700 1 -- start start 2024-04-16T05:29:14.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bfac5e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf4102b90 0x7f6bf4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bfac5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4103d90 0x7f6bf41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bfac5e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf4104730 0x7f6bf41a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bfac5e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410e760 con 0x7f6bf4103d90 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bfac5e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410e5e0 con 0x7f6bf4104730 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bfac5e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410e8e0 con 0x7f6bf4102b90 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4103d90 0x7f6bf41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4103d90 0x7f6bf41991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55922/0 (socket says 172.21.15.32:55922) 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bebfff700 1 -- 172.21.15.32:0/128307500 learned_addr learned my addr 172.21.15.32:0/128307500 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bf89fa700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf4102b90 0x7f6bf4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bf91fb700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf4104730 0x7f6bf41a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bebfff700 1 -- 172.21.15.32:0/128307500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf4102b90 msgr2=0x7f6bf4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bebfff700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf4102b90 0x7f6bf4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bebfff700 1 -- 172.21.15.32:0/128307500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf4104730 msgr2=0x7f6bf41a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bebfff700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf4104730 0x7f6bf41a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bebfff700 1 -- 172.21.15.32:0/128307500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf41a0910 con 0x7f6bf4103d90 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.191+0000 7f6bf91fb700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf4104730 0x7f6bf41a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.192+0000 7f6bebfff700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4103d90 0x7f6bf41991d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6be400d950 tx=0x7f6be400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.192+0000 7f6be9ffb700 1 -- 172.21.15.32:0/128307500 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4016070 con 0x7f6bf4103d90 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.192+0000 7f6be9ffb700 1 -- 172.21.15.32:0/128307500 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be4009e20 con 0x7f6bf4103d90 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.192+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf41a0c00 con 0x7f6bf4103d90 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.192+0000 7f6be9ffb700 1 -- 172.21.15.32:0/128307500 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be400f620 con 0x7f6bf4103d90 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.192+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf41a1140 con 0x7f6bf4103d90 2024-04-16T05:29:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.193+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf40697e0 con 0x7f6bf4103d90 2024-04-16T05:29:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.194+0000 7f6be9ffb700 1 -- 172.21.15.32:0/128307500 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6be4019070 con 0x7f6bf4103d90 2024-04-16T05:29:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.195+0000 7f6be9ffb700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bdc0a3c40 0x7f6bdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.195+0000 7f6bf89fa700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bdc0a3c40 0x7f6bdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.195+0000 7f6be9ffb700 1 -- 172.21.15.32:0/128307500 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f6be40cf1a0 con 0x7f6bf4103d90 2024-04-16T05:29:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.195+0000 7f6bf89fa700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bdc0a3c40 0x7f6bdc0a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4199b70 tx=0x7f6bf000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.199+0000 7f6be9ffb700 1 -- 172.21.15.32:0/128307500 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6be409c980 con 0x7f6bf4103d90 2024-04-16T05:29:14.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:14.417+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6bf4062ff0 con 0x7f6bdc0a3c40 2024-04-16T05:29:14.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:14 smithi032 ceph-mon[32351]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:14.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:14 smithi032 ceph-mon[32351]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T05:29:14.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:14 smithi134 ceph-mon[36153]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:14.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:14 smithi134 ceph-mon[36153]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T05:29:14.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:14 smithi082 ceph-mon[36195]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:14 smithi082 ceph-mon[36195]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T05:29:15.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:15 smithi134 ceph-mon[36153]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T05:29:15.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:15 smithi134 ceph-mon[36153]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:15.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:15 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:15.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:15 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:15.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:15 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:15.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:15 smithi082 ceph-mon[36195]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T05:29:15.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:15 smithi082 ceph-mon[36195]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:15.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:15 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:15.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:15 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:15.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:15 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:15.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:15 smithi032 ceph-mon[32351]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T05:29:15.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:15 smithi032 ceph-mon[32351]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:15.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:15 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:15.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:15 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:15.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:15 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:16.613 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:16 smithi032 ceph-mon[32351]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:29:16.614 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:16 smithi032 ceph-mon[32351]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T05:29:16.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:16 smithi134 ceph-mon[36153]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:29:16.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:16 smithi134 ceph-mon[36153]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T05:29:16.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:16 smithi082 ceph-mon[36195]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T05:29:16.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:16 smithi082 ceph-mon[36195]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T05:29:17.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:17 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/483919568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3c0af7-426d-4be6-8d1a-87854c3eb8ed"}]: dispatch 2024-04-16T05:29:17.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:17 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/483919568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc3c0af7-426d-4be6-8d1a-87854c3eb8ed"}]': finished 2024-04-16T05:29:17.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:17 smithi032 ceph-mon[32351]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T05:29:17.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:17 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:17 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/483919568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3c0af7-426d-4be6-8d1a-87854c3eb8ed"}]: dispatch 2024-04-16T05:29:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:17 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/483919568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc3c0af7-426d-4be6-8d1a-87854c3eb8ed"}]': finished 2024-04-16T05:29:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:17 smithi134 ceph-mon[36153]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T05:29:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:17 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:17.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:17 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/483919568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc3c0af7-426d-4be6-8d1a-87854c3eb8ed"}]: dispatch 2024-04-16T05:29:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:17 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/483919568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc3c0af7-426d-4be6-8d1a-87854c3eb8ed"}]': finished 2024-04-16T05:29:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:17 smithi082 ceph-mon[36195]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T05:29:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:17 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:18 smithi134 ceph-mon[36153]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-16T05:29:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:18 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/25544453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:18.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:18 smithi082 ceph-mon[36195]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-16T05:29:18.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:18 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/25544453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:18.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:18 smithi032 ceph-mon[32351]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-16T05:29:18.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:18 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/25544453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:20.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:20 smithi134 ceph-mon[36153]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-16T05:29:20.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:20 smithi082 ceph-mon[36195]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-16T05:29:20.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:20 smithi032 ceph-mon[32351]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-16T05:29:22.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:22 smithi134 ceph-mon[36153]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:22.640 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:22 smithi032 ceph-mon[32351]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:22 smithi082 ceph-mon[36195]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:24.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:24 smithi032 ceph-mon[32351]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:24.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:24 smithi134 ceph-mon[36153]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:24.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:24 smithi082 ceph-mon[36195]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:25.604 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T05:29:25.604 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:25.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T05:29:25.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:25.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T05:29:25.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:26.602 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:26 smithi032 ceph-mon[32351]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:26.602 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:26 smithi032 ceph-mon[32351]: Deploying daemon osd.6 on smithi032 2024-04-16T05:29:26.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:26 smithi134 ceph-mon[36153]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:26.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:26 smithi134 ceph-mon[36153]: Deploying daemon osd.6 on smithi032 2024-04-16T05:29:26.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:26 smithi082 ceph-mon[36195]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:26.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:26 smithi082 ceph-mon[36195]: Deploying daemon osd.6 on smithi032 2024-04-16T05:29:28.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:28 smithi134 ceph-mon[36153]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:28 smithi032 ceph-mon[32351]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:28.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:28 smithi082 ceph-mon[36195]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:30.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:30 smithi134 ceph-mon[36153]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:30.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:30 smithi082 ceph-mon[36195]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:30.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:30 smithi032 ceph-mon[32351]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:31.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:31.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:31.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:31.930 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:31.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:31.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:32.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:32 smithi032 ceph-mon[32351]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:32.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:32 smithi134 ceph-mon[36153]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:32.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:32 smithi082 ceph-mon[36195]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.288+0000 7f6be9ffb700 1 -- 172.21.15.32:0/128307500 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6bf4062ff0 con 0x7f6bdc0a3c40 2024-04-16T05:29:34.287 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 6 on host 'smithi032' 2024-04-16T05:29:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bdc0a3c40 msgr2=0x7f6bdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bdc0a3c40 0x7f6bdc0a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4199b70 tx=0x7f6bf000c040 comp rx=0 tx=0).stop 2024-04-16T05:29:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4103d90 msgr2=0x7f6bf41991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4103d90 0x7f6bf41991d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6be400d950 tx=0x7f6be400dc60 comp rx=0 tx=0).stop 2024-04-16T05:29:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 shutdown_connections 2024-04-16T05:29:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bdc0a3c40 0x7f6bdc0a6100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf4104730 0x7f6bf41a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf4102b90 0x7f6bf4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 --2- 172.21.15.32:0/128307500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf4103d90 0x7f6bf41991d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.290+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 >> 172.21.15.32:0/128307500 conn(0x7f6bf40fe130 msgr2=0x7f6bf4107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.291+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 shutdown_connections 2024-04-16T05:29:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:29:34.291+0000 7f6bfac5e700 1 -- 172.21.15.32:0/128307500 wait complete. 2024-04-16T05:29:34.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:34 smithi032 ceph-mon[32351]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:34.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:34.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:34.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:34 smithi032 ceph-mon[32351]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T05:29:34.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:34 smithi134 ceph-mon[36153]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:34.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:34.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:34.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:34 smithi134 ceph-mon[36153]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T05:29:34.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:34 smithi082 ceph-mon[36195]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:34 smithi082 ceph-mon[36195]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T05:29:35.045 DEBUG:teuthology.orchestra.run.smithi032:osd.6> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.6.service 2024-04-16T05:29:35.047 INFO:tasks.cephadm:Deploying osd.7 on smithi082 with /dev/vg_nvme/lv_2... 2024-04-16T05:29:35.047 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T05:29:35.293 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:29:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:36 smithi134 ceph-mon[36153]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:36 smithi134 ceph-mon[36153]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T05:29:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:36 smithi134 ceph-mon[36153]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T05:29:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:36 smithi134 ceph-mon[36153]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:29:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:36 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:36 smithi082 ceph-mon[36195]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:36 smithi082 ceph-mon[36195]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T05:29:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:36 smithi082 ceph-mon[36195]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T05:29:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:36 smithi082 ceph-mon[36195]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:29:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:36 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:36.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-mon[32351]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-mon[32351]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T05:29:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-mon[32351]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T05:29:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-mon[32351]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:29:36.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:36.983 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:29:36.719+0000 7f4b4227a700 -1 osd.6 0 waiting for initial osdmap 2024-04-16T05:29:36.983 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:29:36 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:29:36.727+0000 7f4b3a065700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:29:37.310 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:37 smithi082 ceph-mon[36195]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:29:37.310 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:37 smithi082 ceph-mon[36195]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T05:29:37.310 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:37 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:37.311 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:37 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:37.311 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:37 smithi082 ceph-mon[36195]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' 2024-04-16T05:29:37.413 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T05:29:37.413 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T05:29:37.413 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-16T05:29:37.413 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-16T05:29:37.413 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0164607 s, 637 MB/s 2024-04-16T05:29:37.413 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-16T05:29:37.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:37 smithi134 ceph-mon[36153]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:29:37.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:37 smithi134 ceph-mon[36153]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T05:29:37.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:37 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:37.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:37 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:37.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:37 smithi134 ceph-mon[36153]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' 2024-04-16T05:29:37.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:37 smithi032 ceph-mon[32351]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:29:37.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:37 smithi032 ceph-mon[32351]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T05:29:37.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:37.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:37.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:37 smithi032 ceph-mon[32351]: from='osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227]' entity='osd.6' 2024-04-16T05:29:38.108 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2024-04-16T05:29:38.349 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:29:38.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:38 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:29:38.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:38 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:29:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:38 smithi082 ceph-mon[36195]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:38 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:38 smithi082 ceph-mon[36195]: osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] boot 2024-04-16T05:29:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:38 smithi082 ceph-mon[36195]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T05:29:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:38 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:38.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:38 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:29:38.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:38 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:29:38.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:38 smithi134 ceph-mon[36153]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:38.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:38 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:38.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:38 smithi134 ceph-mon[36153]: osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] boot 2024-04-16T05:29:38.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:38 smithi134 ceph-mon[36153]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T05:29:38.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:38 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:38 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:38 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:38 smithi032 ceph-mon[32351]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:38 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:38 smithi032 ceph-mon[32351]: osd.6 [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] boot 2024-04-16T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:38 smithi032 ceph-mon[32351]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:38 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:29:39.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- 172.21.15.82:0/811221185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c14103690 msgr2=0x7f8c14105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:39.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/811221185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c14103690 0x7f8c14105a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009990 tx=0x7f8c08009ca0 comp rx=0 tx=0).stop 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- 172.21.15.82:0/811221185 shutdown_connections 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/811221185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c141088f0 0x7f8c1410ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/811221185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c14103690 0x7f8c14105a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/811221185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c14105fc0 0x7f8c141083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- 172.21.15.82:0/811221185 >> 172.21.15.82:0/811221185 conn(0x7f8c140fcfe0 msgr2=0x7f8c140ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- 172.21.15.82:0/811221185 shutdown_connections 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- 172.21.15.82:0/811221185 wait complete. 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 Processor -- start 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- start start 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c14103690 0x7f8c14198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c14105fc0 0x7f8c14199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c141088f0 0x7f8c141a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1410e700 con 0x7f8c14105fc0 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1410e580 con 0x7f8c141088f0 2024-04-16T05:29:39.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.879+0000 7f8c1ab08700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1410e880 con 0x7f8c14103690 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c141088f0 0x7f8c141a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c141088f0 0x7f8c141a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56290/0 (socket says 172.21.15.82:56290) 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c14105fc0 0x7f8c14199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 -- 172.21.15.82:0/4292476348 learned_addr learned my addr 172.21.15.82:0/4292476348 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c188a4700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c14103690 0x7f8c14198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 -- 172.21.15.82:0/4292476348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c14103690 msgr2=0x7f8c14198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c14103690 0x7f8c14198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 -- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c14105fc0 msgr2=0x7f8c14199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c14105fc0 0x7f8c14199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 -- 172.21.15.82:0/4292476348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c141a0950 con 0x7f8c141088f0 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c190a5700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c141088f0 0x7f8c141a0280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c007ee0 tx=0x7f8c0c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c11ffb700 1 -- 172.21.15.82:0/4292476348 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c004bb0 con 0x7f8c141088f0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c141a0c40 con 0x7f8c141088f0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c11ffb700 1 -- 172.21.15.82:0/4292476348 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c004d50 con 0x7f8c141088f0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.880+0000 7f8c11ffb700 1 -- 172.21.15.82:0/4292476348 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c005960 con 0x7f8c141088f0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.881+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141a1180 con 0x7f8c141088f0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.882+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c140697e0 con 0x7f8c141088f0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.883+0000 7f8c11ffb700 1 -- 172.21.15.82:0/4292476348 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8c0c01a020 con 0x7f8c141088f0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.883+0000 7f8c11ffb700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8c040a3c40 0x7f8c040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.883+0000 7f8c11ffb700 1 -- 172.21.15.82:0/4292476348 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f8c0c0ced70 con 0x7f8c141088f0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.883+0000 7f8c188a4700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8c040a3c40 0x7f8c040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:29:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.884+0000 7f8c188a4700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8c040a3c40 0x7f8c040a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c08003890 tx=0x7f8c08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:29:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:39.888+0000 7f8c11ffb700 1 -- 172.21.15.82:0/4292476348 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c0c010070 con 0x7f8c141088f0 2024-04-16T05:29:40.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:40.094+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8c14062f30 con 0x7f8c040a3c40 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi032 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:40.094 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:40.095 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.095 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:39 smithi082 ceph-mon[36195]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T05:29:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi032 2024-04-16T05:29:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:39 smithi134 ceph-mon[36153]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T05:29:40.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi032 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:39 smithi032 ceph-mon[32351]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T05:29:40.996 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:40 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi032 to 5697M 2024-04-16T05:29:40.996 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:40 smithi082 ceph-mon[36195]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:40.996 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:40 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:40.996 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:40 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:40.996 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:40 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:40 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi032 to 5697M 2024-04-16T05:29:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:40 smithi134 ceph-mon[36153]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:40 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:40 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:40 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:40 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi032 to 5697M 2024-04-16T05:29:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:40 smithi032 ceph-mon[32351]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:40 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:29:41.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:40 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:29:41.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:40 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:41.959 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:41 smithi082 ceph-mon[36195]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:41 smithi134 ceph-mon[36153]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:42.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:41 smithi032 ceph-mon[32351]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:29:43.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:42 smithi134 ceph-mon[36153]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:43.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:42 smithi082 ceph-mon[36195]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:43.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:42 smithi032 ceph-mon[32351]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:43 smithi134 ceph-mon[36153]: from='client.? 172.21.15.82:0/275819886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]: dispatch 2024-04-16T05:29:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:43 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]: dispatch 2024-04-16T05:29:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:43 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]': finished 2024-04-16T05:29:44.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:43 smithi134 ceph-mon[36153]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T05:29:44.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:43 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:29:44.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:43 smithi134 ceph-mon[36153]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:44.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:43 smithi134 ceph-mon[36153]: from='client.? 172.21.15.82:0/3623245353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:43 smithi082 ceph-mon[36195]: from='client.? 172.21.15.82:0/275819886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]: dispatch 2024-04-16T05:29:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:43 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]: dispatch 2024-04-16T05:29:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:43 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]': finished 2024-04-16T05:29:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:43 smithi082 ceph-mon[36195]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T05:29:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:43 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:29:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:43 smithi082 ceph-mon[36195]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:43 smithi082 ceph-mon[36195]: from='client.? 172.21.15.82:0/3623245353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:43 smithi032 ceph-mon[32351]: from='client.? 172.21.15.82:0/275819886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]: dispatch 2024-04-16T05:29:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:43 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]: dispatch 2024-04-16T05:29:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:43 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a3fe236-c01c-45f0-896a-f97bee5a4fb3"}]': finished 2024-04-16T05:29:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:43 smithi032 ceph-mon[32351]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T05:29:44.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:43 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:29:44.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:43 smithi032 ceph-mon[32351]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:44.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:43 smithi032 ceph-mon[32351]: from='client.? 172.21.15.82:0/3623245353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:29:46.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:45 smithi032 ceph-mon[32351]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:46.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:45 smithi134 ceph-mon[36153]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:46.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:45 smithi082 ceph-mon[36195]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:47.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:47 smithi032 ceph-mon[32351]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:47.996 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:47 smithi082 ceph-mon[36195]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:48.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:47 smithi134 ceph-mon[36153]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:50.074 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:49 smithi082 ceph-mon[36195]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:50.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:49 smithi032 ceph-mon[32351]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:50.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:49 smithi134 ceph-mon[36153]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:51.036 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:50 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T05:29:51.036 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:50 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:51.036 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:50 smithi082 ceph-mon[36195]: Deploying daemon osd.7 on smithi082 2024-04-16T05:29:51.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:50 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T05:29:51.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:50 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:51.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:50 smithi032 ceph-mon[32351]: Deploying daemon osd.7 on smithi082 2024-04-16T05:29:51.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:50 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T05:29:51.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:50 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:51.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:50 smithi134 ceph-mon[36153]: Deploying daemon osd.7 on smithi082 2024-04-16T05:29:52.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:51 smithi082 ceph-mon[36195]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:52.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:51 smithi032 ceph-mon[32351]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:52.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:51 smithi134 ceph-mon[36153]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:54.068 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:53 smithi082 ceph-mon[36195]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:54.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:53 smithi032 ceph-mon[32351]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:54.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:53 smithi134 ceph-mon[36153]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:56.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:55 smithi032 ceph-mon[32351]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:56.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:55 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:56.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:55 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:56.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:55 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:56.384 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:55 smithi082 ceph-mon[36195]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:56.384 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:55 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:56.384 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:55 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:56.384 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:55 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:56.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:55 smithi134 ceph-mon[36153]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:56.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:55 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:56.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:55 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:56.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:55 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:58.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:57 smithi082 ceph-mon[36195]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:58.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:57 smithi032 ceph-mon[32351]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:58.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:57 smithi134 ceph-mon[36153]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:29:59.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.715+0000 7f8c11ffb700 1 -- 172.21.15.82:0/4292476348 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8c14062f30 con 0x7f8c040a3c40 2024-04-16T05:29:59.713 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 7 on host 'smithi082' 2024-04-16T05:29:59.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8c040a3c40 msgr2=0x7f8c040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:59.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8c040a3c40 0x7f8c040a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c08003890 tx=0x7f8c08005fd0 comp rx=0 tx=0).stop 2024-04-16T05:29:59.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c141088f0 msgr2=0x7f8c141a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c141088f0 0x7f8c141a0280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c007ee0 tx=0x7f8c0c00d3b0 comp rx=0 tx=0).stop 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 shutdown_connections 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8c040a3c40 0x7f8c040a6100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c141088f0 0x7f8c141a0280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c14103690 0x7f8c14198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 --2- 172.21.15.82:0/4292476348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c14105fc0 0x7f8c14199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.716+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 >> 172.21.15.82:0/4292476348 conn(0x7f8c140fcfe0 msgr2=0x7f8c14103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.717+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 shutdown_connections 2024-04-16T05:29:59.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:29:59.717+0000 7f8c1ab08700 1 -- 172.21.15.82:0/4292476348 wait complete. 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:29:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:29:59 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:29:59 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:29:59 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:00.375 DEBUG:teuthology.orchestra.run.smithi082:osd.7> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.7.service 2024-04-16T05:30:00.378 INFO:tasks.cephadm:Deploying osd.8 on smithi134 with /dev/vg_nvme/lv_2... 2024-04-16T05:30:00.378 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T05:30:00.639 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:30:00.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-mon[36195]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:00.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-mon[36195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T05:30:00.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-mon[36195]: from='osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:30:00.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-mon[36195]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T05:30:00.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:00.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-mon[36195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:30:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-mon[36195]: overall HEALTH_OK 2024-04-16T05:30:00.901 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:30:00.775+0000 7f78b1843700 -1 osd.7 0 waiting for initial osdmap 2024-04-16T05:30:00.901 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:30:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:30:00.784+0000 7f78ad636700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:30:01.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:00 smithi134 ceph-mon[36153]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:01.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:00 smithi134 ceph-mon[36153]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T05:30:01.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:00 smithi134 ceph-mon[36153]: from='osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:30:01.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:00 smithi134 ceph-mon[36153]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T05:30:01.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:00 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:01.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:00 smithi134 ceph-mon[36153]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:30:01.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:00 smithi134 ceph-mon[36153]: overall HEALTH_OK 2024-04-16T05:30:01.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:00 smithi032 ceph-mon[32351]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:00 smithi032 ceph-mon[32351]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T05:30:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:00 smithi032 ceph-mon[32351]: from='osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:30:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:00 smithi032 ceph-mon[32351]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T05:30:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:00 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:00 smithi032 ceph-mon[32351]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:30:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:00 smithi032 ceph-mon[32351]: overall HEALTH_OK 2024-04-16T05:30:01.877 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:01 smithi082 ceph-mon[36195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:30:01.878 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:01 smithi082 ceph-mon[36195]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T05:30:01.878 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:01 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:01.878 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:01 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:02.045 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:01 smithi134 ceph-mon[36153]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:30:02.045 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:01 smithi134 ceph-mon[36153]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T05:30:02.045 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:01 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:02.045 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:01 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:02.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:01 smithi032 ceph-mon[32351]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:30:02.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:01 smithi032 ceph-mon[32351]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T05:30:02.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:01 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:02.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:01 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:02.704 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T05:30:02.704 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T05:30:02.704 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-16T05:30:02.704 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-16T05:30:02.704 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113424 s, 924 MB/s 2024-04-16T05:30:02.704 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-16T05:30:02.947 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:02 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:30:02.947 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:02 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:30:02.947 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:02 smithi082 ceph-mon[36195]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:02.947 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:02 smithi082 ceph-mon[36195]: osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] boot 2024-04-16T05:30:02.947 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:02 smithi082 ceph-mon[36195]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T05:30:02.948 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:02 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:02.948 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:02 smithi082 ceph-mon[36195]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T05:30:03.050 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:02 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:30:03.050 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:02 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:30:03.051 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:02 smithi134 ceph-mon[36153]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:03.051 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:02 smithi134 ceph-mon[36153]: osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] boot 2024-04-16T05:30:03.051 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:02 smithi134 ceph-mon[36153]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T05:30:03.051 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:02 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:03.051 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:02 smithi134 ceph-mon[36153]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T05:30:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:02 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:30:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:02 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:30:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:02 smithi032 ceph-mon[32351]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T05:30:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:02 smithi032 ceph-mon[32351]: osd.7 [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] boot 2024-04-16T05:30:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:02 smithi032 ceph-mon[32351]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T05:30:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:30:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:02 smithi032 ceph-mon[32351]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T05:30:03.376 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-04-16T05:30:03.623 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:30:04.606 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:04.606 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi082 2024-04-16T05:30:04.606 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi082 to 5693M 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:04 smithi134 ceph-mon[36153]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T05:30:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi082 2024-04-16T05:30:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi082 to 5693M 2024-04-16T05:30:04.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:04.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:04.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:04 smithi082 ceph-mon[36195]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T05:30:04.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:04.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi082 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi082 to 5693M 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:04.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:04 smithi032 ceph-mon[32351]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T05:30:05.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.233+0000 7f9b3488c700 1 -- 172.21.15.134:0/2980405829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30103e00 msgr2=0x7f9b30104260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:05.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.233+0000 7f9b3488c700 1 --2- 172.21.15.134:0/2980405829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30103e00 0x7f9b30104260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009990 tx=0x7f9b18009ca0 comp rx=0 tx=0).stop 2024-04-16T05:30:05.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.233+0000 7f9b3488c700 1 -- 172.21.15.134:0/2980405829 shutdown_connections 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.233+0000 7f9b3488c700 1 --2- 172.21.15.134:0/2980405829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b301047a0 0x7f9b3010b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.233+0000 7f9b3488c700 1 --2- 172.21.15.134:0/2980405829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b30100e20 0x7f9b30101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.233+0000 7f9b3488c700 1 --2- 172.21.15.134:0/2980405829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30103e00 0x7f9b30104260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.233+0000 7f9b3488c700 1 -- 172.21.15.134:0/2980405829 >> 172.21.15.134:0/2980405829 conn(0x7f9b300fc760 msgr2=0x7f9b300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 -- 172.21.15.134:0/2980405829 shutdown_connections 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 -- 172.21.15.134:0/2980405829 wait complete. 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 Processor -- start 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 -- start start 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30100e20 0x7f9b30198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b30103e00 0x7f9b301992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b301047a0 0x7f9b301a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010e7b0 con 0x7f9b301047a0 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010e630 con 0x7f9b30103e00 2024-04-16T05:30:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.234+0000 7f9b3488c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010e930 con 0x7f9b30100e20 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30100e20 0x7f9b30198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30100e20 0x7f9b30198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56474/0 (socket says 172.21.15.134:56474) 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 -- 172.21.15.134:0/1296682738 learned_addr learned my addr 172.21.15.134:0/1296682738 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b27fff700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b30103e00 0x7f9b301992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2ed9d700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b301047a0 0x7f9b301a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 -- 172.21.15.134:0/1296682738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b30103e00 msgr2=0x7f9b301992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b30103e00 0x7f9b301992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 -- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b301047a0 msgr2=0x7f9b301a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b301047a0 0x7f9b301a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 -- 172.21.15.134:0/1296682738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b301a09c0 con 0x7f9b30100e20 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b2e59c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30100e20 0x7f9b30198d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b530 tx=0x7f9b2000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b277fe700 1 -- 172.21.15.134:0/1296682738 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20015020 con 0x7f9b30100e20 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b277fe700 1 -- 172.21.15.134:0/1296682738 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b20011d10 con 0x7f9b30100e20 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b277fe700 1 -- 172.21.15.134:0/1296682738 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000f2e0 con 0x7f9b30100e20 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b301a0cb0 con 0x7f9b30100e20 2024-04-16T05:30:05.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.235+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b301a11a0 con 0x7f9b30100e20 2024-04-16T05:30:05.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.236+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b300697e0 con 0x7f9b30100e20 2024-04-16T05:30:05.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.237+0000 7f9b277fe700 1 -- 172.21.15.134:0/1296682738 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b20018020 con 0x7f9b30100e20 2024-04-16T05:30:05.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.238+0000 7f9b277fe700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:05.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.238+0000 7f9b277fe700 1 -- 172.21.15.134:0/1296682738 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f9b200cf5d0 con 0x7f9b30100e20 2024-04-16T05:30:05.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.238+0000 7f9b27fff700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:05.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.238+0000 7f9b27fff700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800b5c0 tx=0x7f9b18005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:05.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.242+0000 7f9b277fe700 1 -- 172.21.15.134:0/1296682738 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b2001f020 con 0x7f9b30100e20 2024-04-16T05:30:05.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:05.447+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9b30062f30 con 0x7f9b1c0a3c40 2024-04-16T05:30:05.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:05 smithi134 ceph-mon[36153]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T05:30:05.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:05.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:05.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:05.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:05 smithi082 ceph-mon[36195]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T05:30:05.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:05.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:05.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:05.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:05 smithi032 ceph-mon[32351]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T05:30:05.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:05.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:05.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:06 smithi134 ceph-mon[36153]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:06 smithi134 ceph-mon[36153]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:06 smithi134 ceph-mon[36153]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T05:30:06.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:06 smithi082 ceph-mon[36195]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:06.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:06 smithi082 ceph-mon[36195]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:06.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:06 smithi082 ceph-mon[36195]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T05:30:06.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:06 smithi032 ceph-mon[32351]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:06.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:06 smithi032 ceph-mon[32351]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:06.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:06 smithi032 ceph-mon[32351]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T05:30:08.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:08 smithi134 ceph-mon[36153]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:08.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:08 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/1848610635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]: dispatch 2024-04-16T05:30:08.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:08 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]: dispatch 2024-04-16T05:30:08.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:08 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]': finished 2024-04-16T05:30:08.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:08 smithi134 ceph-mon[36153]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T05:30:08.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:08 smithi082 ceph-mon[36195]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:08 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/1848610635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]: dispatch 2024-04-16T05:30:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:08 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]: dispatch 2024-04-16T05:30:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:08 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]': finished 2024-04-16T05:30:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:08 smithi082 ceph-mon[36195]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T05:30:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:08.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:08 smithi032 ceph-mon[32351]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:08.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:08 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/1848610635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]: dispatch 2024-04-16T05:30:08.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:08 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]: dispatch 2024-04-16T05:30:08.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:08 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3917ee4-9f4e-400e-be3e-0e0558dd8881"}]': finished 2024-04-16T05:30:08.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:08 smithi032 ceph-mon[32351]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T05:30:08.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:09.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:09 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/3388504840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:09.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:09 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/3388504840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:09.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:09 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/3388504840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:10.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:10 smithi134 ceph-mon[36153]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:10.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:10 smithi082 ceph-mon[36195]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:10.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:10 smithi032 ceph-mon[32351]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:12.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:12 smithi134 ceph-mon[36153]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:12.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:12 smithi082 ceph-mon[36195]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:12.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:12 smithi032 ceph-mon[32351]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:14 smithi134 ceph-mon[36153]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:14.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:14 smithi082 ceph-mon[36195]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:14.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:14 smithi032 ceph-mon[32351]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:15.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:15 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T05:30:15.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:15 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:15.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:15 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T05:30:15.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:15 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:15.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:15 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T05:30:15.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:15 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:16.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:16 smithi032 ceph-mon[32351]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:16.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:16 smithi032 ceph-mon[32351]: Deploying daemon osd.8 on smithi134 2024-04-16T05:30:16.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:16 smithi134 ceph-mon[36153]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:16.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:16 smithi134 ceph-mon[36153]: Deploying daemon osd.8 on smithi134 2024-04-16T05:30:16.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:16 smithi082 ceph-mon[36195]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:16.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:16 smithi082 ceph-mon[36195]: Deploying daemon osd.8 on smithi134 2024-04-16T05:30:18.835 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:18 smithi134 ceph-mon[36153]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:18 smithi082 ceph-mon[36195]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:18.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:18 smithi032 ceph-mon[32351]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:20.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:20 smithi134 ceph-mon[36153]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:20.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:20 smithi082 ceph-mon[36195]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:20.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:20 smithi032 ceph-mon[32351]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:22.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:22 smithi134 ceph-mon[36153]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:22.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:22.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:22.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:22 smithi032 ceph-mon[32351]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:22 smithi082 ceph-mon[36195]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:24.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:24 smithi032 ceph-mon[32351]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:24.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:24.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:24.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:24 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.612 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:24 smithi134 ceph-mon[36153]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:24.612 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.612 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.612 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:24.612 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:24.612 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:24 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:24 smithi082 ceph-mon[36195]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:24.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:24 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:24 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:24 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:24.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:24 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:24.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:24 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:24.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.919+0000 7f9b277fe700 1 -- 172.21.15.134:0/1296682738 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b30062f30 con 0x7f9b1c0a3c40 2024-04-16T05:30:24.919 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 8 on host 'smithi134' 2024-04-16T05:30:24.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9b1c0a3c40 msgr2=0x7f9b1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:24.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800b5c0 tx=0x7f9b18005fb0 comp rx=0 tx=0).stop 2024-04-16T05:30:24.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30100e20 msgr2=0x7f9b30198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:24.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30100e20 0x7f9b30198d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b530 tx=0x7f9b2000b8f0 comp rx=0 tx=0).stop 2024-04-16T05:30:24.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 shutdown_connections 2024-04-16T05:30:24.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:24.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b301047a0 0x7f9b301a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:24.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30100e20 0x7f9b30198d60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:24.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 --2- 172.21.15.134:0/1296682738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b30103e00 0x7f9b301992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:24.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 >> 172.21.15.134:0/1296682738 conn(0x7f9b300fc760 msgr2=0x7f9b30107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:24.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 shutdown_connections 2024-04-16T05:30:24.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:30:24.922+0000 7f9b3488c700 1 -- 172.21.15.134:0/1296682738 wait complete. 2024-04-16T05:30:25.592 DEBUG:teuthology.orchestra.run.smithi134:osd.8> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.8.service 2024-04-16T05:30:25.594 INFO:tasks.cephadm:Deploying osd.9 on smithi032 with /dev/vg_nvme/lv_1... 2024-04-16T05:30:25.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T05:30:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:30:26.213 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:25 smithi032 ceph-mon[32351]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:26.213 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:26.213 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:26.213 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:26.213 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:25 smithi032 ceph-mon[32351]: from='osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:26.213 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:25 smithi032 ceph-mon[32351]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:26.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:25 smithi134 ceph-mon[36153]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:26.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:26.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:26.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:26.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:25 smithi134 ceph-mon[36153]: from='osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:26.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:25 smithi134 ceph-mon[36153]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:26.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:25 smithi082 ceph-mon[36195]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:26.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:26.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:26.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:26.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:25 smithi082 ceph-mon[36195]: from='osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:26.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:25 smithi082 ceph-mon[36195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T05:30:27.060 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:26 smithi134 ceph-mon[36153]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T05:30:27.061 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:26 smithi134 ceph-mon[36153]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T05:30:27.061 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:26 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:27.061 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:26 smithi134 ceph-mon[36153]: from='osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:30:27.061 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:26 smithi134 ceph-mon[36153]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:30:27.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:26 smithi032 ceph-mon[32351]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T05:30:27.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:26 smithi032 ceph-mon[32351]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T05:30:27.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:26 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:27.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:26 smithi032 ceph-mon[32351]: from='osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:30:27.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:26 smithi032 ceph-mon[32351]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:30:27.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:26 smithi082 ceph-mon[36195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T05:30:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:26 smithi082 ceph-mon[36195]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T05:30:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:26 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:26 smithi082 ceph-mon[36195]: from='osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:30:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:26 smithi082 ceph-mon[36195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:30:27.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:30:27.352+0000 7f3720344700 -1 osd.8 0 waiting for initial osdmap 2024-04-16T05:30:27.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:30:27.363+0000 7f371812f700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:30:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T05:30:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T05:30:28.179 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-04-16T05:30:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-04-16T05:30:28.179 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00860035 s, 1.2 GB/s 2024-04-16T05:30:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-04-16T05:30:28.179 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:28.179 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:30:28.179 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T05:30:28.179 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: from='osd.8 ' entity='osd.8' 2024-04-16T05:30:28.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] boot 2024-04-16T05:30:28.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T05:30:28.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:27 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.306 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:28.306 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:30:28.306 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T05:30:28.307 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.307 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.307 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: from='osd.8 ' entity='osd.8' 2024-04-16T05:30:28.307 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] boot 2024-04-16T05:30:28.307 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T05:30:28.307 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:27 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: from='osd.8 ' entity='osd.8' 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: osd.8 [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] boot 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T05:30:28.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:27 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:30:29.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_1 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi134 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi134 to 5693M 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:29.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:29.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.254 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:30:29.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:30:29.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:30:29.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T05:30:29.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi134 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi134 to 5693M 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:29.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi134 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi134 to 5693M 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:29.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:29.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:30.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:30 smithi032 ceph-mon[32351]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:30.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:30 smithi032 ceph-mon[32351]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T05:30:30.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:30 smithi134 ceph-mon[36153]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:30.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:30 smithi134 ceph-mon[36153]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T05:30:30.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:30 smithi082 ceph-mon[36195]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:30.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:30 smithi082 ceph-mon[36195]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T05:30:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 -- 172.21.15.32:0/874729399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144101a50 msgr2=0x7f5144101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 --2- 172.21.15.32:0/874729399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144101a50 0x7f5144101e70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f51340099c0 tx=0x7f5134009cd0 comp rx=0 tx=0).stop 2024-04-16T05:30:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 -- 172.21.15.32:0/874729399 shutdown_connections 2024-04-16T05:30:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 --2- 172.21.15.32:0/874729399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51441035f0 0x7f5144109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 --2- 172.21.15.32:0/874729399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144101a50 0x7f5144101e70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 --2- 172.21.15.32:0/874729399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5144102c50 0x7f51441030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 -- 172.21.15.32:0/874729399 >> 172.21.15.32:0/874729399 conn(0x7f51440fcff0 msgr2=0x7f51440ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 -- 172.21.15.32:0/874729399 shutdown_connections 2024-04-16T05:30:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.946+0000 7f514a3ae700 1 -- 172.21.15.32:0/874729399 wait complete. 2024-04-16T05:30:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514a3ae700 1 Processor -- start 2024-04-16T05:30:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514a3ae700 1 -- start start 2024-04-16T05:30:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514a3ae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5144101a50 0x7f5144197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514a3ae700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5144102c50 0x7f51441981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514a3ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51441035f0 0x7f514419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514a3ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f514410d660 con 0x7f51441035f0 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514a3ae700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f514410d4e0 con 0x7f5144101a50 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514a3ae700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f514410d7e0 con 0x7f5144102c50 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514894b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51441035f0 0x7f514419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514894b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51441035f0 0x7f514419f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37162/0 (socket says 172.21.15.32:37162) 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f514894b700 1 -- 172.21.15.32:0/1833204003 learned_addr learned my addr 172.21.15.32:0/1833204003 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f51437fe700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5144102c50 0x7f51441981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.947+0000 7f5143fff700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5144101a50 0x7f5144197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f514894b700 1 -- 172.21.15.32:0/1833204003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5144102c50 msgr2=0x7f51441981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f514894b700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5144102c50 0x7f51441981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f514894b700 1 -- 172.21.15.32:0/1833204003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5144101a50 msgr2=0x7f5144197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f514894b700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5144101a50 0x7f5144197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f514894b700 1 -- 172.21.15.32:0/1833204003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514406b220 con 0x7f51441035f0 2024-04-16T05:30:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f514894b700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51441035f0 0x7f514419f1e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f513c007ee0 tx=0x7f513c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f51417fa700 1 -- 172.21.15.32:0/1833204003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c004bb0 con 0x7f51441035f0 2024-04-16T05:30:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f51417fa700 1 -- 172.21.15.32:0/1833204003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513c004d50 con 0x7f51441035f0 2024-04-16T05:30:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514406b510 con 0x7f51441035f0 2024-04-16T05:30:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f51417fa700 1 -- 172.21.15.32:0/1833204003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c005960 con 0x7f51441035f0 2024-04-16T05:30:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.948+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514406ba50 con 0x7f51441035f0 2024-04-16T05:30:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.949+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51440697e0 con 0x7f51441035f0 2024-04-16T05:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.950+0000 7f51417fa700 1 -- 172.21.15.32:0/1833204003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f513c01a020 con 0x7f51441035f0 2024-04-16T05:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.951+0000 7f51417fa700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f512c0a3c40 0x7f512c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.951+0000 7f5143fff700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f512c0a3c40 0x7f512c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.951+0000 7f51417fa700 1 -- 172.21.15.32:0/1833204003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7f513c0cf120 con 0x7f51441035f0 2024-04-16T05:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.951+0000 7f5143fff700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f512c0a3c40 0x7f512c0a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5134009990 tx=0x7f5134005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:30.956+0000 7f51417fa700 1 -- 172.21.15.32:0/1833204003 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f513c010070 con 0x7f51441035f0 2024-04-16T05:30:31.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:31.174+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f514406bcb0 con 0x7f512c0a3c40 2024-04-16T05:30:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:31 smithi134 ceph-mon[36153]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T05:30:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:31.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:31 smithi082 ceph-mon[36195]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T05:30:31.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:31.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:31.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:31.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:31 smithi032 ceph-mon[32351]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T05:30:31.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:31.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:31.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:32.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:32 smithi134 ceph-mon[36153]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:32.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:32 smithi134 ceph-mon[36153]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:32.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:32 smithi134 ceph-mon[36153]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T05:30:32.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:32 smithi082 ceph-mon[36195]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:32.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:32 smithi082 ceph-mon[36195]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:32.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:32 smithi082 ceph-mon[36195]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T05:30:32.923 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:32 smithi032 ceph-mon[32351]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:32.923 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:32 smithi032 ceph-mon[32351]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:32.923 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:32 smithi032 ceph-mon[32351]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T05:30:34.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:34 smithi032 ceph-mon[32351]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-16T05:30:34.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:34 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3811979723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981db02c-51a8-4779-bc37-1c111ef20bc9"}]: dispatch 2024-04-16T05:30:34.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:34 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3811979723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "981db02c-51a8-4779-bc37-1c111ef20bc9"}]': finished 2024-04-16T05:30:34.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:34 smithi032 ceph-mon[32351]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T05:30:34.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:34.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:34 smithi134 ceph-mon[36153]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-16T05:30:34.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:34 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3811979723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981db02c-51a8-4779-bc37-1c111ef20bc9"}]: dispatch 2024-04-16T05:30:34.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:34 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3811979723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "981db02c-51a8-4779-bc37-1c111ef20bc9"}]': finished 2024-04-16T05:30:34.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:34 smithi134 ceph-mon[36153]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T05:30:34.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:34 smithi082 ceph-mon[36195]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-16T05:30:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:34 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3811979723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981db02c-51a8-4779-bc37-1c111ef20bc9"}]: dispatch 2024-04-16T05:30:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:34 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3811979723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "981db02c-51a8-4779-bc37-1c111ef20bc9"}]': finished 2024-04-16T05:30:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:34 smithi082 ceph-mon[36195]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T05:30:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:35.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:35 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2754953008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:35.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:35 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2754953008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:35.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:35 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2754953008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:30:36.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:36 smithi134 ceph-mon[36153]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-16T05:30:36.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:36 smithi082 ceph-mon[36195]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-16T05:30:36.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:36 smithi032 ceph-mon[32351]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-16T05:30:38.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:38 smithi134 ceph-mon[36153]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-16T05:30:38.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:38 smithi082 ceph-mon[36195]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-16T05:30:38.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:38 smithi032 ceph-mon[32351]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-16T05:30:40.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:40 smithi032 ceph-mon[32351]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:30:40.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:40 smithi134 ceph-mon[36153]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:30:40.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:40 smithi082 ceph-mon[36195]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:30:42.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:42 smithi134 ceph-mon[36153]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-16T05:30:42.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:42 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T05:30:42.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:42 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:42.886 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:42 smithi032 ceph-mon[32351]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-16T05:30:42.886 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:42 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T05:30:42.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:42 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:42.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:42 smithi082 ceph-mon[36195]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-16T05:30:42.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:42 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T05:30:42.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:42 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:43.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:43 smithi082 ceph-mon[36195]: Deploying daemon osd.9 on smithi032 2024-04-16T05:30:43.954 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:43 smithi032 ceph-mon[32351]: Deploying daemon osd.9 on smithi032 2024-04-16T05:30:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:43 smithi134 ceph-mon[36153]: Deploying daemon osd.9 on smithi032 2024-04-16T05:30:44.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:44 smithi032 ceph-mon[32351]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:44.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:44 smithi082 ceph-mon[36195]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:44 smithi134 ceph-mon[36153]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:46.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:46 smithi082 ceph-mon[36195]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:46.948 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:46 smithi134 ceph-mon[36153]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:46.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:46 smithi032 ceph-mon[32351]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:48.900 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:48 smithi032 ceph-mon[32351]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:48.900 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:48.900 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:48.900 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:48.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:48 smithi082 ceph-mon[36195]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:48.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:48.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:48.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:49.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:48 smithi134 ceph-mon[36153]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:49.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:49.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:49.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:50.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:50 smithi032 ceph-mon[32351]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:50.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:50 smithi082 ceph-mon[36195]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:51.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:50 smithi134 ceph-mon[36153]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:52.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.387+0000 7f51417fa700 1 -- 172.21.15.32:0/1833204003 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f514406bcb0 con 0x7f512c0a3c40 2024-04-16T05:30:52.385 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 9 on host 'smithi032' 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f512c0a3c40 msgr2=0x7f512c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f512c0a3c40 0x7f512c0a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5134009990 tx=0x7f5134005fb0 comp rx=0 tx=0).stop 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51441035f0 msgr2=0x7f514419f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51441035f0 0x7f514419f1e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f513c007ee0 tx=0x7f513c00d3b0 comp rx=0 tx=0).stop 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 shutdown_connections 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f512c0a3c40 0x7f512c0a6100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51441035f0 0x7f514419f1e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5144101a50 0x7f5144197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 --2- 172.21.15.32:0/1833204003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5144102c50 0x7f51441981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.389+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 >> 172.21.15.32:0/1833204003 conn(0x7f51440fcff0 msgr2=0x7f5144105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.390+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 shutdown_connections 2024-04-16T05:30:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:30:52.390+0000 7f514a3ae700 1 -- 172.21.15.32:0/1833204003 wait complete. 2024-04-16T05:30:52.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:52.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T05:30:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:52 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:52.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:52.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:52.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:52.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T05:30:52.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:52 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:52 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:30:53.143 DEBUG:teuthology.orchestra.run.smithi032:osd.9> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.9.service 2024-04-16T05:30:53.145 INFO:tasks.cephadm:Deploying osd.10 on smithi082 with /dev/vg_nvme/lv_1... 2024-04-16T05:30:53.146 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T05:30:53.390 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:30:53.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:53 smithi032 ceph-mon[32351]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T05:30:53.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:53 smithi032 ceph-mon[32351]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T05:30:53.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:53 smithi032 ceph-mon[32351]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:30:53.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:53 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:53.733 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:30:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:30:53.659+0000 7f23a0ddc700 -1 osd.9 0 waiting for initial osdmap 2024-04-16T05:30:53.733 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:30:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:30:53.669+0000 7f239cbcf700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:30:53.795 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:53 smithi082 ceph-mon[36195]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T05:30:53.795 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:53 smithi082 ceph-mon[36195]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T05:30:53.795 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:53 smithi082 ceph-mon[36195]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:30:53.795 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:53 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:53 smithi134 ceph-mon[36153]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T05:30:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:53 smithi134 ceph-mon[36153]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T05:30:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:53 smithi134 ceph-mon[36153]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-04-16T05:30:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:53 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:54.713 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:54 smithi082 ceph-mon[36195]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:54.713 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:54 smithi082 ceph-mon[36195]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:30:54.714 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:54 smithi082 ceph-mon[36195]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T05:30:54.714 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:54 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:54.714 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:54 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:54 smithi032 ceph-mon[32351]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:54 smithi032 ceph-mon[32351]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:30:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:54 smithi032 ceph-mon[32351]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T05:30:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:54 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:54 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:54 smithi134 ceph-mon[36153]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-16T05:30:55.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:54 smithi134 ceph-mon[36153]: from='osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-04-16T05:30:55.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:54 smithi134 ceph-mon[36153]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T05:30:55.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:54 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:55.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:54 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T05:30:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T05:30:55.275 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-16T05:30:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-16T05:30:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0119823 s, 875 MB/s 2024-04-16T05:30:55.275 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 2024-04-16T05:30:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-16T05:30:55.835 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:55 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:30:55.835 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:55 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:30:55.835 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:55 smithi032 ceph-mon[32351]: osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] boot 2024-04-16T05:30:55.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:55 smithi032 ceph-mon[32351]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T05:30:55.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:55 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:55.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:55 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:30:55.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:55 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:30:55.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:55 smithi082 ceph-mon[36195]: osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] boot 2024-04-16T05:30:55.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:55 smithi082 ceph-mon[36195]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T05:30:55.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:55 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:55.968 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2024-04-16T05:30:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:55 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:30:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:55 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:30:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:55 smithi134 ceph-mon[36153]: osd.9 [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] boot 2024-04-16T05:30:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:55 smithi134 ceph-mon[36153]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T05:30:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:55 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:30:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:30:56.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:30:56.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.073 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:30:57.073 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T05:30:57.073 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.073 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.073 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:57.073 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:57.074 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:57.074 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:57.074 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.074 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:57.074 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:57.074 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:30:57.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:30:57.912 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:57 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi032 2024-04-16T05:30:57.912 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:57 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi032 to 4273M 2024-04-16T05:30:57.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 -- 172.21.15.82:0/558361083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 msgr2=0x7f2e0c109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 --2- 172.21.15.82:0/558361083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 0x7f2e0c109ec0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400b0d0 tx=0x7f2e0400b3e0 comp rx=0 tx=0).stop 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 -- 172.21.15.82:0/558361083 shutdown_connections 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 --2- 172.21.15.82:0/558361083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 0x7f2e0c109ec0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 --2- 172.21.15.82:0/558361083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0c101a70 0x7f2e0c101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 --2- 172.21.15.82:0/558361083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0c102c70 0x7f2e0c1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 -- 172.21.15.82:0/558361083 >> 172.21.15.82:0/558361083 conn(0x7f2e0c0fd050 msgr2=0x7f2e0c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 -- 172.21.15.82:0/558361083 shutdown_connections 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 -- 172.21.15.82:0/558361083 wait complete. 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 Processor -- start 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.880+0000 7f2e11507700 1 -- start start 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e11507700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0c101a70 0x7f2e0c06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e11507700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0c102c70 0x7f2e0c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:57.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e11507700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 0x7f2e0c1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e11507700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10d6b0 con 0x7f2e0c102c70 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e11507700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10d530 con 0x7f2e0c103610 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e11507700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10d830 con 0x7f2e0c101a70 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 0x7f2e0c1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 0x7f2e0c1a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54798/0 (socket says 172.21.15.82:54798) 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 -- 172.21.15.82:0/1743316969 learned_addr learned my addr 172.21.15.82:0/1743316969 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0affd700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0c101a70 0x7f2e0c06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e03fff700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0c102c70 0x7f2e0c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 -- 172.21.15.82:0/1743316969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0c101a70 msgr2=0x7f2e0c06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0c101a70 0x7f2e0c06bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 -- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0c102c70 msgr2=0x7f2e0c19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:30:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0c102c70 0x7f2e0c19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 -- 172.21.15.82:0/1743316969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c1a09b0 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e0b7fe700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 0x7f2e0c1a00e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400b770 tx=0x7f2e04007a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e08ff9700 1 -- 172.21.15.82:0/1743316969 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04003e10 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c1a0c40 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e08ff9700 1 -- 172.21.15.82:0/1743316969 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e04005550 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e08ff9700 1 -- 172.21.15.82:0/1743316969 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0401ba60 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.881+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c1a1090 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.882+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c02dfe0 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.883+0000 7f2e08ff9700 1 -- 172.21.15.82:0/1743316969 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2e04005020 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.884+0000 7f2e08ff9700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2df80a3c40 0x7f2df80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.884+0000 7f2e08ff9700 1 -- 172.21.15.82:0/1743316969 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f2e0400f020 con 0x7f2e0c103610 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.884+0000 7f2e0affd700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2df80a3c40 0x7f2df80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.884+0000 7f2e0affd700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2df80a3c40 0x7f2df80a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c102ad0 tx=0x7f2dfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:30:57.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:57.887+0000 7f2e08ff9700 1 -- 172.21.15.82:0/1743316969 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e0409e630 con 0x7f2e0c103610 2024-04-16T05:30:57.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:57 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi032 2024-04-16T05:30:57.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:57 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi032 to 4273M 2024-04-16T05:30:58.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:30:58.095+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2e0c062f30 con 0x7f2df80a3c40 2024-04-16T05:30:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:57 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi032 2024-04-16T05:30:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:57 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi032 to 4273M 2024-04-16T05:30:58.956 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:58 smithi082 ceph-mon[36195]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:30:58.956 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:58 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:58.956 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:58 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:58.956 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:58 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:58.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:58 smithi032 ceph-mon[32351]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:30:58.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:58.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:58.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:58 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:58 smithi134 ceph-mon[36153]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:30:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:58 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:30:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:58 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:30:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:58 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:30:59.948 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:30:59 smithi082 ceph-mon[36195]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:30:59.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:30:59 smithi032 ceph-mon[32351]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:30:59 smithi134 ceph-mon[36153]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:00.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:00 smithi032 ceph-mon[32351]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:01.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:00 smithi134 ceph-mon[36153]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:00 smithi082 ceph-mon[36195]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.82:0/1207558013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]: dispatch 2024-04-16T05:31:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:01 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]: dispatch 2024-04-16T05:31:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:01 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]': finished 2024-04-16T05:31:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:01 smithi032 ceph-mon[32351]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T05:31:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:01 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.82:0/780618294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.82:0/1207558013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]: dispatch 2024-04-16T05:31:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:01 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]: dispatch 2024-04-16T05:31:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:01 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]': finished 2024-04-16T05:31:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:01 smithi134 ceph-mon[36153]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T05:31:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:01 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.82:0/780618294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:02.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.82:0/1207558013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]: dispatch 2024-04-16T05:31:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:01 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]: dispatch 2024-04-16T05:31:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:01 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a54f6a-923a-4ce0-b6fb-f5331706fe76"}]': finished 2024-04-16T05:31:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:01 smithi082 ceph-mon[36195]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T05:31:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:01 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.82:0/780618294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:02.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:02 smithi032 ceph-mon[32351]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:02 smithi134 ceph-mon[36153]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:02 smithi082 ceph-mon[36195]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:04.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:04 smithi082 ceph-mon[36195]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:04.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:04 smithi032 ceph-mon[32351]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:05.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:04 smithi134 ceph-mon[36153]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:06.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:06 smithi082 ceph-mon[36195]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:06.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:06 smithi032 ceph-mon[32351]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:07.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:06 smithi134 ceph-mon[36153]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:08 smithi082 ceph-mon[36195]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T05:31:08.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:08.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:08 smithi032 ceph-mon[32351]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:08.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T05:31:08.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:08 smithi134 ceph-mon[36153]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T05:31:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:09.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:09 smithi032 ceph-mon[32351]: Deploying daemon osd.10 on smithi082 2024-04-16T05:31:10.087 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:09 smithi082 ceph-mon[36195]: Deploying daemon osd.10 on smithi082 2024-04-16T05:31:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:09 smithi134 ceph-mon[36153]: Deploying daemon osd.10 on smithi082 2024-04-16T05:31:10.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:10 smithi032 ceph-mon[32351]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:11.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:10 smithi134 ceph-mon[36153]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:11.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:10 smithi082 ceph-mon[36195]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:12.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:12 smithi032 ceph-mon[32351]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:13.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:12 smithi134 ceph-mon[36153]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:13.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:12 smithi082 ceph-mon[36195]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:14.895 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:14 smithi082 ceph-mon[36195]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:14.896 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:14.896 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:14.896 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:14.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:14 smithi032 ceph-mon[32351]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:14.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:14.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:14 smithi134 ceph-mon[36153]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:16.839 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:16 smithi082 ceph-mon[36195]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:16.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:16 smithi032 ceph-mon[32351]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:17.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:16 smithi134 ceph-mon[36153]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:17.667 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 10 on host 'smithi082' 2024-04-16T05:31:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.668+0000 7f2e08ff9700 1 -- 172.21.15.82:0/1743316969 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2e0c062f30 con 0x7f2df80a3c40 2024-04-16T05:31:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2df80a3c40 msgr2=0x7f2df80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2df80a3c40 0x7f2df80a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c102ad0 tx=0x7f2dfc009260 comp rx=0 tx=0).stop 2024-04-16T05:31:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 msgr2=0x7f2e0c1a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 0x7f2e0c1a00e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400b770 tx=0x7f2e04007a50 comp rx=0 tx=0).stop 2024-04-16T05:31:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 shutdown_connections 2024-04-16T05:31:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2df80a3c40 0x7f2df80a6100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c103610 0x7f2e0c1a00e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0c101a70 0x7f2e0c06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 --2- 172.21.15.82:0/1743316969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0c102c70 0x7f2e0c19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 >> 172.21.15.82:0/1743316969 conn(0x7f2e0c0fd050 msgr2=0x7f2e0c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 shutdown_connections 2024-04-16T05:31:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-16T05:31:17.670+0000 7f2e11507700 1 -- 172.21.15.82:0/1743316969 wait complete. 2024-04-16T05:31:18.337 DEBUG:teuthology.orchestra.run.smithi082:osd.10> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.10.service 2024-04-16T05:31:18.339 INFO:tasks.cephadm:Deploying osd.11 on smithi134 with /dev/vg_nvme/lv_1... 2024-04-16T05:31:18.339 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T05:31:18.586 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:31:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:18 smithi134 ceph-mon[36153]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:18 smithi134 ceph-mon[36153]: from='osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:18 smithi134 ceph-mon[36153]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:18 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:18 smithi082 ceph-mon[36195]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:18 smithi082 ceph-mon[36195]: from='osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:18 smithi082 ceph-mon[36195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:18 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:18 smithi032 ceph-mon[32351]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:18.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:18 smithi032 ceph-mon[32351]: from='osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:18.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:18 smithi032 ceph-mon[32351]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T05:31:18.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:18.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:18 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:19.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:19 smithi134 ceph-mon[36153]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T05:31:19.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:19 smithi134 ceph-mon[36153]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T05:31:19.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:19 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:19.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:19 smithi134 ceph-mon[36153]: from='osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:31:19.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:19 smithi134 ceph-mon[36153]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:31:19.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:19 smithi032 ceph-mon[32351]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T05:31:19.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:19 smithi032 ceph-mon[32351]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T05:31:19.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:19 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:19.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:19 smithi032 ceph-mon[32351]: from='osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:31:19.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:19 smithi032 ceph-mon[32351]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:31:20.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:19 smithi082 ceph-mon[36195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T05:31:20.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:19 smithi082 ceph-mon[36195]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T05:31:20.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:19 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:20.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:19 smithi082 ceph-mon[36195]: from='osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:31:20.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:19 smithi082 ceph-mon[36195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-16T05:31:20.151 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:31:19 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:31:19.725+0000 7fc76cb9d700 -1 osd.10 0 waiting for initial osdmap 2024-04-16T05:31:20.151 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:31:19 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:31:19.737+0000 7fc768990700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:31:20.738 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T05:31:20.738 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T05:31:20.739 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-16T05:31:20.739 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-16T05:31:20.739 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116183 s, 903 MB/s 2024-04-16T05:31:20.739 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-16T05:31:20.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:20 smithi134 ceph-mon[36153]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:20.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:20 smithi134 ceph-mon[36153]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:31:20.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:20 smithi134 ceph-mon[36153]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T05:31:20.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:20.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:20.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:20 smithi082 ceph-mon[36195]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:20.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:20 smithi082 ceph-mon[36195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:31:20.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:20 smithi082 ceph-mon[36195]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T05:31:20.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:20.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:20.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:20 smithi032 ceph-mon[32351]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:20.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:20 smithi032 ceph-mon[32351]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-16T05:31:20.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:20 smithi032 ceph-mon[32351]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T05:31:20.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:20.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:21.498 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-04-16T05:31:21.735 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:31:21.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:21 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:31:21.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:21 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:31:21.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:21 smithi032 ceph-mon[32351]: osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] boot 2024-04-16T05:31:21.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:21 smithi032 ceph-mon[32351]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T05:31:21.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:21 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:22.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:21 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:31:22.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:21 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:31:22.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:21 smithi082 ceph-mon[36195]: osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] boot 2024-04-16T05:31:22.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:21 smithi082 ceph-mon[36195]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T05:31:22.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:21 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:22.015 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:21 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:31:22.015 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:21 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:31:22.015 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:21 smithi134 ceph-mon[36153]: osd.10 [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] boot 2024-04-16T05:31:22.015 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:21 smithi134 ceph-mon[36153]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T05:31:22.015 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:21 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:31:22.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:22.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:23.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:23.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T05:31:23.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:23.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:23.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:23.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:23.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:23.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:23.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:23.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:23.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:23.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:23.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:23.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:23.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 -- 172.21.15.134:0/2606886828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70101200 msgr2=0x7f8e70101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 --2- 172.21.15.134:0/2606886828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70101200 0x7f8e70101620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009990 tx=0x7f8e6c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 -- 172.21.15.134:0/2606886828 shutdown_connections 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 --2- 172.21.15.134:0/2606886828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e70102da0 0x7f8e70109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 --2- 172.21.15.134:0/2606886828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70101200 0x7f8e70101620 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 --2- 172.21.15.134:0/2606886828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e70102400 0x7f8e70102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 -- 172.21.15.134:0/2606886828 >> 172.21.15.134:0/2606886828 conn(0x7f8e700fc7a0 msgr2=0x7f8e700febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 -- 172.21.15.134:0/2606886828 shutdown_connections 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 -- 172.21.15.134:0/2606886828 wait complete. 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.459+0000 7f8e778f7700 1 Processor -- start 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e778f7700 1 -- start start 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e778f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e70101200 0x7f8e70105140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e778f7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70102400 0x7f8e701037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:23.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e778f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e70102da0 0x7f8e70103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e778f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010cec0 con 0x7f8e70102da0 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e778f7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010cd40 con 0x7f8e70101200 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e778f7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010d040 con 0x7f8e70102400 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70102400 0x7f8e701037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70102400 0x7f8e701037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42546/0 (socket says 172.21.15.134:42546) 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 -- 172.21.15.134:0/135927632 learned_addr learned my addr 172.21.15.134:0/135927632 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e75693700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e70101200 0x7f8e70105140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 -- 172.21.15.134:0/135927632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e70101200 msgr2=0x7f8e70105140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e70101200 0x7f8e70105140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 -- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e70102da0 msgr2=0x7f8e70103d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e75e94700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e70102da0 0x7f8e70103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e70102da0 0x7f8e70103d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 -- 172.21.15.134:0/135927632 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e701045f0 con 0x7f8e70102400 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e74e92700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70102400 0x7f8e701037e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000b530 tx=0x7f8e6000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:23.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e667fc700 1 -- 172.21.15.134:0/135927632 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60015020 con 0x7f8e70102400 2024-04-16T05:31:23.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e701a3060 con 0x7f8e70102400 2024-04-16T05:31:23.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.460+0000 7f8e667fc700 1 -- 172.21.15.134:0/135927632 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e60011d10 con 0x7f8e70102400 2024-04-16T05:31:23.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.461+0000 7f8e667fc700 1 -- 172.21.15.134:0/135927632 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60010e50 con 0x7f8e70102400 2024-04-16T05:31:23.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.461+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e701a3640 con 0x7f8e70102400 2024-04-16T05:31:23.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.461+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e700697e0 con 0x7f8e70102400 2024-04-16T05:31:23.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.463+0000 7f8e667fc700 1 -- 172.21.15.134:0/135927632 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e60018020 con 0x7f8e70102400 2024-04-16T05:31:23.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.463+0000 7f8e667fc700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8e5c0a3de0 0x7f8e5c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:23.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.464+0000 7f8e667fc700 1 -- 172.21.15.134:0/135927632 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f8e600cfc80 con 0x7f8e70102400 2024-04-16T05:31:23.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.464+0000 7f8e75693700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8e5c0a3de0 0x7f8e5c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:23.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.464+0000 7f8e75693700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8e5c0a3de0 0x7f8e5c0a62a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0039c0 tx=0x7f8e6c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:23.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.467+0000 7f8e667fc700 1 -- 172.21.15.134:0/135927632 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e6001f020 con 0x7f8e70102400 2024-04-16T05:31:23.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:23.675+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8e701a3960 con 0x7f8e5c0a3de0 2024-04-16T05:31:24.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi082 2024-04-16T05:31:24.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi082 to 4270M 2024-04-16T05:31:24.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T05:31:24.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:24.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:24.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:24.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:24.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:24.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:23 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi082 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi082 to 4270M 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:23 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:24.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi082 2024-04-16T05:31:24.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi082 to 4270M 2024-04-16T05:31:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T05:31:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T05:31:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T05:31:24.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:24.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:24 smithi134 ceph-mon[36153]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:24.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:24 smithi134 ceph-mon[36153]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:24.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:24 smithi134 ceph-mon[36153]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T05:31:25.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:24 smithi082 ceph-mon[36195]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:25.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:24 smithi082 ceph-mon[36195]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:25.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:24 smithi082 ceph-mon[36195]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T05:31:25.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:24 smithi032 ceph-mon[32351]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:25.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:24 smithi032 ceph-mon[32351]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:25.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:24 smithi032 ceph-mon[32351]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T05:31:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:25 smithi134 ceph-mon[36153]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:25 smithi134 ceph-mon[36153]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T05:31:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:25 smithi032 ceph-mon[32351]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:25 smithi032 ceph-mon[32351]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T05:31:26.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:25 smithi082 ceph-mon[36195]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:26.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:25 smithi082 ceph-mon[36195]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T05:31:27.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:26 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/4235847132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]: dispatch 2024-04-16T05:31:27.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:26 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]: dispatch 2024-04-16T05:31:27.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:26 smithi032 ceph-mon[32351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]': finished 2024-04-16T05:31:27.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:26 smithi032 ceph-mon[32351]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T05:31:27.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:26 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:27.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:26 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/4235847132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]: dispatch 2024-04-16T05:31:27.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:26 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]: dispatch 2024-04-16T05:31:27.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:26 smithi134 ceph-mon[36153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]': finished 2024-04-16T05:31:27.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:26 smithi134 ceph-mon[36153]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T05:31:27.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:26 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:26 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/4235847132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]: dispatch 2024-04-16T05:31:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:26 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]: dispatch 2024-04-16T05:31:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:26 smithi082 ceph-mon[36195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67e3e8a5-0d88-445e-b37a-d7cd3db91a06"}]': finished 2024-04-16T05:31:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:26 smithi082 ceph-mon[36195]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T05:31:27.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:26 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:28.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:27 smithi032 ceph-mon[32351]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-16T05:31:28.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:27 smithi032 ceph-mon[32351]: from='client.? 172.21.15.134:0/563852409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:28.314 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:27 smithi082 ceph-mon[36195]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-16T05:31:28.314 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:27 smithi082 ceph-mon[36195]: from='client.? 172.21.15.134:0/563852409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:28.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:27 smithi134 ceph-mon[36153]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-16T05:31:28.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:27 smithi134 ceph-mon[36153]: from='client.? 172.21.15.134:0/563852409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T05:31:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:29 smithi032 ceph-mon[32351]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:31:30.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:29 smithi134 ceph-mon[36153]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:31:30.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:29 smithi082 ceph-mon[36195]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T05:31:32.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:31 smithi134 ceph-mon[36153]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-16T05:31:32.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:31 smithi032 ceph-mon[32351]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-16T05:31:32.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:31 smithi082 ceph-mon[36195]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-16T05:31:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:33 smithi134 ceph-mon[36153]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:31:34.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:33 smithi032 ceph-mon[32351]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:31:34.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:33 smithi082 ceph-mon[36195]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T05:31:35.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T05:31:35.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:35.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:34 smithi032 ceph-mon[32351]: Deploying daemon osd.11 on smithi134 2024-04-16T05:31:35.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T05:31:35.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:35.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:34 smithi134 ceph-mon[36153]: Deploying daemon osd.11 on smithi134 2024-04-16T05:31:35.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T05:31:35.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:35.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:34 smithi082 ceph-mon[36195]: Deploying daemon osd.11 on smithi134 2024-04-16T05:31:36.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:35 smithi134 ceph-mon[36153]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T05:31:36.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:35 smithi032 ceph-mon[32351]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T05:31:36.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:35 smithi082 ceph-mon[36195]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T05:31:38.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:37 smithi134 ceph-mon[36153]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-16T05:31:38.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:37 smithi032 ceph-mon[32351]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-16T05:31:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:37 smithi082 ceph-mon[36195]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-16T05:31:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:39 smithi032 ceph-mon[32351]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:39 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:39 smithi134 ceph-mon[36153]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:39 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:39 smithi082 ceph-mon[36195]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:39 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:42.214 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:41 smithi134 ceph-mon[36153]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:41 smithi032 ceph-mon[32351]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:42.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:41 smithi082 ceph-mon[36195]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:44.043 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 11 on host 'smithi134' 2024-04-16T05:31:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.044+0000 7f8e667fc700 1 -- 172.21.15.134:0/135927632 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8e701a3960 con 0x7f8e5c0a3de0 2024-04-16T05:31:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8e5c0a3de0 msgr2=0x7f8e5c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8e5c0a3de0 0x7f8e5c0a62a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0039c0 tx=0x7f8e6c005fb0 comp rx=0 tx=0).stop 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70102400 msgr2=0x7f8e701037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70102400 0x7f8e701037e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000b530 tx=0x7f8e6000b8f0 comp rx=0 tx=0).stop 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 shutdown_connections 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8e5c0a3de0 0x7f8e5c0a62a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e70102da0 0x7f8e70103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e70101200 0x7f8e70105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 --2- 172.21.15.134:0/135927632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e70102400 0x7f8e701037e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.046+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 >> 172.21.15.134:0/135927632 conn(0x7f8e700fc7a0 msgr2=0x7f8e700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.047+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 shutdown_connections 2024-04-16T05:31:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:44.047+0000 7f8e778f7700 1 -- 172.21.15.134:0/135927632 wait complete. 2024-04-16T05:31:44.369 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:44.369 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.369 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.370 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:44.370 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:44.370 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.370 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:31:44.370 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:31:44.370 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.370 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.370 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:44 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:44 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:44.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:44 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:44.690 DEBUG:teuthology.orchestra.run.smithi134:osd.11> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.11.service 2024-04-16T05:31:44.692 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-16T05:31:44.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd stat -f json 2024-04-16T05:31:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:31:45.164 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:45 smithi134 ceph-mon[36153]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T05:31:45.164 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:45 smithi134 ceph-mon[36153]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T05:31:45.165 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:45 smithi134 ceph-mon[36153]: from='osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:31:45.165 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:45 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:45.165 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:45 smithi134 ceph-mon[36153]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:31:45.165 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:31:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:31:45.148+0000 7f5210937700 -1 osd.11 0 waiting for initial osdmap 2024-04-16T05:31:45.165 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:31:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:31:45.165+0000 7f520c72a700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T05:31:45.326 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:45 smithi032 ceph-mon[32351]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T05:31:45.326 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:45 smithi032 ceph-mon[32351]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T05:31:45.326 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:45 smithi032 ceph-mon[32351]: from='osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:31:45.326 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:45 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:45.326 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:45 smithi032 ceph-mon[32351]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:31:45.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:45 smithi082 ceph-mon[36195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T05:31:45.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:45 smithi082 ceph-mon[36195]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T05:31:45.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:45 smithi082 ceph-mon[36195]: from='osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:31:45.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:45 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:45.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:45 smithi082 ceph-mon[36195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-16T05:31:46.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:46 smithi134 ceph-mon[36153]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:46.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:46 smithi134 ceph-mon[36153]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:31:46.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:46 smithi134 ceph-mon[36153]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T05:31:46.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:46 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:46.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:46 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:46.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:46 smithi082 ceph-mon[36195]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:46.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:46 smithi082 ceph-mon[36195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:31:46.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:46 smithi082 ceph-mon[36195]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T05:31:46.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:46 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:46.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:46 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:46.437 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:46 smithi032 ceph-mon[32351]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:46.438 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:46 smithi032 ceph-mon[32351]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-16T05:31:46.438 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:46 smithi032 ceph-mon[32351]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T05:31:46.438 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:46 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:46.438 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:46 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 -- 172.21.15.32:0/3760817849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89901035e0 msgr2=0x7f8990109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 --2- 172.21.15.32:0/3760817849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89901035e0 0x7f8990109e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f898800b0d0 tx=0x7f898800b3e0 comp rx=0 tx=0).stop 2024-04-16T05:31:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 -- 172.21.15.32:0/3760817849 shutdown_connections 2024-04-16T05:31:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 --2- 172.21.15.32:0/3760817849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89901035e0 0x7f8990109e90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 --2- 172.21.15.32:0/3760817849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8990101a40 0x7f8990101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 --2- 172.21.15.32:0/3760817849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8990102c40 0x7f89901030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 -- 172.21.15.32:0/3760817849 >> 172.21.15.32:0/3760817849 conn(0x7f89900fcfe0 msgr2=0x7f89900ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 -- 172.21.15.32:0/3760817849 shutdown_connections 2024-04-16T05:31:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.787+0000 7f8996066700 1 -- 172.21.15.32:0/3760817849 wait complete. 2024-04-16T05:31:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.788+0000 7f8996066700 1 Processor -- start 2024-04-16T05:31:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.788+0000 7f8996066700 1 -- start start 2024-04-16T05:31:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.788+0000 7f8996066700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8990101a40 0x7f899006bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.788+0000 7f8996066700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8990102c40 0x7f899019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.788+0000 7f8996066700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89901035e0 0x7f89901a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.788+0000 7f8996066700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f899010d610 con 0x7f8990102c40 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.788+0000 7f8996066700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f899010d490 con 0x7f8990101a40 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.788+0000 7f8996066700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f899010d790 con 0x7f89901035e0 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8990102c40 0x7f899019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8990102c40 0x7f899019da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34996/0 (socket says 172.21.15.32:34996) 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 -- 172.21.15.32:0/4212647740 learned_addr learned my addr 172.21.15.32:0/4212647740 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f898ffff700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89901035e0 0x7f89901a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f898f7fe700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8990101a40 0x7f899006bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 -- 172.21.15.32:0/4212647740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89901035e0 msgr2=0x7f89901a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89901035e0 0x7f89901a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 -- 172.21.15.32:0/4212647740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8990101a40 msgr2=0x7f899006bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:46.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8990101a40 0x7f899006bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 -- 172.21.15.32:0/4212647740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89901a0910 con 0x7f8990102c40 2024-04-16T05:31:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8986ffd700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8990102c40 0x7f899019da00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f897800d950 tx=0x7f897800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f898d7fa700 1 -- 172.21.15.32:0/4212647740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978016070 con 0x7f8990102c40 2024-04-16T05:31:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89901a0c00 con 0x7f8990102c40 2024-04-16T05:31:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f898d7fa700 1 -- 172.21.15.32:0/4212647740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897800fd70 con 0x7f8990102c40 2024-04-16T05:31:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f898d7fa700 1 -- 172.21.15.32:0/4212647740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897800e610 con 0x7f8990102c40 2024-04-16T05:31:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.789+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901a1140 con 0x7f8990102c40 2024-04-16T05:31:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.790+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89900697e0 con 0x7f8990102c40 2024-04-16T05:31:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.792+0000 7f898d7fa700 1 -- 172.21.15.32:0/4212647740 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8978019070 con 0x7f8990102c40 2024-04-16T05:31:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.792+0000 7f898d7fa700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f897c0a3c40 0x7f897c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.792+0000 7f898f7fe700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f897c0a3c40 0x7f897c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.792+0000 7f898d7fa700 1 -- 172.21.15.32:0/4212647740 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f89780cfa10 con 0x7f8990102c40 2024-04-16T05:31:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.793+0000 7f898f7fe700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f897c0a3c40 0x7f897c0a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8990102aa0 tx=0x7f898000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:46.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:46.796+0000 7f898d7fa700 1 -- 172.21.15.32:0/4212647740 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f897809c890 con 0x7f8990102c40 2024-04-16T05:31:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.007+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8990063700 con 0x7f8990102c40 2024-04-16T05:31:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.007+0000 7f898d7fa700 1 -- 172.21.15.32:0/4212647740 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f897809c190 con 0x7f8990102c40 2024-04-16T05:31:47.006 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:31:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f897c0a3c40 msgr2=0x7f897c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f897c0a3c40 0x7f897c0a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8990102aa0 tx=0x7f898000b410 comp rx=0 tx=0).stop 2024-04-16T05:31:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8990102c40 msgr2=0x7f899019da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8990102c40 0x7f899019da00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f897800d950 tx=0x7f897800dc60 comp rx=0 tx=0).stop 2024-04-16T05:31:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 shutdown_connections 2024-04-16T05:31:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f897c0a3c40 0x7f897c0a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89901035e0 0x7f89901a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8990101a40 0x7f899006bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 --2- 172.21.15.32:0/4212647740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8990102c40 0x7f899019da00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 >> 172.21.15.32:0/4212647740 conn(0x7f89900fcfe0 msgr2=0x7f8990105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 shutdown_connections 2024-04-16T05:31:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:47.009+0000 7f8996066700 1 -- 172.21.15.32:0/4212647740 wait complete. 2024-04-16T05:31:47.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:47 smithi032 ceph-mon[32351]: purged_snaps scrub starts 2024-04-16T05:31:47.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:47 smithi032 ceph-mon[32351]: purged_snaps scrub ok 2024-04-16T05:31:47.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:47 smithi032 ceph-mon[32351]: osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] boot 2024-04-16T05:31:47.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:47 smithi032 ceph-mon[32351]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T05:31:47.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:47 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:47.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:47 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4212647740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T05:31:47.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:47 smithi134 ceph-mon[36153]: purged_snaps scrub starts 2024-04-16T05:31:47.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:47 smithi134 ceph-mon[36153]: purged_snaps scrub ok 2024-04-16T05:31:47.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:47 smithi134 ceph-mon[36153]: osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] boot 2024-04-16T05:31:47.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:47 smithi134 ceph-mon[36153]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T05:31:47.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:47 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:47.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:47 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4212647740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T05:31:47.578 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:47 smithi082 ceph-mon[36195]: purged_snaps scrub starts 2024-04-16T05:31:47.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:47 smithi082 ceph-mon[36195]: purged_snaps scrub ok 2024-04-16T05:31:47.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:47 smithi082 ceph-mon[36195]: osd.11 [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] boot 2024-04-16T05:31:47.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:47 smithi082 ceph-mon[36195]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T05:31:47.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:47 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:31:47.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:47 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4212647740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T05:31:47.799 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1713245506,"num_in_osds":12,"osd_in_since":1713245486,"num_remapped_pgs":0} 2024-04-16T05:31:47.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd dump --format=json 2024-04-16T05:31:48.058 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:31:48.434 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:48.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:48 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.522 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:48.522 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T05:31:48.522 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.522 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.522 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.522 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.523 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.523 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.523 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.523 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:48.523 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:48.523 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:48 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:48.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:48 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:49.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:49 smithi032 ceph-mon[32351]: Detected new or changed devices on smithi134 2024-04-16T05:31:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:49 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi134 to 4270M 2024-04-16T05:31:49.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:49 smithi134 ceph-mon[36153]: Detected new or changed devices on smithi134 2024-04-16T05:31:49.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:49 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi134 to 4270M 2024-04-16T05:31:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:49 smithi082 ceph-mon[36195]: Detected new or changed devices on smithi134 2024-04-16T05:31:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:49 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi134 to 4270M 2024-04-16T05:31:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 -- 172.21.15.32:0/629604346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 msgr2=0x7f56e8101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 --2- 172.21.15.32:0/629604346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 0x7f56e8101e70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00b0a0 tx=0x7f56dc00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 -- 172.21.15.32:0/629604346 shutdown_connections 2024-04-16T05:31:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 --2- 172.21.15.32:0/629604346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56e81023b0 0x7f56e810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 --2- 172.21.15.32:0/629604346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e8107a40 0x7f56e8107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 --2- 172.21.15.32:0/629604346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 0x7f56e8101e70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 -- 172.21.15.32:0/629604346 >> 172.21.15.32:0/629604346 conn(0x7f56e80789a0 msgr2=0x7f56e8078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 -- 172.21.15.32:0/629604346 shutdown_connections 2024-04-16T05:31:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.698+0000 7f56eebc1700 1 -- 172.21.15.32:0/629604346 wait complete. 2024-04-16T05:31:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56eebc1700 1 Processor -- start 2024-04-16T05:31:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56eebc1700 1 -- start start 2024-04-16T05:31:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56eebc1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 0x7f56e819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56eebc1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81023b0 0x7f56e819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56eebc1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56e8107a40 0x7f56e819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56eebc1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8111cf0 con 0x7f56e81019f0 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56eebc1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8111b70 con 0x7f56e81023b0 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56eebc1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8111e70 con 0x7f56e8107a40 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 0x7f56e819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 0x7f56e819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58992/0 (socket says 172.21.15.32:58992) 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 -- 172.21.15.32:0/2059689920 learned_addr learned my addr 172.21.15.32:0/2059689920 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ed15e700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56e8107a40 0x7f56e819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56e7fff700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81023b0 0x7f56e819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 -- 172.21.15.32:0/2059689920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56e8107a40 msgr2=0x7f56e819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56e8107a40 0x7f56e819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 -- 172.21.15.32:0/2059689920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81023b0 msgr2=0x7f56e819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81023b0 0x7f56e819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 -- 172.21.15.32:0/2059689920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e81a0460 con 0x7f56e81019f0 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.699+0000 7f56ec95d700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 0x7f56e819b450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c920 tx=0x7f56d800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.700+0000 7f56e5ffb700 1 -- 172.21.15.32:0/2059689920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d8007b00 con 0x7f56e81019f0 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.700+0000 7f56e5ffb700 1 -- 172.21.15.32:0/2059689920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56d8004d10 con 0x7f56e81019f0 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.700+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e81a0750 con 0x7f56e81019f0 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.700+0000 7f56e5ffb700 1 -- 172.21.15.32:0/2059689920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d800f450 con 0x7f56e81019f0 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.700+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e81a7fd0 con 0x7f56e81019f0 2024-04-16T05:31:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.701+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e80697e0 con 0x7f56e81019f0 2024-04-16T05:31:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.702+0000 7f56e5ffb700 1 -- 172.21.15.32:0/2059689920 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f56d8004750 con 0x7f56e81019f0 2024-04-16T05:31:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.703+0000 7f56e5ffb700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f56d00a3c40 0x7f56d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.703+0000 7f56e7fff700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f56d00a3c40 0x7f56d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.703+0000 7f56e5ffb700 1 -- 172.21.15.32:0/2059689920 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f56d80cff90 con 0x7f56e81019f0 2024-04-16T05:31:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.703+0000 7f56e7fff700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f56d00a3c40 0x7f56d00a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00b070 tx=0x7f56dc006080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.707+0000 7f56e5ffb700 1 -- 172.21.15.32:0/2059689920 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56d8014030 con 0x7f56e81019f0 2024-04-16T05:31:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.919+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56e802dfe0 con 0x7f56e81019f0 2024-04-16T05:31:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.920+0000 7f56e5ffb700 1 -- 172.21.15.32:0/2059689920 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18420 (secure 0 0 0) 0x7f56d809cd70 con 0x7f56e81019f0 2024-04-16T05:31:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:31:49.919 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:31:47.151909+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:31:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f56d00a3c40 msgr2=0x7f56d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f56d00a3c40 0x7f56d00a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00b070 tx=0x7f56dc006080 comp rx=0 tx=0).stop 2024-04-16T05:31:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 msgr2=0x7f56e819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 0x7f56e819b450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c920 tx=0x7f56d800cce0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 shutdown_connections 2024-04-16T05:31:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f56d00a3c40 0x7f56d00a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56e8107a40 0x7f56e819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e81019f0 0x7f56e819b450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 --2- 172.21.15.32:0/2059689920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81023b0 0x7f56e819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 >> 172.21.15.32:0/2059689920 conn(0x7f56e80789a0 msgr2=0x7f56e80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 shutdown_connections 2024-04-16T05:31:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:49.922+0000 7f56eebc1700 1 -- 172.21.15.32:0/2059689920 wait complete. 2024-04-16T05:31:50.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:50 smithi032 ceph-mon[32351]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:50.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:50 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2059689920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:31:50.628 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T05:31:50.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-16T05:31:50.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:50 smithi134 ceph-mon[36153]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:50.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:50 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2059689920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:31:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:50 smithi082 ceph-mon[36195]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:50 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2059689920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:31:50.878 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:31:52.326 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:52 smithi032 ceph-mon[32351]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 -- 172.21.15.32:0/1619461941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b90107a30 msgr2=0x7f0b90107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 --2- 172.21.15.32:0/1619461941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b90107a30 0x7f0b90107e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009990 tx=0x7f0b8c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 -- 172.21.15.32:0/1619461941 shutdown_connections 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 --2- 172.21.15.32:0/1619461941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b901023a0 0x7f0b9010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 --2- 172.21.15.32:0/1619461941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b90107a30 0x7f0b90107e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 --2- 172.21.15.32:0/1619461941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b901019e0 0x7f0b90101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 -- 172.21.15.32:0/1619461941 >> 172.21.15.32:0/1619461941 conn(0x7f0b90078960 msgr2=0x7f0b90078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 -- 172.21.15.32:0/1619461941 shutdown_connections 2024-04-16T05:31:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.579+0000 7f0b976f4700 1 -- 172.21.15.32:0/1619461941 wait complete. 2024-04-16T05:31:52.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b976f4700 1 Processor -- start 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b976f4700 1 -- start start 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b976f4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b901019e0 0x7f0b90198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b976f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b901023a0 0x7f0b90198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b976f4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b90107a30 0x7f0b9019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b976f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90111d40 con 0x7f0b901023a0 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b976f4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90111bc0 con 0x7f0b90107a30 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b976f4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90111ec0 con 0x7f0b901019e0 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.580+0000 7f0b94c8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b901023a0 0x7f0b90198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b94c8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b901023a0 0x7f0b90198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59010/0 (socket says 172.21.15.32:59010) 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b94c8f700 1 -- 172.21.15.32:0/2022882493 learned_addr learned my addr 172.21.15.32:0/2022882493 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b95490700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b901019e0 0x7f0b90198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b95c91700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b90107a30 0x7f0b9019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b94c8f700 1 -- 172.21.15.32:0/2022882493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b901019e0 msgr2=0x7f0b90198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b94c8f700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b901019e0 0x7f0b90198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b94c8f700 1 -- 172.21.15.32:0/2022882493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b90107a30 msgr2=0x7f0b9019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b94c8f700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b90107a30 0x7f0b9019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b94c8f700 1 -- 172.21.15.32:0/2022882493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9019d220 con 0x7f0b901023a0 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b95490700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b901019e0 0x7f0b90198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b94c8f700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b901023a0 0x7f0b90198750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000b530 tx=0x7f0b8000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b867fc700 1 -- 172.21.15.32:0/2022882493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80015020 con 0x7f0b901023a0 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b867fc700 1 -- 172.21.15.32:0/2022882493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b80011d10 con 0x7f0b901023a0 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9019d510 con 0x7f0b901023a0 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b867fc700 1 -- 172.21.15.32:0/2022882493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000f2e0 con 0x7f0b901023a0 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.581+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b901a4d90 con 0x7f0b901023a0 2024-04-16T05:31:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.582+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b90109f50 con 0x7f0b901023a0 2024-04-16T05:31:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.584+0000 7f0b867fc700 1 -- 172.21.15.32:0/2022882493 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0b80018020 con 0x7f0b901023a0 2024-04-16T05:31:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.584+0000 7f0b867fc700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b7c0a3c40 0x7f0b7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.584+0000 7f0b95490700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b7c0a3c40 0x7f0b7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.584+0000 7f0b867fc700 1 -- 172.21.15.32:0/2022882493 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0b800cfbf0 con 0x7f0b901023a0 2024-04-16T05:31:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.585+0000 7f0b95490700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b7c0a3c40 0x7f0b7c0a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c006010 tx=0x7f0b8c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.588+0000 7f0b867fc700 1 -- 172.21.15.32:0/2022882493 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b8001f020 con 0x7f0b901023a0 2024-04-16T05:31:52.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:52 smithi134 ceph-mon[36153]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:52.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:52 smithi082 ceph-mon[36195]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.793+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b90199570 con 0x7f0b901023a0 2024-04-16T05:31:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.793+0000 7f0b867fc700 1 -- 172.21.15.32:0/2022882493 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f0b8009ca70 con 0x7f0b901023a0 2024-04-16T05:31:52.792 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:31:52.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b7c0a3c40 msgr2=0x7f0b7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b7c0a3c40 0x7f0b7c0a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c006010 tx=0x7f0b8c00b540 comp rx=0 tx=0).stop 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b901023a0 msgr2=0x7f0b90198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b901023a0 0x7f0b90198750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000b530 tx=0x7f0b8000b840 comp rx=0 tx=0).stop 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 shutdown_connections 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b7c0a3c40 0x7f0b7c0a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b90107a30 0x7f0b9019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b901019e0 0x7f0b90198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 --2- 172.21.15.32:0/2022882493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b901023a0 0x7f0b90198750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.795+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 >> 172.21.15.32:0/2022882493 conn(0x7f0b90078960 msgr2=0x7f0b901007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.796+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 shutdown_connections 2024-04-16T05:31:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:52.796+0000 7f0b976f4700 1 -- 172.21.15.32:0/2022882493 wait complete. 2024-04-16T05:31:53.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:53 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2022882493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:31:53.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:53 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2022882493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.a on smithi032 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.d on smithi032 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.g on smithi032 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.j on smithi032 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.b on smithi082 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.e on smithi082 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.h on smithi082 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.k on smithi082 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.c on smithi134 2024-04-16T05:31:53.646 INFO:tasks.cephadm:Adding mds.f on smithi134 2024-04-16T05:31:53.647 INFO:tasks.cephadm:Adding mds.i on smithi134 2024-04-16T05:31:53.647 INFO:tasks.cephadm:Adding mds.l on smithi134 2024-04-16T05:31:53.647 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph orch apply mds all '12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l' 2024-04-16T05:31:53.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:53 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2022882493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:31:53.895 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:31:54.307 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:54 smithi134 ceph-mon[36153]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:54.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:54 smithi032 ceph-mon[32351]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:54.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:54 smithi082 ceph-mon[36195]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 -- 172.21.15.134:0/702221448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 msgr2=0x7f6c7c102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 --2- 172.21.15.134:0/702221448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 0x7f6c7c102f90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009990 tx=0x7f6c6c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 -- 172.21.15.134:0/702221448 shutdown_connections 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 --2- 172.21.15.134:0/702221448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c104710 0x7f6c7c10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 --2- 172.21.15.134:0/702221448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 0x7f6c7c102f90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 --2- 172.21.15.134:0/702221448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c7c103d70 0x7f6c7c1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 -- 172.21.15.134:0/702221448 >> 172.21.15.134:0/702221448 conn(0x7f6c7c0fe110 msgr2=0x7f6c7c100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 -- 172.21.15.134:0/702221448 shutdown_connections 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.519+0000 7f6c80fea700 1 -- 172.21.15.134:0/702221448 wait complete. 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c80fea700 1 Processor -- start 2024-04-16T05:31:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c80fea700 1 -- start start 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c80fea700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 0x7f6c7c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c80fea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c7c103d70 0x7f6c7c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c80fea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c104710 0x7f6c7c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c80fea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10e750 con 0x7f6c7c104710 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c80fea700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10e5d0 con 0x7f6c7c103d70 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c80fea700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10e8d0 con 0x7f6c7c102b70 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 0x7f6c7c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 0x7f6c7c198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:55954/0 (socket says 172.21.15.134:55954) 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 -- 172.21.15.134:0/2958388256 learned_addr learned my addr 172.21.15.134:0/2958388256 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7ad9d700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c104710 0x7f6c7c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 -- 172.21.15.134:0/2958388256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c7c103d70 msgr2=0x7f6c7c199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c79d9b700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c7c103d70 0x7f6c7c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c7c103d70 0x7f6c7c199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 -- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c104710 msgr2=0x7f6c7c1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c104710 0x7f6c7c1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 -- 172.21.15.134:0/2958388256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c1a09f0 con 0x7f6c7c102b70 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7ad9d700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c104710 0x7f6c7c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c79d9b700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c7c103d70 0x7f6c7c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.520+0000 7f6c7a59c700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 0x7f6c7c198e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0039c0 tx=0x7f6c6c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.521+0000 7f6c6b7fe700 1 -- 172.21.15.134:0/2958388256 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c003d30 con 0x7f6c7c102b70 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.521+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c1a0c80 con 0x7f6c7c102b70 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.521+0000 7f6c6b7fe700 1 -- 172.21.15.134:0/2958388256 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c004830 con 0x7f6c7c102b70 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.521+0000 7f6c6b7fe700 1 -- 172.21.15.134:0/2958388256 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c005480 con 0x7f6c7c102b70 2024-04-16T05:31:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.521+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c1a10d0 con 0x7f6c7c102b70 2024-04-16T05:31:55.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.522+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c0697e0 con 0x7f6c7c102b70 2024-04-16T05:31:55.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.523+0000 7f6c6b7fe700 1 -- 172.21.15.134:0/2958388256 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c6c003ed0 con 0x7f6c7c102b70 2024-04-16T05:31:55.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.523+0000 7f6c6b7fe700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c640a3c40 0x7f6c640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:55.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.523+0000 7f6c6b7fe700 1 -- 172.21.15.134:0/2958388256 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6c6c0d0fd0 con 0x7f6c7c102b70 2024-04-16T05:31:55.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.523+0000 7f6c79d9b700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c640a3c40 0x7f6c640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:55.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.524+0000 7f6c79d9b700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c640a3c40 0x7f6c640a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005fd0 tx=0x7f6c70005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.528+0000 7f6c6b7fe700 1 -- 172.21.15.134:0/2958388256 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c6c013070 con 0x7f6c7c102b70 2024-04-16T05:31:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.735+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}) -- 0x7f6c7c0631f0 con 0x7f6c640a3c40 2024-04-16T05:31:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.743+0000 7f6c6b7fe700 1 -- 172.21.15.134:0/2958388256 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6c7c0631f0 con 0x7f6c640a3c40 2024-04-16T05:31:55.742 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mds.all update... 2024-04-16T05:31:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c640a3c40 msgr2=0x7f6c640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c640a3c40 0x7f6c640a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005fd0 tx=0x7f6c70005ee0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 msgr2=0x7f6c7c198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 0x7f6c7c198e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0039c0 tx=0x7f6c6c0047b0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 shutdown_connections 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c640a3c40 0x7f6c640a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c104710 0x7f6c7c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7c102b70 0x7f6c7c198e40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 --2- 172.21.15.134:0/2958388256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c7c103d70 0x7f6c7c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.745+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 >> 172.21.15.134:0/2958388256 conn(0x7f6c7c0fe110 msgr2=0x7f6c7c107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.746+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 shutdown_connections 2024-04-16T05:31:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-16T05:31:55.746+0000 7f6c80fea700 1 -- 172.21.15.134:0/2958388256 wait complete. 2024-04-16T05:31:56.454 DEBUG:teuthology.orchestra.run.smithi032:mds.a> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.a.service 2024-04-16T05:31:56.457 DEBUG:teuthology.orchestra.run.smithi032:mds.d> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.d.service 2024-04-16T05:31:56.500 DEBUG:teuthology.orchestra.run.smithi032:mds.g> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.g.service 2024-04-16T05:31:56.543 DEBUG:teuthology.orchestra.run.smithi032:mds.j> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.j.service 2024-04-16T05:31:56.586 DEBUG:teuthology.orchestra.run.smithi082:mds.b> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.b.service 2024-04-16T05:31:56.589 DEBUG:teuthology.orchestra.run.smithi082:mds.e> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.e.service 2024-04-16T05:31:56.631 DEBUG:teuthology.orchestra.run.smithi082:mds.h> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.h.service 2024-04-16T05:31:56.674 DEBUG:teuthology.orchestra.run.smithi082:mds.k> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.k.service 2024-04-16T05:31:56.717 DEBUG:teuthology.orchestra.run.smithi134:mds.c> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.c.service 2024-04-16T05:31:56.720 DEBUG:teuthology.orchestra.run.smithi134:mds.f> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.f.service 2024-04-16T05:31:56.762 DEBUG:teuthology.orchestra.run.smithi134:mds.i> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.i.service 2024-04-16T05:31:56.805 DEBUG:teuthology.orchestra.run.smithi134:mds.l> sudo journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.l.service 2024-04-16T05:31:56.849 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-16T05:31:56.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:31:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:31:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:56 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 -- 172.21.15.32:0/1121181133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 msgr2=0x7f28ac10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 --2- 172.21.15.32:0/1121181133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 0x7f28ac10a4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009900 tx=0x7f28a402bc30 comp rx=0 tx=0).stop 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 -- 172.21.15.32:0/1121181133 shutdown_connections 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 --2- 172.21.15.32:0/1121181133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 0x7f28ac10a4b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 --2- 172.21.15.32:0/1121181133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac103260 0x7f28ac1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 --2- 172.21.15.32:0/1121181133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28ac102070 0x7f28ac102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 -- 172.21.15.32:0/1121181133 >> 172.21.15.32:0/1121181133 conn(0x7f28ac0fd600 msgr2=0x7f28ac0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 -- 172.21.15.32:0/1121181133 shutdown_connections 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 -- 172.21.15.32:0/1121181133 wait complete. 2024-04-16T05:31:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.973+0000 7f28b470e640 1 Processor -- start 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b470e640 1 -- start start 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b470e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac102070 0x7f28ac075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b470e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28ac103260 0x7f28ac0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b470e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 0x7f28ac074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b470e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10dd50 con 0x7f28ac103c20 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b470e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10dbd0 con 0x7f28ac102070 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b470e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10ded0 con 0x7f28ac103260 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 0x7f28ac074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 0x7f28ac074930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59024/0 (socket says 172.21.15.32:59024) 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 -- 172.21.15.32:0/3186548088 learned_addr learned my addr 172.21.15.32:0/3186548088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b24aa640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac102070 0x7f28ac075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b1ca9640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28ac103260 0x7f28ac0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 -- 172.21.15.32:0/3186548088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28ac103260 msgr2=0x7f28ac0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28ac103260 0x7f28ac0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 -- 172.21.15.32:0/3186548088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac102070 msgr2=0x7f28ac075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac102070 0x7f28ac075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 -- 172.21.15.32:0/3186548088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac0751c0 con 0x7f28ac103c20 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b24aa640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac102070 0x7f28ac075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28b2cab640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 0x7f28ac074930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f28a402c8e0 tx=0x7f28a402c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.974+0000 7f28ab7fe640 1 -- 172.21.15.32:0/3186548088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4043020 con 0x7f28ac103c20 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.975+0000 7f28ab7fe640 1 -- 172.21.15.32:0/3186548088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a40054e0 con 0x7f28ac103c20 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.975+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac1a2430 con 0x7f28ac103c20 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.975+0000 7f28ab7fe640 1 -- 172.21.15.32:0/3186548088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4003ce0 con 0x7f28ac103c20 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.975+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac1a2980 con 0x7f28ac103c20 2024-04-16T05:31:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.975+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ac1092c0 con 0x7f28ac103c20 2024-04-16T05:31:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.977+0000 7f28ab7fe640 1 -- 172.21.15.32:0/3186548088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f28a4046020 con 0x7f28ac103c20 2024-04-16T05:31:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.978+0000 7f28ab7fe640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28940a3bb0 0x7f28940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.978+0000 7f28b24aa640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28940a3bb0 0x7f28940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.978+0000 7f28ab7fe640 1 -- 172.21.15.32:0/3186548088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f28a40f2430 con 0x7f28ac103c20 2024-04-16T05:31:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.978+0000 7f28b24aa640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28940a3bb0 0x7f28940a6070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f289c009fd0 tx=0x7f289c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:56.981+0000 7f28ab7fe640 1 -- 172.21.15.32:0/3186548088 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28a404d020 con 0x7f28ac103c20 2024-04-16T05:31:57.047 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:57.047 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:57.047 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-16T05:31:57.047 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:57.047 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:57.047 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:57.048 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:57.048 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:57.048 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:31:57.048 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:31:57.048 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:56 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:57.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.168+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28ac062c00 con 0x7f28ac103c20 2024-04-16T05:31:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.169+0000 7f28ab7fe640 1 -- 172.21.15.32:0/3186548088 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18420 (secure 0 0 0) 0x7f28a40bf290 con 0x7f28ac103c20 2024-04-16T05:31:57.168 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:31:57.168 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:31:47.151909+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:31:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28940a3bb0 msgr2=0x7f28940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28940a3bb0 0x7f28940a6070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f289c009fd0 tx=0x7f289c009680 comp rx=0 tx=0).stop 2024-04-16T05:31:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 msgr2=0x7f28ac074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 0x7f28ac074930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f28a402c8e0 tx=0x7f28a402c6a0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 shutdown_connections 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28940a3bb0 0x7f28940a6070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28ac103c20 0x7f28ac074930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28ac103260 0x7f28ac0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 --2- 172.21.15.32:0/3186548088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac102070 0x7f28ac075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.171+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 >> 172.21.15.32:0/3186548088 conn(0x7f28ac0fd600 msgr2=0x7f28ac106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.172+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 shutdown_connections 2024-04-16T05:31:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.172+0000 7f28b470e640 1 -- 172.21.15.32:0/3186548088 wait complete. 2024-04-16T05:31:57.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:57.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:31:57.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-16T05:31:57.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:57.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:31:57.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:57.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:31:57.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:31:57.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:31:57.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:31:57.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:56 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:31:57.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T05:31:57.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:31:57.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2125569317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38102970 msgr2=0x7f9d38109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2125569317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38102970 0x7f9d38109290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009ca0 tx=0x7f9d3002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2125569317 shutdown_connections 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2125569317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38102970 0x7f9d38109290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2125569317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d38101fb0 0x7f9d38102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2125569317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d38100e60 0x7f9d38101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2125569317 >> 172.21.15.32:0/2125569317 conn(0x7f9d380fc3f0 msgr2=0x7f9d380fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2125569317 shutdown_connections 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2125569317 wait complete. 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 Processor -- start 2024-04-16T05:31:57.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.293+0000 7f9d3d3ff640 1 -- start start 2024-04-16T05:31:57.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d3d3ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d38100e60 0x7f9d3806b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:57.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d3d3ff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38101fb0 0x7f9d3819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d3d3ff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d38102970 0x7f9d3819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d3d3ff640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810cae0 con 0x7f9d38101fb0 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d3d3ff640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810c960 con 0x7f9d38100e60 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d3d3ff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810cc60 con 0x7f9d38102970 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38101fb0 0x7f9d3819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38101fb0 0x7f9d3819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59040/0 (socket says 172.21.15.32:59040) 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 -- 172.21.15.32:0/2857064779 learned_addr learned my addr 172.21.15.32:0/2857064779 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d377fe640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d38102970 0x7f9d3819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d36ffd640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d38100e60 0x7f9d3806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 -- 172.21.15.32:0/2857064779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d38102970 msgr2=0x7f9d3819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d38102970 0x7f9d3819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 -- 172.21.15.32:0/2857064779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d38100e60 msgr2=0x7f9d3806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d38100e60 0x7f9d3806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 -- 172.21.15.32:0/2857064779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3819fe20 con 0x7f9d38101fb0 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d36ffd640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d38100e60 0x7f9d3806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d367fc640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38101fb0 0x7f9d3819cf20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00d950 tx=0x7f9d2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d1ffff640 1 -- 172.21.15.32:0/2857064779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c017020 con 0x7f9d38101fb0 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d1ffff640 1 -- 172.21.15.32:0/2857064779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2c009c30 con 0x7f9d38101fb0 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d381a0110 con 0x7f9d38101fb0 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d1ffff640 1 -- 172.21.15.32:0/2857064779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00f720 con 0x7f9d38101fb0 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.294+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d381a0660 con 0x7f9d38101fb0 2024-04-16T05:31:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.295+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d38108110 con 0x7f9d38101fb0 2024-04-16T05:31:57.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.297+0000 7f9d1ffff640 1 -- 172.21.15.32:0/2857064779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d2c01a020 con 0x7f9d38101fb0 2024-04-16T05:31:57.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.297+0000 7f9d1ffff640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9d200a3bb0 0x7f9d200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:57.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.297+0000 7f9d36ffd640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9d200a3bb0 0x7f9d200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:57.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.297+0000 7f9d1ffff640 1 -- 172.21.15.32:0/2857064779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9d2c0d01c0 con 0x7f9d38101fb0 2024-04-16T05:31:57.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.298+0000 7f9d36ffd640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9d200a3bb0 0x7f9d200a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d28005d00 tx=0x7f9d28005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.301+0000 7f9d1ffff640 1 -- 172.21.15.32:0/2857064779 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d2c021020 con 0x7f9d38101fb0 2024-04-16T05:31:57.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.487+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d3802d8b0 con 0x7f9d38101fb0 2024-04-16T05:31:57.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.487+0000 7f9d1ffff640 1 -- 172.21.15.32:0/2857064779 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f9d2c009de0 con 0x7f9d38101fb0 2024-04-16T05:31:57.486 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:31:57.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9d200a3bb0 msgr2=0x7f9d200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9d200a3bb0 0x7f9d200a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d28005d00 tx=0x7f9d28005c70 comp rx=0 tx=0).stop 2024-04-16T05:31:57.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38101fb0 msgr2=0x7f9d3819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38101fb0 0x7f9d3819cf20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00d950 tx=0x7f9d2c00de10 comp rx=0 tx=0).stop 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 shutdown_connections 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9d200a3bb0 0x7f9d200a6070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d38102970 0x7f9d3819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d38101fb0 0x7f9d3819cf20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 --2- 172.21.15.32:0/2857064779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d38100e60 0x7f9d3806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.489+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 >> 172.21.15.32:0/2857064779 conn(0x7f9d380fc3f0 msgr2=0x7f9d381052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.490+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 shutdown_connections 2024-04-16T05:31:57.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.490+0000 7f9d3d3ff640 1 -- 172.21.15.32:0/2857064779 wait complete. 2024-04-16T05:31:57.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 -- 172.21.15.32:0/3821706152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 msgr2=0x7fdaa0102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3821706152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 0x7fdaa0102550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fda88009900 tx=0x7fda8802bc30 comp rx=0 tx=0).stop 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 -- 172.21.15.32:0/3821706152 shutdown_connections 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3821706152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa0102a90 0x7fdaa01093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3821706152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 0x7fdaa0102550 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3821706152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa0100f00 0x7fdaa0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 -- 172.21.15.32:0/3821706152 >> 172.21.15.32:0/3821706152 conn(0x7fdaa00fc4e0 msgr2=0x7fdaa00fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 -- 172.21.15.32:0/3821706152 shutdown_connections 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 -- 172.21.15.32:0/3821706152 wait complete. 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.607+0000 7fdaa600c640 1 Processor -- start 2024-04-16T05:31:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fdaa600c640 1 -- start start 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fdaa600c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa0100f00 0x7fdaa01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fdaa600c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 0x7fdaa01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fdaa600c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa0102a90 0x7fdaa019f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fdaa600c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa010cbb0 con 0x7fdaa01020d0 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fdaa600c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa010ca30 con 0x7fdaa0102a90 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fdaa600c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa010cd30 con 0x7fdaa0100f00 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 0x7fdaa01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 0x7fdaa01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59064/0 (socket says 172.21.15.32:59064) 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9effd640 1 -- 172.21.15.32:0/3307111442 learned_addr learned my addr 172.21.15.32:0/3307111442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9f7fe640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa0100f00 0x7fdaa01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9ffff640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa0102a90 0x7fdaa019f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9effd640 1 -- 172.21.15.32:0/3307111442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa0100f00 msgr2=0x7fdaa01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9effd640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa0100f00 0x7fdaa01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9effd640 1 -- 172.21.15.32:0/3307111442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa0102a90 msgr2=0x7fdaa019f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9effd640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa0102a90 0x7fdaa019f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.608+0000 7fda9effd640 1 -- 172.21.15.32:0/3307111442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa019ffd0 con 0x7fdaa01020d0 2024-04-16T05:31:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.609+0000 7fda9effd640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 0x7fdaa01988f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fda88009230 tx=0x7fda88005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.609+0000 7fda9cff9640 1 -- 172.21.15.32:0/3307111442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88041070 con 0x7fdaa01020d0 2024-04-16T05:31:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.609+0000 7fda9cff9640 1 -- 172.21.15.32:0/3307111442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda880054e0 con 0x7fdaa01020d0 2024-04-16T05:31:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.609+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa01a0260 con 0x7fdaa01020d0 2024-04-16T05:31:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.609+0000 7fda9cff9640 1 -- 172.21.15.32:0/3307111442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88034440 con 0x7fdaa01020d0 2024-04-16T05:31:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.609+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa01a07a0 con 0x7fdaa01020d0 2024-04-16T05:31:57.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.610+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa01a0a00 con 0x7fdaa01020d0 2024-04-16T05:31:57.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.611+0000 7fda9cff9640 1 -- 172.21.15.32:0/3307111442 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fda88044070 con 0x7fdaa01020d0 2024-04-16T05:31:57.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.612+0000 7fda9cff9640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda8c0a3c30 0x7fda8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:57.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.612+0000 7fda9f7fe640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda8c0a3c30 0x7fda8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:57.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.612+0000 7fda9cff9640 1 -- 172.21.15.32:0/3307111442 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fda880f1b50 con 0x7fdaa01020d0 2024-04-16T05:31:57.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.612+0000 7fda9f7fe640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda8c0a3c30 0x7fda8c0a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0100c20 tx=0x7fda9000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.617+0000 7fda9cff9640 1 -- 172.21.15.32:0/3307111442 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda880be380 con 0x7fdaa01020d0 2024-04-16T05:31:57.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:57 smithi032 ceph-mon[32351]: Deploying daemon mds.l on smithi134 2024-04-16T05:31:57.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3186548088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:31:57.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2857064779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:31:57.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.844+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaa01a0bd0 con 0x7fdaa01020d0 2024-04-16T05:31:57.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.845+0000 7fda9cff9640 1 -- 172.21.15.32:0/3307111442 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fda880c2680 con 0x7fdaa01020d0 2024-04-16T05:31:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:31:57.844 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-16T05:31:57.844 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 1 2024-04-16T05:31:57.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda8c0a3c30 msgr2=0x7fda8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda8c0a3c30 0x7fda8c0a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0100c20 tx=0x7fda9000b390 comp rx=0 tx=0).stop 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 msgr2=0x7fdaa01988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 0x7fdaa01988f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fda88009230 tx=0x7fda88005dd0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 shutdown_connections 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fda8c0a3c30 0x7fda8c0a60f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa0102a90 0x7fdaa019f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa01020d0 0x7fdaa01988f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 --2- 172.21.15.32:0/3307111442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa0100f00 0x7fdaa01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 >> 172.21.15.32:0/3307111442 conn(0x7fdaa00fc4e0 msgr2=0x7fdaa00fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 shutdown_connections 2024-04-16T05:31:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:57.847+0000 7fdaa600c640 1 -- 172.21.15.32:0/3307111442 wait complete. 2024-04-16T05:31:58.025 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:57 smithi082 ceph-mon[36195]: Deploying daemon mds.l on smithi134 2024-04-16T05:31:58.025 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:57 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3186548088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:31:58.025 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:57 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2857064779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:31:58.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:57 smithi134 ceph-mon[36153]: Deploying daemon mds.l on smithi134 2024-04-16T05:31:58.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:57 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3186548088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:31:58.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:57 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2857064779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:31:58.748 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:58 smithi134 systemd[1]: Starting Ceph mds.l for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:31:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:58 smithi082 ceph-mon[36195]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:31:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3307111442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:31:59.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:58 smithi134 ceph-mon[36153]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:59.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:31:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3307111442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:31:59.094 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:58 smithi134 podman[66954]: 2024-04-16 05:31:58.683862629 +0000 UTC m=+0.017961729 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:31:59.094 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:58 smithi134 podman[66954]: 2024-04-16 05:31:58.807954777 +0000 UTC m=+0.142053867 container create d50418adbc417477dd78663270c4dffa447b40733b56478412773b46e06bb912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:31:59.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:58 smithi032 ceph-mon[32351]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:31:59.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:31:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3307111442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:31:59.384 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:59 smithi134 podman[66954]: 2024-04-16 05:31:59.096412489 +0000 UTC m=+0.430511576 container init d50418adbc417477dd78663270c4dffa447b40733b56478412773b46e06bb912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:31:59.385 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:59 smithi134 podman[66954]: 2024-04-16 05:31:59.099261614 +0000 UTC m=+0.433360704 container start d50418adbc417477dd78663270c4dffa447b40733b56478412773b46e06bb912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-16T05:31:59.385 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:59 smithi134 ceph-mds[66973]: -- 172.21.15.134:0/1659463088 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dda4ef3a40 con 0x55dda4ec6800 2024-04-16T05:31:59.385 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:59 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l[66969]: starting mds.l at 2024-04-16T05:31:59.385 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:59 smithi134 bash[66954]: d50418adbc417477dd78663270c4dffa447b40733b56478412773b46e06bb912 2024-04-16T05:31:59.385 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:31:59 smithi134 systemd[1]: Started Ceph mds.l for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:31:59.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:31:59.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 -- 172.21.15.32:0/726816806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414102070 msgr2=0x7f0414102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:59.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 --2- 172.21.15.32:0/726816806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414102070 0x7f0414102490 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0404009900 tx=0x7f040402bc30 comp rx=0 tx=0).stop 2024-04-16T05:31:59.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 -- 172.21.15.32:0/726816806 shutdown_connections 2024-04-16T05:31:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 --2- 172.21.15.32:0/726816806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0414103c20 0x7f041410a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 --2- 172.21.15.32:0/726816806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0414103260 0x7f04141036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 --2- 172.21.15.32:0/726816806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414102070 0x7f0414102490 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 -- 172.21.15.32:0/726816806 >> 172.21.15.32:0/726816806 conn(0x7f04140fd600 msgr2=0x7f04140ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:31:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 -- 172.21.15.32:0/726816806 shutdown_connections 2024-04-16T05:31:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.978+0000 7f0419c2a640 1 -- 172.21.15.32:0/726816806 wait complete. 2024-04-16T05:31:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0419c2a640 1 Processor -- start 2024-04-16T05:31:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0419c2a640 1 -- start start 2024-04-16T05:31:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0419c2a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0414102070 0x7f04141982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0419c2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414103260 0x7f04141987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0419c2a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0414103c20 0x7f041419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0419c2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f041410dd00 con 0x7f0414103260 2024-04-16T05:31:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0419c2a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f041410db80 con 0x7f0414102070 2024-04-16T05:31:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0419c2a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f041410de80 con 0x7f0414103c20 2024-04-16T05:31:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0412ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414103260 0x7f04141987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f04137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0414102070 0x7f04141982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0412ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414103260 0x7f04141987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59112/0 (socket says 172.21.15.32:59112) 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.979+0000 7f0412ffd640 1 -- 172.21.15.32:0/2354433105 learned_addr learned my addr 172.21.15.32:0/2354433105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0413fff640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0414103c20 0x7f041419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0412ffd640 1 -- 172.21.15.32:0/2354433105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0414103c20 msgr2=0x7f041419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0412ffd640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0414103c20 0x7f041419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0412ffd640 1 -- 172.21.15.32:0/2354433105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0414102070 msgr2=0x7f04141982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0412ffd640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0414102070 0x7f04141982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0412ffd640 1 -- 172.21.15.32:0/2354433105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041419fed0 con 0x7f0414103260 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0413fff640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0414103c20 0x7f041419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0412ffd640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414103260 0x7f04141987f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f040800c920 tx=0x7f040800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0410ff9640 1 -- 172.21.15.32:0/2354433105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408007c70 con 0x7f0414103260 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0410ff9640 1 -- 172.21.15.32:0/2354433105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0408007e10 con 0x7f0414103260 2024-04-16T05:31:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0410ff9640 1 -- 172.21.15.32:0/2354433105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408018890 con 0x7f0414103260 2024-04-16T05:31:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04141a01c0 con 0x7f0414103260 2024-04-16T05:31:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.980+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04141a0710 con 0x7f0414103260 2024-04-16T05:31:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.981+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0414109330 con 0x7f0414103260 2024-04-16T05:31:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.983+0000 7f0410ff9640 1 -- 172.21.15.32:0/2354433105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0408018a70 con 0x7f0414103260 2024-04-16T05:31:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.983+0000 7f0410ff9640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03fc0a3bb0 0x7f03fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:31:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.983+0000 7f04137fe640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03fc0a3bb0 0x7f03fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:31:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.983+0000 7f0410ff9640 1 -- 172.21.15.32:0/2354433105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f04080d00a0 con 0x7f0414103260 2024-04-16T05:31:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.984+0000 7f04137fe640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03fc0a3bb0 0x7f03fc0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0404000c00 tx=0x7f0404005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:31:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:31:59.988+0000 7f0410ff9640 1 -- 172.21.15.32:0/2354433105 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0408014030 con 0x7f0414103260 2024-04-16T05:32:00.200 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.134:6832/1600676004,v1:172.21.15.134:6833/1600676004] up:boot 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: fsmap 1 up:standby 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:00.201 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:00 smithi082 ceph-mon[36195]: Deploying daemon mds.k on smithi082 2024-04-16T05:32:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.215+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f041402d620 con 0x7f0414103260 2024-04-16T05:32:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.216+0000 7f0410ff9640 1 -- 172.21.15.32:0/2354433105 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f040809cf20 con 0x7f0414103260 2024-04-16T05:32:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:00.215 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T05:32:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2024-04-16T05:32:00.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03fc0a3bb0 msgr2=0x7f03fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03fc0a3bb0 0x7f03fc0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0404000c00 tx=0x7f0404005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414103260 msgr2=0x7f04141987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414103260 0x7f04141987f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f040800c920 tx=0x7f040800cde0 comp rx=0 tx=0).stop 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 shutdown_connections 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03fc0a3bb0 0x7f03fc0a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0414103c20 0x7f041419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0414103260 0x7f04141987f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 --2- 172.21.15.32:0/2354433105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0414102070 0x7f04141982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 >> 172.21.15.32:0/2354433105 conn(0x7f04140fd600 msgr2=0x7f0414106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 shutdown_connections 2024-04-16T05:32:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:00.218+0000 7f0419c2a640 1 -- 172.21.15.32:0/2354433105 wait complete. 2024-04-16T05:32:00.226 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.134:6832/1600676004,v1:172.21.15.134:6833/1600676004] up:boot 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: fsmap 1 up:standby 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:00.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:00 smithi032 ceph-mon[32351]: Deploying daemon mds.k on smithi082 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.134:6832/1600676004,v1:172.21.15.134:6833/1600676004] up:boot 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: fsmap 1 up:standby 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:00.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:00.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:00.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:00 smithi134 ceph-mon[36153]: Deploying daemon mds.k on smithi082 2024-04-16T05:32:01.242 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2354433105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:01.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2354433105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:01.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2354433105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:01.640 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:01 smithi082 systemd[1]: Starting Ceph mds.k for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:01.900 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:01 smithi082 podman[65457]: 2024-04-16 05:32:01.544758891 +0000 UTC m=+0.017553546 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:01.900 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:01 smithi082 podman[65457]: 2024-04-16 05:32:01.65073421 +0000 UTC m=+0.123528870 container create c7b28af0361065d24b50eefe8efba2efc64275b2ed0d9d8d892125555ff78d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-16T05:32:02.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.340+0000 7fb629e7b640 1 -- 172.21.15.32:0/2975329849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102090 msgr2=0x7fb624102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.340+0000 7fb629e7b640 1 --2- 172.21.15.32:0/2975329849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102090 0x7fb624102510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb614009900 tx=0x7fb61402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.341+0000 7fb629e7b640 1 -- 172.21.15.32:0/2975329849 shutdown_connections 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.341+0000 7fb629e7b640 1 --2- 172.21.15.32:0/2975329849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb624102a50 0x7fb624109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.341+0000 7fb629e7b640 1 --2- 172.21.15.32:0/2975329849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102090 0x7fb624102510 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.341+0000 7fb629e7b640 1 --2- 172.21.15.32:0/2975329849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb624100f40 0x7fb624101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.341+0000 7fb629e7b640 1 -- 172.21.15.32:0/2975329849 >> 172.21.15.32:0/2975329849 conn(0x7fb6240fc4b0 msgr2=0x7fb6240fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.341+0000 7fb629e7b640 1 -- 172.21.15.32:0/2975329849 shutdown_connections 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.341+0000 7fb629e7b640 1 -- 172.21.15.32:0/2975329849 wait complete. 2024-04-16T05:32:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.341+0000 7fb629e7b640 1 Processor -- start 2024-04-16T05:32:02.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb629e7b640 1 -- start start 2024-04-16T05:32:02.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb629e7b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb624100f40 0x7fb624069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:02.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb629e7b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb624102090 0x7fb62419d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:02.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb629e7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102a50 0x7fb62419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb629e7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410cc40 con 0x7fb624102a50 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb629e7b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410cac0 con 0x7fb624102090 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb629e7b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410cdc0 con 0x7fb624100f40 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102a50 0x7fb62419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102a50 0x7fb62419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59138/0 (socket says 172.21.15.32:59138) 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb622ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb624102090 0x7fb62419d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 -- 172.21.15.32:0/3985868666 learned_addr learned my addr 172.21.15.32:0/3985868666 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb6237fe640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb624100f40 0x7fb624069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 -- 172.21.15.32:0/3985868666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb624100f40 msgr2=0x7fb624069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb624100f40 0x7fb624069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 -- 172.21.15.32:0/3985868666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb624102090 msgr2=0x7fb62419d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb624102090 0x7fb62419d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 -- 172.21.15.32:0/3985868666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6241a0040 con 0x7fb624102a50 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.342+0000 7fb623fff640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102a50 0x7fb62419f780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009fd0 tx=0x7fb61c00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.343+0000 7fb620ff9640 1 -- 172.21.15.32:0/3985868666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c017070 con 0x7fb624102a50 2024-04-16T05:32:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.343+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6241a0330 con 0x7fb624102a50 2024-04-16T05:32:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.343+0000 7fb620ff9640 1 -- 172.21.15.32:0/3985868666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb61c003e50 con 0x7fb624102a50 2024-04-16T05:32:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.343+0000 7fb620ff9640 1 -- 172.21.15.32:0/3985868666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c010460 con 0x7fb624102a50 2024-04-16T05:32:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.343+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6241a0800 con 0x7fb624102a50 2024-04-16T05:32:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.344+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6241082a0 con 0x7fb624102a50 2024-04-16T05:32:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.345+0000 7fb620ff9640 1 -- 172.21.15.32:0/3985868666 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb61c003970 con 0x7fb624102a50 2024-04-16T05:32:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.346+0000 7fb620ff9640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.346+0000 7fb6237fe640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.346+0000 7fb620ff9640 1 -- 172.21.15.32:0/3985868666 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb61c0cfcd0 con 0x7fb624102a50 2024-04-16T05:32:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.346+0000 7fb6237fe640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6100a3bb0 0x7fb6100a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb60c009990 tx=0x7fb60c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:02.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.350+0000 7fb620ff9640 1 -- 172.21.15.32:0/3985868666 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb61c09cab0 con 0x7fb624102a50 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:02 smithi082 ceph-mon[36195]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:02 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:02 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:01 smithi082 podman[65457]: 2024-04-16 05:32:01.917045936 +0000 UTC m=+0.389840578 container init c7b28af0361065d24b50eefe8efba2efc64275b2ed0d9d8d892125555ff78d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:01 smithi082 podman[65457]: 2024-04-16 05:32:01.919583945 +0000 UTC m=+0.392378580 container start c7b28af0361065d24b50eefe8efba2efc64275b2ed0d9d8d892125555ff78d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:01 smithi082 ceph-mds[65476]: -- 172.21.15.82:0/2367875575 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dac807da40 con 0x55dac8050800 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:01 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k[65472]: starting mds.k at 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:02 smithi082 bash[65457]: c7b28af0361065d24b50eefe8efba2efc64275b2ed0d9d8d892125555ff78d39 2024-04-16T05:32:02.400 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:32:02 smithi082 systemd[1]: Started Ceph mds.k for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:02.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:02 smithi032 ceph-mon[32351]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:02.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:02.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:02 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:02.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.576+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb624062c00 con 0x7fb624102a50 2024-04-16T05:32:02.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.577+0000 7fb620ff9640 1 -- 172.21.15.32:0/3985868666 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fb61c09c3b0 con 0x7fb624102a50 2024-04-16T05:32:02.576 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:02.576 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T05:32:02.577 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2024-04-16T05:32:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6100a3bb0 msgr2=0x7fb6100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6100a3bb0 0x7fb6100a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb60c009990 tx=0x7fb60c009110 comp rx=0 tx=0).stop 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102a50 msgr2=0x7fb62419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102a50 0x7fb62419f780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009fd0 tx=0x7fb61c00def0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 shutdown_connections 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb624102a50 0x7fb62419f780 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb624102090 0x7fb62419d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 --2- 172.21.15.32:0/3985868666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb624100f40 0x7fb624069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 >> 172.21.15.32:0/3985868666 conn(0x7fb6240fc4b0 msgr2=0x7fb624105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 shutdown_connections 2024-04-16T05:32:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:02.579+0000 7fb629e7b640 1 -- 172.21.15.32:0/3985868666 wait complete. 2024-04-16T05:32:02.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:02 smithi134 ceph-mon[36153]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:02.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:02 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:02.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:02 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6832/1596016884,v1:172.21.15.82:6833/1596016884] up:boot 2024-04-16T05:32:03.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: fsmap 2 up:standby 2024-04-16T05:32:03.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:32:03.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:03.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:03.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:03.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:03.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: Deploying daemon mds.j on smithi032 2024-04-16T05:32:03.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:03 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3985868666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:03.372 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6832/1596016884,v1:172.21.15.82:6833/1596016884] up:boot 2024-04-16T05:32:03.373 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: fsmap 2 up:standby 2024-04-16T05:32:03.373 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:32:03.373 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:03.373 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:03.373 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:03.373 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:03.373 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: Deploying daemon mds.j on smithi032 2024-04-16T05:32:03.373 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:03 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3985868666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6832/1596016884,v1:172.21.15.82:6833/1596016884] up:boot 2024-04-16T05:32:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: fsmap 2 up:standby 2024-04-16T05:32:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:32:03.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:03.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:03.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:03.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:03.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: Deploying daemon mds.j on smithi032 2024-04-16T05:32:03.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:03 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3985868666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:04.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:04 smithi032 ceph-mon[32351]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:04.483 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 systemd[1]: Starting Ceph mds.j for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:04.536 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:04 smithi134 ceph-mon[36153]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:04.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:04 smithi082 ceph-mon[36195]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 -- 172.21.15.32:0/2236267191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0102aa0 msgr2=0x7f35e01093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 --2- 172.21.15.32:0/2236267191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0102aa0 0x7f35e01093c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009ca0 tx=0x7f35d802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 -- 172.21.15.32:0/2236267191 shutdown_connections 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 --2- 172.21.15.32:0/2236267191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0102aa0 0x7f35e01093c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 --2- 172.21.15.32:0/2236267191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35e01020e0 0x7f35e0102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 --2- 172.21.15.32:0/2236267191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0100f10 0x7f35e0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 -- 172.21.15.32:0/2236267191 >> 172.21.15.32:0/2236267191 conn(0x7f35e00fc4f0 msgr2=0x7f35e00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 -- 172.21.15.32:0/2236267191 shutdown_connections 2024-04-16T05:32:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.702+0000 7f35e590d640 1 -- 172.21.15.32:0/2236267191 wait complete. 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35e590d640 1 Processor -- start 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35e590d640 1 -- start start 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35e590d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0100f10 0x7f35e0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35e590d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35e01020e0 0x7f35e01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35e590d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102aa0 0x7f35e019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35e590d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f35e010cc30 con 0x7f35e0100f10 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35e590d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f35e010cab0 con 0x7f35e0102aa0 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35e590d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35e010cdb0 con 0x7f35e01020e0 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35deffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0100f10 0x7f35e0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35deffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0100f10 0x7f35e0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59150/0 (socket says 172.21.15.32:59150) 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.703+0000 7f35deffd640 1 -- 172.21.15.32:0/3846706711 learned_addr learned my addr 172.21.15.32:0/3846706711 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35d7fff640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35e01020e0 0x7f35e01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35df7fe640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102aa0 0x7f35e019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35deffd640 1 -- 172.21.15.32:0/3846706711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35e01020e0 msgr2=0x7f35e01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35deffd640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35e01020e0 0x7f35e01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35deffd640 1 -- 172.21.15.32:0/3846706711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102aa0 msgr2=0x7f35e019f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35deffd640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102aa0 0x7f35e019f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35deffd640 1 -- 172.21.15.32:0/3846706711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e019ffb0 con 0x7f35e0100f10 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35deffd640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0100f10 0x7f35e0198390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c970 tx=0x7f35d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35e490b640 1 -- 172.21.15.32:0/3846706711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0004220 con 0x7f35e0100f10 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e01a02a0 con 0x7f35e0100f10 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35e490b640 1 -- 172.21.15.32:0/3846706711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d00043c0 con 0x7f35e0100f10 2024-04-16T05:32:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35e490b640 1 -- 172.21.15.32:0/3846706711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d000f450 con 0x7f35e0100f10 2024-04-16T05:32:04.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.704+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e01a07f0 con 0x7f35e0100f10 2024-04-16T05:32:04.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.705+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e01a0ab0 con 0x7f35e0100f10 2024-04-16T05:32:04.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.707+0000 7f35e490b640 1 -- 172.21.15.32:0/3846706711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f35d00079b0 con 0x7f35e0100f10 2024-04-16T05:32:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.707+0000 7f35e490b640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f35cc0a3bb0 0x7f35cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.707+0000 7f35d7fff640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f35cc0a3bb0 0x7f35cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.708+0000 7f35e490b640 1 -- 172.21.15.32:0/3846706711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f35d00cfd70 con 0x7f35e0100f10 2024-04-16T05:32:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.708+0000 7f35d7fff640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f35cc0a3bb0 0x7f35cc0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35e01998a0 tx=0x7f35c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:04.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.712+0000 7f35e490b640 1 -- 172.21.15.32:0/3846706711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35d0014030 con 0x7f35e0100f10 2024-04-16T05:32:04.874 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 podman[73142]: 2024-04-16 05:32:04.515106064 +0000 UTC m=+0.017377740 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:04.875 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 podman[73142]: 2024-04-16 05:32:04.613113536 +0000 UTC m=+0.115385185 container create ad68f6820de12141c28d739206c67d66ad6be6e8aa9b111fc4b4d841c6219ca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-16T05:32:04.875 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 podman[73142]: 2024-04-16 05:32:04.849789805 +0000 UTC m=+0.352061469 container init ad68f6820de12141c28d739206c67d66ad6be6e8aa9b111fc4b4d841c6219ca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:32:04.875 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 podman[73142]: 2024-04-16 05:32:04.853256502 +0000 UTC m=+0.355528158 container start ad68f6820de12141c28d739206c67d66ad6be6e8aa9b111fc4b4d841c6219ca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-16T05:32:04.875 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 ceph-mds[73200]: -- 172.21.15.32:0/2325002333 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3eff2ba40 con 0x55a3efefe800 2024-04-16T05:32:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.944+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e01a0da0 con 0x7f35e0100f10 2024-04-16T05:32:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.945+0000 7f35e490b640 1 -- 172.21.15.32:0/3846706711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f35d009cbf0 con 0x7f35e0100f10 2024-04-16T05:32:04.944 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:04.945 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T05:32:04.945 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2024-04-16T05:32:04.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f35cc0a3bb0 msgr2=0x7f35cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f35cc0a3bb0 0x7f35cc0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35e01998a0 tx=0x7f35c8008040 comp rx=0 tx=0).stop 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0100f10 msgr2=0x7f35e0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0100f10 0x7f35e0198390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c970 tx=0x7f35d000ce30 comp rx=0 tx=0).stop 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 shutdown_connections 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f35cc0a3bb0 0x7f35cc0a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102aa0 0x7f35e019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35e01020e0 0x7f35e01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 --2- 172.21.15.32:0/3846706711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e0100f10 0x7f35e0198390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 >> 172.21.15.32:0/3846706711 conn(0x7f35e00fc4f0 msgr2=0x7f35e00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 shutdown_connections 2024-04-16T05:32:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:04.947+0000 7f35e590d640 1 -- 172.21.15.32:0/3846706711 wait complete. 2024-04-16T05:32:05.168 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j[73196]: starting mds.j at 2024-04-16T05:32:05.169 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 bash[73142]: ad68f6820de12141c28d739206c67d66ad6be6e8aa9b111fc4b4d841c6219ca3 2024-04-16T05:32:05.169 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:32:04 smithi032 systemd[1]: Started Ceph mds.j for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:05.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:05 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3846706711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:05 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:05.547 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:05 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3846706711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:05.547 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.547 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.547 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.547 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:05.547 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:05.547 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:05 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:05 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3846706711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:05 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:06.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:06 smithi032 ceph-mon[32351]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:06.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:06 smithi032 ceph-mon[32351]: Deploying daemon mds.d on smithi032 2024-04-16T05:32:06.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:06 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.32:6834/3476798351,v1:172.21.15.32:6835/3476798351] up:boot 2024-04-16T05:32:06.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:06 smithi032 ceph-mon[32351]: fsmap 3 up:standby 2024-04-16T05:32:06.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:06 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:32:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:06 smithi134 ceph-mon[36153]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:06 smithi134 ceph-mon[36153]: Deploying daemon mds.d on smithi032 2024-04-16T05:32:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:06 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.32:6834/3476798351,v1:172.21.15.32:6835/3476798351] up:boot 2024-04-16T05:32:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:06 smithi134 ceph-mon[36153]: fsmap 3 up:standby 2024-04-16T05:32:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:06 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:32:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:06 smithi082 ceph-mon[36195]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:06 smithi082 ceph-mon[36195]: Deploying daemon mds.d on smithi032 2024-04-16T05:32:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:06 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.32:6834/3476798351,v1:172.21.15.32:6835/3476798351] up:boot 2024-04-16T05:32:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:06 smithi082 ceph-mon[36195]: fsmap 3 up:standby 2024-04-16T05:32:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:06 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:32:06.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.119+0000 7fde302e4640 1 -- 172.21.15.32:0/3487345853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28102070 msgr2=0x7fde28102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.119+0000 7fde302e4640 1 --2- 172.21.15.32:0/3487345853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28102070 0x7fde28102490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde24009900 tx=0x7fde2402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 -- 172.21.15.32:0/3487345853 shutdown_connections 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 --2- 172.21.15.32:0/3487345853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde28103c20 0x7fde2810a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 --2- 172.21.15.32:0/3487345853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28103260 0x7fde281036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 --2- 172.21.15.32:0/3487345853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28102070 0x7fde28102490 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 -- 172.21.15.32:0/3487345853 >> 172.21.15.32:0/3487345853 conn(0x7fde280fd5e0 msgr2=0x7fde280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 -- 172.21.15.32:0/3487345853 shutdown_connections 2024-04-16T05:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 -- 172.21.15.32:0/3487345853 wait complete. 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 Processor -- start 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 -- start start 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde28102070 0x7fde28198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28103260 0x7fde281988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28103c20 0x7fde2819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fde2810dd00 con 0x7fde28103260 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fde2810db80 con 0x7fde28102070 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde302e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde2810de80 con 0x7fde28103c20 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.120+0000 7fde2d87f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28103260 0x7fde281988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2d87f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28103260 0x7fde281988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59164/0 (socket says 172.21.15.32:59164) 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2d87f640 1 -- 172.21.15.32:0/1277241712 learned_addr learned my addr 172.21.15.32:0/1277241712 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2e080640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde28102070 0x7fde28198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2e881640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28103c20 0x7fde2819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2d87f640 1 -- 172.21.15.32:0/1277241712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28103c20 msgr2=0x7fde2819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2d87f640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28103c20 0x7fde2819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2d87f640 1 -- 172.21.15.32:0/1277241712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde28102070 msgr2=0x7fde28198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2d87f640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde28102070 0x7fde28198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2d87f640 1 -- 172.21.15.32:0/1277241712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2819ffa0 con 0x7fde28103260 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde2d87f640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28103260 0x7fde281988c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde1800d770 tx=0x7fde1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde1f7fe640 1 -- 172.21.15.32:0/1277241712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18017070 con 0x7fde28103260 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde281a0290 con 0x7fde28103260 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde1f7fe640 1 -- 172.21.15.32:0/1277241712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde18005500 con 0x7fde28103260 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde1f7fe640 1 -- 172.21.15.32:0/1277241712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18004e00 con 0x7fde28103260 2024-04-16T05:32:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.121+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281a0790 con 0x7fde28103260 2024-04-16T05:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.122+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde28109410 con 0x7fde28103260 2024-04-16T05:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.124+0000 7fde1f7fe640 1 -- 172.21.15.32:0/1277241712 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde18004fa0 con 0x7fde28103260 2024-04-16T05:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.125+0000 7fde1f7fe640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fde140a3bb0 0x7fde140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.125+0000 7fde2e080640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fde140a3bb0 0x7fde140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.125+0000 7fde1f7fe640 1 -- 172.21.15.32:0/1277241712 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fde180cfe90 con 0x7fde28103260 2024-04-16T05:32:07.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.125+0000 7fde2e080640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fde140a3bb0 0x7fde140a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde24000c00 tx=0x7fde24005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:07.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.132+0000 7fde1f7fe640 1 -- 172.21.15.32:0/1277241712 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde1809cc70 con 0x7fde28103260 2024-04-16T05:32:07.323 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 systemd[1]: Starting Ceph mds.d for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.369+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde2802d620 con 0x7fde28103260 2024-04-16T05:32:07.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.370+0000 7fde1f7fe640 1 -- 172.21.15.32:0/1277241712 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fde180cf070 con 0x7fde28103260 2024-04-16T05:32:07.368 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:07.369 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T05:32:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 4 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fde140a3bb0 msgr2=0x7fde140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fde140a3bb0 0x7fde140a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde24000c00 tx=0x7fde24005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28103260 msgr2=0x7fde281988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28103260 0x7fde281988c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde1800d770 tx=0x7fde1800dc30 comp rx=0 tx=0).stop 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 shutdown_connections 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fde140a3bb0 0x7fde140a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28103c20 0x7fde2819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde28103260 0x7fde281988c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 --2- 172.21.15.32:0/1277241712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde28102070 0x7fde28198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 >> 172.21.15.32:0/1277241712 conn(0x7fde280fd5e0 msgr2=0x7fde281065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 shutdown_connections 2024-04-16T05:32:07.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:07.372+0000 7fde302e4640 1 -- 172.21.15.32:0/1277241712 wait complete. 2024-04-16T05:32:07.631 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 podman[73526]: 2024-04-16 05:32:07.388370057 +0000 UTC m=+0.020091453 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:07.632 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 podman[73526]: 2024-04-16 05:32:07.511864564 +0000 UTC m=+0.143585975 container create d1d1f5c2e00536caf8f2f51aad12e9cc5e2cfa6d0730d963fe20337e9fa4db1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True, GIT_BRANCH=HEAD) 2024-04-16T05:32:08.178 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 podman[73526]: 2024-04-16 05:32:07.756318627 +0000 UTC m=+0.388039999 container init d1d1f5c2e00536caf8f2f51aad12e9cc5e2cfa6d0730d963fe20337e9fa4db1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:32:08.178 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 podman[73526]: 2024-04-16 05:32:07.759213959 +0000 UTC m=+0.390935329 container start d1d1f5c2e00536caf8f2f51aad12e9cc5e2cfa6d0730d963fe20337e9fa4db1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409) 2024-04-16T05:32:08.178 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 ceph-mds[73557]: -- 172.21.15.32:0/480813768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e80c79a40 con 0x557e80c4cc00 2024-04-16T05:32:08.178 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d[73551]: starting mds.d at 2024-04-16T05:32:08.178 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 bash[73526]: d1d1f5c2e00536caf8f2f51aad12e9cc5e2cfa6d0730d963fe20337e9fa4db1b 2024-04-16T05:32:08.179 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:32:07 smithi032 systemd[1]: Started Ceph mds.d for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:08 smithi032 ceph-mon[32351]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:08 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1277241712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:08.589 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:08 smithi082 ceph-mon[36195]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:08.589 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:08 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1277241712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:08.589 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.589 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.589 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.590 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:08.590 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:08.590 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:08 smithi134 ceph-mon[36153]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:08 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1277241712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:08.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:09.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:09.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:09 smithi032 ceph-mon[32351]: Deploying daemon mds.e on smithi082 2024-04-16T05:32:09.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:09 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.32:6836/3911772856,v1:172.21.15.32:6837/3911772856] up:boot 2024-04-16T05:32:09.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:09 smithi032 ceph-mon[32351]: fsmap 4 up:standby 2024-04-16T05:32:09.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:32:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 -- 172.21.15.32:0/2743815088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 msgr2=0x7f162c101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 --2- 172.21.15.32:0/2743815088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 0x7f162c101640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f161c009ca0 tx=0x7f161c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 -- 172.21.15.32:0/2743815088 shutdown_connections 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 --2- 172.21.15.32:0/2743815088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162c101b80 0x7f162c109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 --2- 172.21.15.32:0/2743815088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 0x7f162c101640 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 --2- 172.21.15.32:0/2743815088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f162c102570 0x7f162c102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 -- 172.21.15.32:0/2743815088 >> 172.21.15.32:0/2743815088 conn(0x7f162c0fbc70 msgr2=0x7f162c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 -- 172.21.15.32:0/2743815088 shutdown_connections 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.509+0000 7f1633088640 1 -- 172.21.15.32:0/2743815088 wait complete. 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1633088640 1 Processor -- start 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1633088640 1 -- start start 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1633088640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 0x7f162c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1633088640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f162c101b80 0x7f162c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1633088640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162c102570 0x7f162c19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1633088640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10cc50 con 0x7f162c1011c0 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1633088640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10cad0 con 0x7f162c102570 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1633088640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10cdd0 con 0x7f162c101b80 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1630e24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 0x7f162c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1630e24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 0x7f162c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53452/0 (socket says 172.21.15.32:53452) 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1630e24640 1 -- 172.21.15.32:0/644620697 learned_addr learned my addr 172.21.15.32:0/644620697 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1631625640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162c102570 0x7f162c19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f162bfff640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f162c101b80 0x7f162c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1630e24640 1 -- 172.21.15.32:0/644620697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f162c101b80 msgr2=0x7f162c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.510+0000 7f1630e24640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f162c101b80 0x7f162c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1630e24640 1 -- 172.21.15.32:0/644620697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162c102570 msgr2=0x7f162c19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1630e24640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162c102570 0x7f162c19f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1630e24640 1 -- 172.21.15.32:0/644620697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1a0040 con 0x7f162c1011c0 2024-04-16T05:32:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f162bfff640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f162c101b80 0x7f162c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1630e24640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 0x7f162c198420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f161400d950 tx=0x7f161400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1629ffb640 1 -- 172.21.15.32:0/644620697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614017070 con 0x7f162c1011c0 2024-04-16T05:32:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1629ffb640 1 -- 172.21.15.32:0/644620697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1614009c30 con 0x7f162c1011c0 2024-04-16T05:32:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1a0330 con 0x7f162c1011c0 2024-04-16T05:32:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1629ffb640 1 -- 172.21.15.32:0/644620697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161400f600 con 0x7f162c1011c0 2024-04-16T05:32:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.511+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1a0880 con 0x7f162c1011c0 2024-04-16T05:32:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.512+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c1a0ae0 con 0x7f162c1011c0 2024-04-16T05:32:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.513+0000 7f1629ffb640 1 -- 172.21.15.32:0/644620697 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f161401a070 con 0x7f162c1011c0 2024-04-16T05:32:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.514+0000 7f1629ffb640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f16180a3c30 0x7f16180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.514+0000 7f162bfff640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f16180a3c30 0x7f16180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.514+0000 7f1629ffb640 1 -- 172.21.15.32:0/644620697 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f16140cfa20 con 0x7f162c1011c0 2024-04-16T05:32:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.515+0000 7f162bfff640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f16180a3c30 0x7f16180a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f161c037040 tx=0x7f161c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.519+0000 7f1629ffb640 1 -- 172.21.15.32:0/644620697 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1614009de0 con 0x7f162c1011c0 2024-04-16T05:32:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:09 smithi134 ceph-mon[36153]: Deploying daemon mds.e on smithi082 2024-04-16T05:32:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:09 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.32:6836/3911772856,v1:172.21.15.32:6837/3911772856] up:boot 2024-04-16T05:32:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:09 smithi134 ceph-mon[36153]: fsmap 4 up:standby 2024-04-16T05:32:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:32:09.638 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:09 smithi082 ceph-mon[36195]: Deploying daemon mds.e on smithi082 2024-04-16T05:32:09.638 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:09 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.32:6836/3911772856,v1:172.21.15.32:6837/3911772856] up:boot 2024-04-16T05:32:09.639 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:09 smithi082 ceph-mon[36195]: fsmap 4 up:standby 2024-04-16T05:32:09.639 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:32:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.745+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162c1a0dd0 con 0x7f162c1011c0 2024-04-16T05:32:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.745+0000 7f1629ffb640 1 -- 172.21.15.32:0/644620697 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f161409c800 con 0x7f162c1011c0 2024-04-16T05:32:09.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:09.744 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T05:32:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2024-04-16T05:32:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.747+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f16180a3c30 msgr2=0x7f16180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.747+0000 7f1633088640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f16180a3c30 0x7f16180a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f161c037040 tx=0x7f161c005c50 comp rx=0 tx=0).stop 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.747+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 msgr2=0x7f162c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.747+0000 7f1633088640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 0x7f162c198420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f161400d950 tx=0x7f161400de10 comp rx=0 tx=0).stop 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.748+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 shutdown_connections 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.748+0000 7f1633088640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f16180a3c30 0x7f16180a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.748+0000 7f1633088640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162c102570 0x7f162c19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.748+0000 7f1633088640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f162c101b80 0x7f162c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.748+0000 7f1633088640 1 --2- 172.21.15.32:0/644620697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c1011c0 0x7f162c198420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.748+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 >> 172.21.15.32:0/644620697 conn(0x7f162c0fbc70 msgr2=0x7f162c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.748+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 shutdown_connections 2024-04-16T05:32:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:09.748+0000 7f1633088640 1 -- 172.21.15.32:0/644620697 wait complete. 2024-04-16T05:32:10.392 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:10 smithi082 ceph-mon[36195]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:10.392 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:10 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/644620697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:10.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:10 smithi032 ceph-mon[32351]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:10 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/644620697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:10 smithi134 ceph-mon[36153]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:10 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/644620697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:10.648 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:32:10 smithi082 podman[65777]: 2024-04-16 05:32:10.62814224 +0000 UTC m=+0.349274600 container start 263d0625fb4237602cf43c56faf5847b8bf69589e8dc7f8323b4de07c6c3bd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD) 2024-04-16T05:32:10.900 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:32:10 smithi082 ceph-mds[65796]: -- 172.21.15.82:0/3894887454 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec7384ba40 con 0x55ec7381e800 2024-04-16T05:32:10.900 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:32:10 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e[65792]: starting mds.e at 2024-04-16T05:32:10.900 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:32:10 smithi082 bash[65777]: 263d0625fb4237602cf43c56faf5847b8bf69589e8dc7f8323b4de07c6c3bd1a 2024-04-16T05:32:10.900 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:32:10 smithi082 systemd[1]: Started Ceph mds.e for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:11.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 -- 172.21.15.32:0/2631731132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0102070 msgr2=0x7f09a0102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 --2- 172.21.15.32:0/2631731132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0102070 0x7f09a0102490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0990009900 tx=0x7f099002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 -- 172.21.15.32:0/2631731132 shutdown_connections 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 --2- 172.21.15.32:0/2631731132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a0103c20 0x7f09a010a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 --2- 172.21.15.32:0/2631731132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09a0103260 0x7f09a01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 --2- 172.21.15.32:0/2631731132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0102070 0x7f09a0102490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 -- 172.21.15.32:0/2631731132 >> 172.21.15.32:0/2631731132 conn(0x7f09a00fd600 msgr2=0x7f09a00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 -- 172.21.15.32:0/2631731132 shutdown_connections 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.880+0000 7f09a698f640 1 -- 172.21.15.32:0/2631731132 wait complete. 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a698f640 1 Processor -- start 2024-04-16T05:32:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a698f640 1 -- start start 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a698f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a0102070 0x7f09a0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a698f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09a0103260 0x7f09a0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a698f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0103c20 0x7f09a019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a698f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09a010dd00 con 0x7f09a0103c20 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a698f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f09a010db80 con 0x7f09a0103260 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a698f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a010de80 con 0x7f09a0102070 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a4f2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0103c20 0x7f09a019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a4f2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0103c20 0x7f09a019f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53476/0 (socket says 172.21.15.32:53476) 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f09a4f2c640 1 -- 172.21.15.32:0/1366491659 learned_addr learned my addr 172.21.15.32:0/1366491659 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.881+0000 7f099ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a0102070 0x7f09a0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f0997fff640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09a0103260 0x7f09a0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f09a4f2c640 1 -- 172.21.15.32:0/1366491659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a0102070 msgr2=0x7f09a0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f09a4f2c640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a0102070 0x7f09a0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f09a4f2c640 1 -- 172.21.15.32:0/1366491659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09a0103260 msgr2=0x7f09a0198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f09a4f2c640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09a0103260 0x7f09a0198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f09a4f2c640 1 -- 172.21.15.32:0/1366491659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a019ff70 con 0x7f09a0103c20 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f0997fff640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09a0103260 0x7f09a0198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f09a4f2c640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0103c20 0x7f09a019f900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f099800eaa0 tx=0x7f099800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f099dffb640 1 -- 172.21.15.32:0/1366491659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099800cdb0 con 0x7f09a0103c20 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f099dffb640 1 -- 172.21.15.32:0/1366491659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998005020 con 0x7f09a0103c20 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a01a0260 con 0x7f09a0103c20 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f099dffb640 1 -- 172.21.15.32:0/1366491659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980116b0 con 0x7f09a0103c20 2024-04-16T05:32:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.882+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a01a07b0 con 0x7f09a0103c20 2024-04-16T05:32:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.883+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a0109390 con 0x7f09a0103c20 2024-04-16T05:32:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.884+0000 7f099dffb640 1 -- 172.21.15.32:0/1366491659 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0998005560 con 0x7f09a0103c20 2024-04-16T05:32:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.885+0000 7f099dffb640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f098c0a3c30 0x7f098c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.885+0000 7f099ffff640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f098c0a3c30 0x7f098c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.886+0000 7f099dffb640 1 -- 172.21.15.32:0/1366491659 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f09980d0500 con 0x7f09a0103c20 2024-04-16T05:32:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.886+0000 7f099ffff640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f098c0a3c30 0x7f098c0a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0990000c00 tx=0x7f0990005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:11.889+0000 7f099dffb640 1 -- 172.21.15.32:0/1366491659 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0998016030 con 0x7f09a0103c20 2024-04-16T05:32:12.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.113+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09a002d620 con 0x7f09a0103c20 2024-04-16T05:32:12.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.114+0000 7f099dffb640 1 -- 172.21.15.32:0/1366491659 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f099809d3e0 con 0x7f09a0103c20 2024-04-16T05:32:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:12.112 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-16T05:32:12.113 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-16T05:32:12.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f098c0a3c30 msgr2=0x7f098c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:12.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f098c0a3c30 0x7f098c0a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0990000c00 tx=0x7f0990005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:12.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0103c20 msgr2=0x7f09a019f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0103c20 0x7f09a019f900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f099800eaa0 tx=0x7f099800ef60 comp rx=0 tx=0).stop 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 shutdown_connections 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f098c0a3c30 0x7f098c0a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09a0103c20 0x7f09a019f900 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09a0103260 0x7f09a0198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 --2- 172.21.15.32:0/1366491659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a0102070 0x7f09a0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 >> 172.21.15.32:0/1366491659 conn(0x7f09a00fd600 msgr2=0x7f09a0106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 shutdown_connections 2024-04-16T05:32:12.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:12.116+0000 7f09a698f640 1 -- 172.21.15.32:0/1366491659 wait complete. 2024-04-16T05:32:12.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:12.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:12.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:12.125 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: Deploying daemon mds.f on smithi134 2024-04-16T05:32:12.125 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6834/1367430466,v1:172.21.15.82:6835/1367430466] up:boot 2024-04-16T05:32:12.125 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: fsmap 5 up:standby 2024-04-16T05:32:12.125 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:11 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:32:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: Deploying daemon mds.f on smithi134 2024-04-16T05:32:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6834/1367430466,v1:172.21.15.82:6835/1367430466] up:boot 2024-04-16T05:32:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: fsmap 5 up:standby 2024-04-16T05:32:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:11 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: Deploying daemon mds.f on smithi134 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6834/1367430466,v1:172.21.15.82:6835/1367430466] up:boot 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: fsmap 5 up:standby 2024-04-16T05:32:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:11 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:32:13.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:12 smithi134 ceph-mon[36153]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:12 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1366491659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:13.102 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:12 smithi134 systemd[1]: Starting Ceph mds.f for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:13.102 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:13 smithi134 podman[67263]: 2024-04-16 05:32:13.00622895 +0000 UTC m=+0.018294103 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:13.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:12 smithi082 ceph-mon[36195]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:13.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:12 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1366491659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:13.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:12 smithi032 ceph-mon[32351]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:13.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:12 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1366491659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:13.371 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:13 smithi134 podman[67263]: 2024-04-16 05:32:13.111818311 +0000 UTC m=+0.123883457 container create 3ad03c21da9a9a42a898515f24413752a7e1a24b79551f2d8fc24640c296a470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T05:32:13.371 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:13 smithi134 podman[67263]: 2024-04-16 05:32:13.372896922 +0000 UTC m=+0.384962074 container init 3ad03c21da9a9a42a898515f24413752a7e1a24b79551f2d8fc24640c296a470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:32:13.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:13 smithi134 podman[67263]: 2024-04-16 05:32:13.375894393 +0000 UTC m=+0.387959546 container start 3ad03c21da9a9a42a898515f24413752a7e1a24b79551f2d8fc24640c296a470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:32:13.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:13 smithi134 ceph-mds[67282]: -- 172.21.15.134:0/155661602 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55feb7953a40 con 0x55feb7926800 2024-04-16T05:32:13.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f[67278]: starting mds.f at 2024-04-16T05:32:13.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:13 smithi134 bash[67263]: 3ad03c21da9a9a42a898515f24413752a7e1a24b79551f2d8fc24640c296a470 2024-04-16T05:32:13.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:32:13 smithi134 systemd[1]: Started Ceph mds.f for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:14.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 -- 172.21.15.32:0/1701559340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 msgr2=0x7f93b8074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 --2- 172.21.15.32:0/1701559340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 0x7f93b8074490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009930 tx=0x7f93b402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 -- 172.21.15.32:0/1701559340 shutdown_connections 2024-04-16T05:32:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 --2- 172.21.15.32:0/1701559340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b810e620 0x7f93b8110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 --2- 172.21.15.32:0/1701559340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93b8074a60 0x7f93b8074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 --2- 172.21.15.32:0/1701559340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 0x7f93b8074490 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 -- 172.21.15.32:0/1701559340 >> 172.21.15.32:0/1701559340 conn(0x7f93b80fd5e0 msgr2=0x7f93b80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 -- 172.21.15.32:0/1701559340 shutdown_connections 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.246+0000 7f93bf460640 1 -- 172.21.15.32:0/1701559340 wait complete. 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bf460640 1 Processor -- start 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bf460640 1 -- start start 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bf460640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b8074a60 0x7f93b819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bf460640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 0x7f93b819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bf460640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93b810e620 0x7f93b81a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bf460640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81142d0 con 0x7f93b8076040 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bf460640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f93b8114150 con 0x7f93b8074a60 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bf460640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93b8114450 con 0x7f93b810e620 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bc9fb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 0x7f93b819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bc9fb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 0x7f93b819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53486/0 (socket says 172.21.15.32:53486) 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bc9fb640 1 -- 172.21.15.32:0/3915845056 learned_addr learned my addr 172.21.15.32:0/3915845056 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:14.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bd9fd640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93b810e620 0x7f93b81a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bd1fc640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b8074a60 0x7f93b819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bc9fb640 1 -- 172.21.15.32:0/3915845056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93b810e620 msgr2=0x7f93b81a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bc9fb640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93b810e620 0x7f93b81a5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bc9fb640 1 -- 172.21.15.32:0/3915845056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b8074a60 msgr2=0x7f93b819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.247+0000 7f93bc9fb640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b8074a60 0x7f93b819e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.248+0000 7f93bc9fb640 1 -- 172.21.15.32:0/3915845056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b81a65f0 con 0x7f93b8076040 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.248+0000 7f93bc9fb640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 0x7f93b819ef10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f93a800e9f0 tx=0x7f93a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.248+0000 7f93ae7fc640 1 -- 172.21.15.32:0/3915845056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a800cd80 con 0x7f93b8076040 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.248+0000 7f93ae7fc640 1 -- 172.21.15.32:0/3915845056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a8005500 con 0x7f93b8076040 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.248+0000 7f93ae7fc640 1 -- 172.21.15.32:0/3915845056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8010690 con 0x7f93b8076040 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.248+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b81a68e0 con 0x7f93b8076040 2024-04-16T05:32:14.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.248+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b81a6e30 con 0x7f93b8076040 2024-04-16T05:32:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.249+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b81047f0 con 0x7f93b8076040 2024-04-16T05:32:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.250+0000 7f93ae7fc640 1 -- 172.21.15.32:0/3915845056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f93a80046c0 con 0x7f93b8076040 2024-04-16T05:32:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.251+0000 7f93ae7fc640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f93a40a3c30 0x7f93a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:14.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.251+0000 7f93bd1fc640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f93a40a3c30 0x7f93a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:14.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.251+0000 7f93ae7fc640 1 -- 172.21.15.32:0/3915845056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f93a80d02e0 con 0x7f93b8076040 2024-04-16T05:32:14.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.252+0000 7f93bd1fc640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f93a40a3c30 0x7f93a40a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000c00 tx=0x7f93b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.256+0000 7f93ae7fc640 1 -- 172.21.15.32:0/3915845056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93a809d160 con 0x7f93b8076040 2024-04-16T05:32:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.480+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93b802d830 con 0x7f93b8076040 2024-04-16T05:32:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.481+0000 7f93ae7fc640 1 -- 172.21.15.32:0/3915845056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f93a809ca60 con 0x7f93b8076040 2024-04-16T05:32:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:14.480 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-16T05:32:14.480 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 7 2024-04-16T05:32:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f93a40a3c30 msgr2=0x7f93a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f93a40a3c30 0x7f93a40a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000c00 tx=0x7f93b4005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 msgr2=0x7f93b819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 0x7f93b819ef10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f93a800e9f0 tx=0x7f93a800edf0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 shutdown_connections 2024-04-16T05:32:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f93a40a3c30 0x7f93a40a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93b810e620 0x7f93b81a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b8076040 0x7f93b819ef10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 --2- 172.21.15.32:0/3915845056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b8074a60 0x7f93b819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:14.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 >> 172.21.15.32:0/3915845056 conn(0x7f93b80fd5e0 msgr2=0x7f93b81019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:14.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 shutdown_connections 2024-04-16T05:32:14.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:14.483+0000 7f93bf460640 1 -- 172.21.15.32:0/3915845056 wait complete. 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: Deploying daemon mds.c on smithi134 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.134:6834/1431936644,v1:172.21.15.134:6835/1431936644] up:boot 2024-04-16T05:32:14.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: fsmap 6 up:standby 2024-04-16T05:32:14.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:32:14.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:14 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3915845056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:14.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:14.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: Deploying daemon mds.c on smithi134 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.134:6834/1431936644,v1:172.21.15.134:6835/1431936644] up:boot 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: fsmap 6 up:standby 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:32:14.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:14 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3915845056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:14.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: Deploying daemon mds.c on smithi134 2024-04-16T05:32:14.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.134:6834/1431936644,v1:172.21.15.134:6835/1431936644] up:boot 2024-04-16T05:32:14.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: fsmap 6 up:standby 2024-04-16T05:32:14.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:32:14.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:14 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3915845056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:16.384 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:32:16 smithi134 systemd[1]: Started Ceph mds.c for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:16.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 -- 172.21.15.32:0/1914193639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 msgr2=0x7f1af0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 --2- 172.21.15.32:0/1914193639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 0x7f1af0102510 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009900 tx=0x7f1ae002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 -- 172.21.15.32:0/1914193639 shutdown_connections 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 --2- 172.21.15.32:0/1914193639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af0102a50 0x7f1af0109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 --2- 172.21.15.32:0/1914193639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 0x7f1af0102510 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 --2- 172.21.15.32:0/1914193639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0100f40 0x7f1af0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 -- 172.21.15.32:0/1914193639 >> 172.21.15.32:0/1914193639 conn(0x7f1af00fc4b0 msgr2=0x7f1af00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 -- 172.21.15.32:0/1914193639 shutdown_connections 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 -- 172.21.15.32:0/1914193639 wait complete. 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 Processor -- start 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.614+0000 7f1af7524640 1 -- start start 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af7524640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af0100f40 0x7f1af0069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af7524640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 0x7f1af019d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af7524640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0102a50 0x7f1af019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af7524640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1af010cbf0 con 0x7f1af0102090 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af7524640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1af010ca70 con 0x7f1af0100f40 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af7524640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1af010cd70 con 0x7f1af0102a50 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 0x7f1af019d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 0x7f1af019d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53500/0 (socket says 172.21.15.32:53500) 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 -- 172.21.15.32:0/3783444453 learned_addr learned my addr 172.21.15.32:0/3783444453 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af52c0640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af0100f40 0x7f1af0069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af5ac1640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0102a50 0x7f1af019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 -- 172.21.15.32:0/3783444453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0102a50 msgr2=0x7f1af019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0102a50 0x7f1af019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 -- 172.21.15.32:0/3783444453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af0100f40 msgr2=0x7f1af0069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af0100f40 0x7f1af0069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 -- 172.21.15.32:0/3783444453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af01a0020 con 0x7f1af0102090 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af5ac1640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0102a50 0x7f1af019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af4abf640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 0x7f1af019d0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0000c00 tx=0x7f1ae0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1ae67fc640 1 -- 172.21.15.32:0/3783444453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0041070 con 0x7f1af0102090 2024-04-16T05:32:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af01a0250 con 0x7f1af0102090 2024-04-16T05:32:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1ae67fc640 1 -- 172.21.15.32:0/3783444453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae00044d0 con 0x7f1af0102090 2024-04-16T05:32:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.615+0000 7f1ae67fc640 1 -- 172.21.15.32:0/3783444453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0034440 con 0x7f1af0102090 2024-04-16T05:32:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.616+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af01a0740 con 0x7f1af0102090 2024-04-16T05:32:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.616+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af0108250 con 0x7f1af0102090 2024-04-16T05:32:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.619+0000 7f1ae67fc640 1 -- 172.21.15.32:0/3783444453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ae0044070 con 0x7f1af0102090 2024-04-16T05:32:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.619+0000 7f1ae67fc640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.619+0000 7f1af52c0640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.619+0000 7f1ae67fc640 1 -- 172.21.15.32:0/3783444453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1ae00f1dc0 con 0x7f1af0102090 2024-04-16T05:32:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.620+0000 7f1af52c0640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0058c0 tx=0x7f1aec00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.623+0000 7f1ae67fc640 1 -- 172.21.15.32:0/3783444453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ae00beba0 con 0x7f1af0102090 2024-04-16T05:32:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.848+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1af002d620 con 0x7f1af0102090 2024-04-16T05:32:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.849+0000 7f1ae67fc640 1 -- 172.21.15.32:0/3783444453 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f1af002d620 con 0x7f1af0102090 2024-04-16T05:32:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:16.847 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-16T05:32:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-16T05:32:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1adc0a3bb0 msgr2=0x7f1adc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0058c0 tx=0x7f1aec00a250 comp rx=0 tx=0).stop 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 msgr2=0x7f1af019d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 0x7f1af019d0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0000c00 tx=0x7f1ae0005d50 comp rx=0 tx=0).stop 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 shutdown_connections 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0102a50 0x7f1af019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0102090 0x7f1af019d0f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 --2- 172.21.15.32:0/3783444453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af0100f40 0x7f1af0069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 >> 172.21.15.32:0/3783444453 conn(0x7f1af00fc4b0 msgr2=0x7f1af0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 shutdown_connections 2024-04-16T05:32:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:16.851+0000 7f1af7524640 1 -- 172.21.15.32:0/3783444453 wait complete. 2024-04-16T05:32:16.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:16 smithi032 ceph-mon[32351]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:16.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:16 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:16 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:16 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:16 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:16.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:16 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:16.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:16 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:16.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:16 smithi134 ceph-mon[36153]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:16.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:16 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:16 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:16 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:16 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:16.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:16 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:16.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:16 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:16.889 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:16 smithi082 ceph-mon[36195]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:16.889 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:16 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.889 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:16 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.889 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:16 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:16.889 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:16 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:16.889 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:16 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:16.889 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:16 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:17.792 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:17 smithi032 ceph-mon[32351]: Deploying daemon mds.h on smithi082 2024-04-16T05:32:17.792 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:17 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.134:6836/2091760007,v1:172.21.15.134:6837/2091760007] up:boot 2024-04-16T05:32:17.792 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:17 smithi032 ceph-mon[32351]: fsmap 7 up:standby 2024-04-16T05:32:17.792 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:17 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:32:17.792 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:17 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3783444453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:17.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:17 smithi134 ceph-mon[36153]: Deploying daemon mds.h on smithi082 2024-04-16T05:32:17.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:17 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.134:6836/2091760007,v1:172.21.15.134:6837/2091760007] up:boot 2024-04-16T05:32:17.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:17 smithi134 ceph-mon[36153]: fsmap 7 up:standby 2024-04-16T05:32:17.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:17 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:32:17.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:17 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3783444453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:17.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:17 smithi082 ceph-mon[36195]: Deploying daemon mds.h on smithi082 2024-04-16T05:32:17.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:17 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.134:6836/2091760007,v1:172.21.15.134:6837/2091760007] up:boot 2024-04-16T05:32:17.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:17 smithi082 ceph-mon[36195]: fsmap 7 up:standby 2024-04-16T05:32:17.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:17 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:32:17.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:17 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3783444453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:18.527 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:18 smithi082 systemd[1]: Starting Ceph mds.h for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:18.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:18 smithi032 ceph-mon[32351]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:18.813 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:18 smithi082 ceph-mon[36195]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:18.814 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:18 smithi082 podman[66098]: 2024-04-16 05:32:18.555373523 +0000 UTC m=+0.017258656 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:18.814 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:18 smithi082 podman[66098]: 2024-04-16 05:32:18.678229218 +0000 UTC m=+0.140114346 container create d9882b97cc8a0900336ae7d45fa7ab602c75e6b83a070b67f7b720309bd9155b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5) 2024-04-16T05:32:18.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:18.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:18 smithi134 ceph-mon[36153]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:18.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 -- 172.21.15.32:0/3533579734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c102ae0 msgr2=0x7f597c104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:18.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 --2- 172.21.15.32:0/3533579734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c102ae0 0x7f597c104ed0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f596c009900 tx=0x7f596c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 -- 172.21.15.32:0/3533579734 shutdown_connections 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 --2- 172.21.15.32:0/3533579734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f597c107d40 0x7f597c10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 --2- 172.21.15.32:0/3533579734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f597c105410 0x7f597c107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 --2- 172.21.15.32:0/3533579734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c102ae0 0x7f597c104ed0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 -- 172.21.15.32:0/3533579734 >> 172.21.15.32:0/3533579734 conn(0x7f597c0fc4f0 msgr2=0x7f597c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 -- 172.21.15.32:0/3533579734 shutdown_connections 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.981+0000 7f598218f640 1 -- 172.21.15.32:0/3533579734 wait complete. 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f598218f640 1 Processor -- start 2024-04-16T05:32:18.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f598218f640 1 -- start start 2024-04-16T05:32:18.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f598218f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f597c102ae0 0x7f597c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:18.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f598218f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c105410 0x7f597c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:18.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f598218f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f597c107d40 0x7f597c19f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:18.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f598218f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10dcf0 con 0x7f597c105410 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f598218f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10db70 con 0x7f597c107d40 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f598218f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10de70 con 0x7f597c102ae0 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f597affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c105410 0x7f597c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.982+0000 7f597affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c105410 0x7f597c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50086/0 (socket says 172.21.15.32:50086) 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597affd640 1 -- 172.21.15.32:0/3983274200 learned_addr learned my addr 172.21.15.32:0/3983274200 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597bfff640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f597c107d40 0x7f597c19f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597b7fe640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f597c102ae0 0x7f597c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597affd640 1 -- 172.21.15.32:0/3983274200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f597c102ae0 msgr2=0x7f597c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597affd640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f597c102ae0 0x7f597c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597affd640 1 -- 172.21.15.32:0/3983274200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f597c107d40 msgr2=0x7f597c19f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597affd640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f597c107d40 0x7f597c19f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597affd640 1 -- 172.21.15.32:0/3983274200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c19fff0 con 0x7f597c105410 2024-04-16T05:32:18.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f597affd640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c105410 0x7f597c198910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f597000c970 tx=0x7f597000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:18.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f5978ff9640 1 -- 172.21.15.32:0/3983274200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970007bb0 con 0x7f597c105410 2024-04-16T05:32:18.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f5978ff9640 1 -- 172.21.15.32:0/3983274200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5970007d50 con 0x7f597c105410 2024-04-16T05:32:18.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c1a02e0 con 0x7f597c105410 2024-04-16T05:32:18.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f5978ff9640 1 -- 172.21.15.32:0/3983274200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597000f450 con 0x7f597c105410 2024-04-16T05:32:18.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.983+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c1a0830 con 0x7f597c105410 2024-04-16T05:32:18.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.984+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c0fdfe0 con 0x7f597c105410 2024-04-16T05:32:18.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.985+0000 7f5978ff9640 1 -- 172.21.15.32:0/3983274200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f59700046c0 con 0x7f597c105410 2024-04-16T05:32:18.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.986+0000 7f5978ff9640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59640a3bb0 0x7f59640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:18.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.986+0000 7f597b7fe640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59640a3bb0 0x7f59640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:18.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.986+0000 7f5978ff9640 1 -- 172.21.15.32:0/3983274200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f59700cfff0 con 0x7f597c105410 2024-04-16T05:32:18.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.987+0000 7f597b7fe640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59640a3bb0 0x7f59640a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f596c006010 tx=0x7f596c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:18.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:18.991+0000 7f5978ff9640 1 -- 172.21.15.32:0/3983274200 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5970014030 con 0x7f597c105410 2024-04-16T05:32:19.150 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:18 smithi082 podman[66098]: 2024-04-16 05:32:18.947831122 +0000 UTC m=+0.409716264 container init d9882b97cc8a0900336ae7d45fa7ab602c75e6b83a070b67f7b720309bd9155b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-16T05:32:19.150 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:18 smithi082 podman[66098]: 2024-04-16 05:32:18.950598174 +0000 UTC m=+0.412483308 container start d9882b97cc8a0900336ae7d45fa7ab602c75e6b83a070b67f7b720309bd9155b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T05:32:19.150 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:18 smithi082 ceph-mds[66117]: -- 172.21.15.82:0/2805267722 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648feffda40 con 0x5648fefd0c00 2024-04-16T05:32:19.150 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:18 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h[66113]: starting mds.h at 2024-04-16T05:32:19.150 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:19 smithi082 bash[66098]: d9882b97cc8a0900336ae7d45fa7ab602c75e6b83a070b67f7b720309bd9155b 2024-04-16T05:32:19.150 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:32:19 smithi082 systemd[1]: Started Ceph mds.h for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.221+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f597c02d620 con 0x7f597c105410 2024-04-16T05:32:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.222+0000 7f5978ff9640 1 -- 172.21.15.32:0/3983274200 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f597009cdf0 con 0x7f597c105410 2024-04-16T05:32:19.221 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:19.221 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-16T05:32:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59640a3bb0 msgr2=0x7f59640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59640a3bb0 0x7f59640a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f596c006010 tx=0x7f596c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c105410 msgr2=0x7f597c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c105410 0x7f597c198910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f597000c970 tx=0x7f597000ce30 comp rx=0 tx=0).stop 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 shutdown_connections 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59640a3bb0 0x7f59640a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f597c107d40 0x7f597c19f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c105410 0x7f597c198910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 --2- 172.21.15.32:0/3983274200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f597c102ae0 0x7f597c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 >> 172.21.15.32:0/3983274200 conn(0x7f597c0fc4f0 msgr2=0x7f597c103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 shutdown_connections 2024-04-16T05:32:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:19.224+0000 7f598218f640 1 -- 172.21.15.32:0/3983274200 wait complete. 2024-04-16T05:32:20.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: Deploying daemon mds.i on smithi134 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3983274200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] up:boot 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: fsmap 8 up:standby 2024-04-16T05:32:20.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:20 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:32:20.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:20.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:20.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:20.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:20.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: Deploying daemon mds.i on smithi134 2024-04-16T05:32:20.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3983274200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:20.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] up:boot 2024-04-16T05:32:20.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: fsmap 8 up:standby 2024-04-16T05:32:20.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:20 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: Deploying daemon mds.i on smithi134 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3983274200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] up:boot 2024-04-16T05:32:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: fsmap 8 up:standby 2024-04-16T05:32:20.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:20 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:32:21.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- 172.21.15.32:0/3208604306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34102b60 msgr2=0x7f4f34104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/3208604306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34102b60 0x7f4f34104f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009900 tx=0x7f4f2402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- 172.21.15.32:0/3208604306 shutdown_connections 2024-04-16T05:32:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/3208604306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f34107dc0 0x7f4f3410a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/3208604306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f34105490 0x7f4f34107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/3208604306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34102b60 0x7f4f34104f50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- 172.21.15.32:0/3208604306 >> 172.21.15.32:0/3208604306 conn(0x7f4f340fc4b0 msgr2=0x7f4f340fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- 172.21.15.32:0/3208604306 shutdown_connections 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- 172.21.15.32:0/3208604306 wait complete. 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 Processor -- start 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- start start 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f34102b60 0x7f4f34198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34105490 0x7f4f341988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f34107dc0 0x7f4f3419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3410dcf0 con 0x7f4f34105490 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3410db70 con 0x7f4f34102b60 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.355+0000 7f4f3aba8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3410de70 con 0x7f4f34107dc0 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34105490 0x7f4f341988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34105490 0x7f4f341988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50112/0 (socket says 172.21.15.32:50112) 2024-04-16T05:32:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 -- 172.21.15.32:0/209210339 learned_addr learned my addr 172.21.15.32:0/209210339 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f39145640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f34107dc0 0x7f4f3419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f38944640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f34102b60 0x7f4f34198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 -- 172.21.15.32:0/209210339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f34107dc0 msgr2=0x7f4f3419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f34107dc0 0x7f4f3419f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 -- 172.21.15.32:0/209210339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f34102b60 msgr2=0x7f4f34198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f34102b60 0x7f4f34198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 -- 172.21.15.32:0/209210339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3419ff90 con 0x7f4f34105490 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f38944640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f34102b60 0x7f4f34198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f33fff640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34105490 0x7f4f341988b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e9c0 tx=0x7f4f2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f31ffb640 1 -- 172.21.15.32:0/209210339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800cd80 con 0x7f4f34105490 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f31ffb640 1 -- 172.21.15.32:0/209210339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f28005500 con 0x7f4f34105490 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f341a0280 con 0x7f4f34105490 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f31ffb640 1 -- 172.21.15.32:0/209210339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28010690 con 0x7f4f34105490 2024-04-16T05:32:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.356+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f341a07d0 con 0x7f4f34105490 2024-04-16T05:32:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.357+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f340fdfa0 con 0x7f4f34105490 2024-04-16T05:32:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.358+0000 7f4f31ffb640 1 -- 172.21.15.32:0/209210339 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4f280046c0 con 0x7f4f34105490 2024-04-16T05:32:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.359+0000 7f4f31ffb640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4f1c0a3bb0 0x7f4f1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.359+0000 7f4f31ffb640 1 -- 172.21.15.32:0/209210339 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4f280d0410 con 0x7f4f34105490 2024-04-16T05:32:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.359+0000 7f4f38944640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4f1c0a3bb0 0x7f4f1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.360+0000 7f4f38944640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4f1c0a3bb0 0x7f4f1c0a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f24006010 tx=0x7f4f2402c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.364+0000 7f4f31ffb640 1 -- 172.21.15.32:0/209210339 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f2809d1f0 con 0x7f4f34105490 2024-04-16T05:32:21.384 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 systemd[1]: Starting Ceph mds.i for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.592+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f3402d620 con 0x7f4f34105490 2024-04-16T05:32:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.593+0000 7f4f31ffb640 1 -- 172.21.15.32:0/209210339 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f4f2809de20 con 0x7f4f34105490 2024-04-16T05:32:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:21.592 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-16T05:32:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2024-04-16T05:32:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4f1c0a3bb0 msgr2=0x7f4f1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4f1c0a3bb0 0x7f4f1c0a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f24006010 tx=0x7f4f2402c8a0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34105490 msgr2=0x7f4f341988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34105490 0x7f4f341988b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e9c0 tx=0x7f4f2800ee80 comp rx=0 tx=0).stop 2024-04-16T05:32:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 shutdown_connections 2024-04-16T05:32:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4f1c0a3bb0 0x7f4f1c0a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f34107dc0 0x7f4f3419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f34105490 0x7f4f341988b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 --2- 172.21.15.32:0/209210339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f34102b60 0x7f4f34198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 >> 172.21.15.32:0/209210339 conn(0x7f4f340fc4b0 msgr2=0x7f4f341034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 shutdown_connections 2024-04-16T05:32:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:21.595+0000 7f4f3aba8640 1 -- 172.21.15.32:0/209210339 wait complete. 2024-04-16T05:32:21.762 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 podman[67883]: 2024-04-16 05:32:21.414905522 +0000 UTC m=+0.017896242 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:21.762 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 podman[67883]: 2024-04-16 05:32:21.520055605 +0000 UTC m=+0.123046344 container create b310c02cc59f188c4fbd5ab0d607a05df2d9284ff84f7ffa2fcfe40711748d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:32:21.762 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 podman[67883]: 2024-04-16 05:32:21.764269571 +0000 UTC m=+0.367260309 container init b310c02cc59f188c4fbd5ab0d607a05df2d9284ff84f7ffa2fcfe40711748d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-16T05:32:22.103 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 podman[67883]: 2024-04-16 05:32:21.76696342 +0000 UTC m=+0.369954146 container start b310c02cc59f188c4fbd5ab0d607a05df2d9284ff84f7ffa2fcfe40711748d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:32:22.104 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 ceph-mds[67903]: -- 172.21.15.134:0/473669895 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec4d7c9a40 con 0x55ec4d79c800 2024-04-16T05:32:22.104 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i[67899]: starting mds.i at 2024-04-16T05:32:22.104 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 bash[67883]: b310c02cc59f188c4fbd5ab0d607a05df2d9284ff84f7ffa2fcfe40711748d56 2024-04-16T05:32:22.104 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:32:21 smithi134 systemd[1]: Started Ceph mds.i for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:22.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:22 smithi134 ceph-mon[36153]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:22.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:22 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/209210339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:22.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:22.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:22.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:22 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:22.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:22 smithi082 ceph-mon[36195]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:22.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:22 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/209210339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:22.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:22.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:22.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:22 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:22.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:22 smithi032 ceph-mon[32351]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:22.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:22 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/209210339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:22 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:23 smithi032 ceph-mon[32351]: Deploying daemon mds.g on smithi032 2024-04-16T05:32:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:23 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] up:boot 2024-04-16T05:32:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:23 smithi032 ceph-mon[32351]: fsmap 9 up:standby 2024-04-16T05:32:23.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:23 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:32:23.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:23 smithi134 ceph-mon[36153]: Deploying daemon mds.g on smithi032 2024-04-16T05:32:23.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:23 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] up:boot 2024-04-16T05:32:23.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:23 smithi134 ceph-mon[36153]: fsmap 9 up:standby 2024-04-16T05:32:23.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:23 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:32:23.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:23 smithi082 ceph-mon[36195]: Deploying daemon mds.g on smithi032 2024-04-16T05:32:23.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:23 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] up:boot 2024-04-16T05:32:23.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:23 smithi082 ceph-mon[36195]: fsmap 9 up:standby 2024-04-16T05:32:23.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:23 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:32:23.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 -- 172.21.15.32:0/2289358879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04107da0 msgr2=0x7f0b0410a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2289358879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04107da0 0x7f0b0410a190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009ca0 tx=0x7f0afc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 -- 172.21.15.32:0/2289358879 shutdown_connections 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2289358879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04107da0 0x7f0b0410a190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2289358879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b04105470 0x7f0b04107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2289358879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b04102b40 0x7f0b04104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 -- 172.21.15.32:0/2289358879 >> 172.21.15.32:0/2289358879 conn(0x7f0b040fc4f0 msgr2=0x7f0b040fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 -- 172.21.15.32:0/2289358879 shutdown_connections 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.721+0000 7f0b090a5640 1 -- 172.21.15.32:0/2289358879 wait complete. 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b090a5640 1 Processor -- start 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b090a5640 1 -- start start 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b090a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b04102b40 0x7f0b041024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b090a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04105470 0x7f0b04100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b090a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b04107da0 0x7f0b04101030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b090a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410dcf0 con 0x7f0b04105470 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b090a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410db70 con 0x7f0b04102b40 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b090a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410de70 con 0x7f0b04107da0 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04105470 0x7f0b04100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04105470 0x7f0b04100af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50126/0 (socket says 172.21.15.32:50126) 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 -- 172.21.15.32:0/2881949931 learned_addr learned my addr 172.21.15.32:0/2881949931 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b0359e640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b04107da0 0x7f0b04101030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b02d9d640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b04102b40 0x7f0b041024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 -- 172.21.15.32:0/2881949931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b04107da0 msgr2=0x7f0b04101030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b04107da0 0x7f0b04101030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 -- 172.21.15.32:0/2881949931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b04102b40 msgr2=0x7f0b041024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b04102b40 0x7f0b041024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 -- 172.21.15.32:0/2881949931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b041018c0 con 0x7f0b04105470 2024-04-16T05:32:23.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0b02d9d640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b04102b40 0x7f0b041024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:23.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.722+0000 7f0afbfff640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04105470 0x7f0b04100af0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00c970 tx=0x7f0aec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:23.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.723+0000 7f0b00d99640 1 -- 172.21.15.32:0/2881949931 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec007bb0 con 0x7f0b04105470 2024-04-16T05:32:23.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.723+0000 7f0b00d99640 1 -- 172.21.15.32:0/2881949931 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aec007d50 con 0x7f0b04105470 2024-04-16T05:32:23.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.723+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b041a2450 con 0x7f0b04105470 2024-04-16T05:32:23.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.723+0000 7f0b00d99640 1 -- 172.21.15.32:0/2881949931 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec00f450 con 0x7f0b04105470 2024-04-16T05:32:23.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.723+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b041a29a0 con 0x7f0b04105470 2024-04-16T05:32:23.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.724+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b040fdfe0 con 0x7f0b04105470 2024-04-16T05:32:23.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.725+0000 7f0b00d99640 1 -- 172.21.15.32:0/2881949931 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0aec0046c0 con 0x7f0b04105470 2024-04-16T05:32:23.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.725+0000 7f0b00d99640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0af00a3bb0 0x7f0af00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:23.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.725+0000 7f0b00d99640 1 -- 172.21.15.32:0/2881949931 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0aec0cffc0 con 0x7f0b04105470 2024-04-16T05:32:23.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.725+0000 7f0b02d9d640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0af00a3bb0 0x7f0af00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:23.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.726+0000 7f0b02d9d640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0af00a3bb0 0x7f0af00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0af40058c0 tx=0x7f0af4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:23.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.730+0000 7f0b00d99640 1 -- 172.21.15.32:0/2881949931 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0aec014030 con 0x7f0b04105470 2024-04-16T05:32:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.966+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b0402d620 con 0x7f0b04105470 2024-04-16T05:32:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.967+0000 7f0b00d99640 1 -- 172.21.15.32:0/2881949931 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f0aec09ce40 con 0x7f0b04105470 2024-04-16T05:32:23.966 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:23.966 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-16T05:32:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0af00a3bb0 msgr2=0x7f0af00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0af00a3bb0 0x7f0af00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0af40058c0 tx=0x7f0af4005850 comp rx=0 tx=0).stop 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04105470 msgr2=0x7f0b04100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04105470 0x7f0b04100af0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00c970 tx=0x7f0aec00ce30 comp rx=0 tx=0).stop 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 shutdown_connections 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0af00a3bb0 0x7f0af00a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b04107da0 0x7f0b04101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b04105470 0x7f0b04100af0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 --2- 172.21.15.32:0/2881949931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b04102b40 0x7f0b041024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 >> 172.21.15.32:0/2881949931 conn(0x7f0b040fc4f0 msgr2=0x7f0b04103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 shutdown_connections 2024-04-16T05:32:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:23.969+0000 7f0b090a5640 1 -- 172.21.15.32:0/2881949931 wait complete. 2024-04-16T05:32:24.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:24 smithi032 ceph-mon[32351]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:24.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:24 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2881949931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:24.233 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 systemd[1]: Starting Ceph mds.g for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:24.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:24 smithi134 ceph-mon[36153]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:24.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:24 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2881949931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:24.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:24 smithi082 ceph-mon[36195]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:24.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:24 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2881949931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:24.694 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 podman[74134]: 2024-04-16 05:32:24.292937578 +0000 UTC m=+0.017659117 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:24.694 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 podman[74134]: 2024-04-16 05:32:24.426651444 +0000 UTC m=+0.151372963 container create 694d2cbe0a4abf6174198d162c08b3800361a6d5f3c760d9ad9397358876857a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-16T05:32:24.694 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 podman[74134]: 2024-04-16 05:32:24.696198821 +0000 UTC m=+0.420920341 container init 694d2cbe0a4abf6174198d162c08b3800361a6d5f3c760d9ad9397358876857a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T05:32:24.983 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 podman[74134]: 2024-04-16 05:32:24.699176868 +0000 UTC m=+0.423898391 container start 694d2cbe0a4abf6174198d162c08b3800361a6d5f3c760d9ad9397358876857a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T05:32:24.983 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 ceph-mds[74153]: -- 172.21.15.32:0/325883016 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560217e3fa40 con 0x560217e12c00 2024-04-16T05:32:24.983 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g[74149]: starting mds.g at 2024-04-16T05:32:24.983 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 bash[74134]: 694d2cbe0a4abf6174198d162c08b3800361a6d5f3c760d9ad9397358876857a 2024-04-16T05:32:24.983 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:32:24 smithi032 systemd[1]: Started Ceph mds.g for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:25.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 -- 172.21.15.32:0/2132337316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 msgr2=0x7ffbc4107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/2132337316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 0x7ffbc4107840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009900 tx=0x7ffbb002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 -- 172.21.15.32:0/2132337316 shutdown_connections 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/2132337316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc4107d80 0x7ffbc410a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/2132337316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 0x7ffbc4107840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/2132337316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc4102b20 0x7ffbc4104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 -- 172.21.15.32:0/2132337316 >> 172.21.15.32:0/2132337316 conn(0x7ffbc40fc4b0 msgr2=0x7ffbc40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 -- 172.21.15.32:0/2132337316 shutdown_connections 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 -- 172.21.15.32:0/2132337316 wait complete. 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 Processor -- start 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.097+0000 7ffbcacc8640 1 -- start start 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbcacc8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc4102b20 0x7ffbc4102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbcacc8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 0x7ffbc4100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbcacc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc4107d80 0x7ffbc4101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbcacc8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc410dd10 con 0x7ffbc4105450 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbcacc8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc410db90 con 0x7ffbc4102b20 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbcacc8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc410de90 con 0x7ffbc4107d80 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 0x7ffbc4100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 0x7ffbc4100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50182/0 (socket says 172.21.15.32:50182) 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 -- 172.21.15.32:0/849190466 learned_addr learned my addr 172.21.15.32:0/849190466 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc9265640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc4107d80 0x7ffbc4101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc8a64640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc4102b20 0x7ffbc4102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 -- 172.21.15.32:0/849190466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc4107d80 msgr2=0x7ffbc4101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc4107d80 0x7ffbc4101010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 -- 172.21.15.32:0/849190466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc4102b20 msgr2=0x7ffbc4102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc4102b20 0x7ffbc4102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 -- 172.21.15.32:0/849190466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc41018a0 con 0x7ffbc4105450 2024-04-16T05:32:26.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc8a64640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc4102b20 0x7ffbc4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc3fff640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 0x7ffbc4100ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005ab0 tx=0x7ffbb0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc1ffb640 1 -- 172.21.15.32:0/849190466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0041070 con 0x7ffbc4105450 2024-04-16T05:32:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc1ffb640 1 -- 172.21.15.32:0/849190466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb00054e0 con 0x7ffbc4105450 2024-04-16T05:32:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc41a24a0 con 0x7ffbc4105450 2024-04-16T05:32:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.098+0000 7ffbc1ffb640 1 -- 172.21.15.32:0/849190466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0034440 con 0x7ffbc4105450 2024-04-16T05:32:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.099+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc41a2a40 con 0x7ffbc4105450 2024-04-16T05:32:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.100+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc40fdfa0 con 0x7ffbc4105450 2024-04-16T05:32:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.101+0000 7ffbc1ffb640 1 -- 172.21.15.32:0/849190466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffbb0044070 con 0x7ffbc4105450 2024-04-16T05:32:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.102+0000 7ffbc1ffb640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ffbb40a3c30 0x7ffbb40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.102+0000 7ffbc8a64640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ffbb40a3c30 0x7ffbb40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.102+0000 7ffbc1ffb640 1 -- 172.21.15.32:0/849190466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ffbb00f1cb0 con 0x7ffbc4105450 2024-04-16T05:32:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.102+0000 7ffbc8a64640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ffbb40a3c30 0x7ffbb40a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009b40 tx=0x7ffbb8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.106+0000 7ffbc1ffb640 1 -- 172.21.15.32:0/849190466 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbb00bea90 con 0x7ffbc4105450 2024-04-16T05:32:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:26.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:26.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:26.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: Deploying daemon mds.a on smithi032 2024-04-16T05:32:26.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:26.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.32:6838/1699429500,v1:172.21.15.32:6839/1699429500] up:boot 2024-04-16T05:32:26.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: fsmap 10 up:standby 2024-04-16T05:32:26.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:25 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: Deploying daemon mds.a on smithi032 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.32:6838/1699429500,v1:172.21.15.32:6839/1699429500] up:boot 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: fsmap 10 up:standby 2024-04-16T05:32:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:25 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:32:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: Deploying daemon mds.a on smithi032 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.32:6838/1699429500,v1:172.21.15.32:6839/1699429500] up:boot 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: fsmap 10 up:standby 2024-04-16T05:32:26.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:25 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:32:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.350+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbc402d620 con 0x7ffbc4105450 2024-04-16T05:32:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.351+0000 7ffbc1ffb640 1 -- 172.21.15.32:0/849190466 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7ffbb00be390 con 0x7ffbc4105450 2024-04-16T05:32:26.349 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:26.349 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-16T05:32:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 11 2024-04-16T05:32:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ffbb40a3c30 msgr2=0x7ffbb40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ffbb40a3c30 0x7ffbb40a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009b40 tx=0x7ffbb8005e50 comp rx=0 tx=0).stop 2024-04-16T05:32:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 msgr2=0x7ffbc4100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 0x7ffbc4100ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005ab0 tx=0x7ffbb0005dc0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 shutdown_connections 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ffbb40a3c30 0x7ffbb40a60f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc4107d80 0x7ffbc4101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc4105450 0x7ffbc4100ad0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 --2- 172.21.15.32:0/849190466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc4102b20 0x7ffbc4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 >> 172.21.15.32:0/849190466 conn(0x7ffbc40fc4b0 msgr2=0x7ffbc4103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 shutdown_connections 2024-04-16T05:32:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:26.353+0000 7ffbcacc8640 1 -- 172.21.15.32:0/849190466 wait complete. 2024-04-16T05:32:27.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:26 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/849190466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:27.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:26 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/849190466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:27.233 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 systemd[1]: Starting Ceph mds.a for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:32:27.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:26 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/849190466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:27.500 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 podman[74492]: 2024-04-16 05:32:27.145944189 +0000 UTC m=+0.018182073 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:32:27.500 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 podman[74492]: 2024-04-16 05:32:27.257516985 +0000 UTC m=+0.129754858 container create de09e00922c0c0bffd10d2a2f6db48b9a75cf836e21e1d50c75ae188037aac89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-16T05:32:27.500 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 podman[74492]: 2024-04-16 05:32:27.502484698 +0000 UTC m=+0.374722571 container init de09e00922c0c0bffd10d2a2f6db48b9a75cf836e21e1d50c75ae188037aac89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-16T05:32:27.982 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 podman[74492]: 2024-04-16 05:32:27.505649438 +0000 UTC m=+0.377887320 container start de09e00922c0c0bffd10d2a2f6db48b9a75cf836e21e1d50c75ae188037aac89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-16T05:32:27.983 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 ceph-mds[74512]: -- 172.21.15.32:0/3551080646 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a70289a40 con 0x562a7025c800 2024-04-16T05:32:27.983 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a[74508]: starting mds.a at 2024-04-16T05:32:27.983 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 bash[74492]: de09e00922c0c0bffd10d2a2f6db48b9a75cf836e21e1d50c75ae188037aac89 2024-04-16T05:32:27.983 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:32:27 smithi032 systemd[1]: Started Ceph mds.a for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:28.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- 172.21.15.32:0/1479205635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754102b70 msgr2=0x7fc754109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/1479205635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754102b70 0x7fc754109400 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009900 tx=0x7fc74c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- 172.21.15.32:0/1479205635 shutdown_connections 2024-04-16T05:32:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/1479205635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754102b70 0x7fc754109400 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/1479205635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7541021b0 0x7fc754102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/1479205635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc754100fc0 0x7fc7541013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- 172.21.15.32:0/1479205635 >> 172.21.15.32:0/1479205635 conn(0x7fc7540fc550 msgr2=0x7fc7540fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- 172.21.15.32:0/1479205635 shutdown_connections 2024-04-16T05:32:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- 172.21.15.32:0/1479205635 wait complete. 2024-04-16T05:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 Processor -- start 2024-04-16T05:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- start start 2024-04-16T05:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754100fc0 0x7fc75406b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7541021b0 0x7fc75419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc754102b70 0x7fc75419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc75410cbe0 con 0x7fc754100fc0 2024-04-16T05:32:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc75410ca60 con 0x7fc754102b70 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.487+0000 7fc75bfd4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc75410cd60 con 0x7fc7541021b0 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754100fc0 0x7fc75406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754100fc0 0x7fc75406b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58584/0 (socket says 172.21.15.32:58584) 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 -- 172.21.15.32:0/3969428551 learned_addr learned my addr 172.21.15.32:0/3969428551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc75956f640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7541021b0 0x7fc75419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 -- 172.21.15.32:0/3969428551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7541021b0 msgr2=0x7fc75419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7541021b0 0x7fc75419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 -- 172.21.15.32:0/3969428551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc754102b70 msgr2=0x7fc75419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc754102b70 0x7fc75419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 -- 172.21.15.32:0/3969428551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75419fed0 con 0x7fc754100fc0 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc759d70640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754100fc0 0x7fc75406b210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc75000ccb0 tx=0x7fc75000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc74affd640 1 -- 172.21.15.32:0/3969428551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750003b70 con 0x7fc754100fc0 2024-04-16T05:32:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7541a01c0 con 0x7fc754100fc0 2024-04-16T05:32:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc74affd640 1 -- 172.21.15.32:0/3969428551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75000ae50 con 0x7fc754100fc0 2024-04-16T05:32:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc74affd640 1 -- 172.21.15.32:0/3969428551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75000f810 con 0x7fc754100fc0 2024-04-16T05:32:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.488+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7541a0710 con 0x7fc754100fc0 2024-04-16T05:32:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.489+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc754108210 con 0x7fc754100fc0 2024-04-16T05:32:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.490+0000 7fc74affd640 1 -- 172.21.15.32:0/3969428551 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc75000f9b0 con 0x7fc754100fc0 2024-04-16T05:32:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.491+0000 7fc74affd640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7400a3bb0 0x7fc7400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.491+0000 7fc75956f640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7400a3bb0 0x7fc7400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.491+0000 7fc74affd640 1 -- 172.21.15.32:0/3969428551 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc7500d0560 con 0x7fc754100fc0 2024-04-16T05:32:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.492+0000 7fc75956f640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7400a3bb0 0x7fc7400a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc744005fd0 tx=0x7fc744005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.495+0000 7fc74affd640 1 -- 172.21.15.32:0/3969428551 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc750014070 con 0x7fc754100fc0 2024-04-16T05:32:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.723+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc75402d860 con 0x7fc754100fc0 2024-04-16T05:32:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.724+0000 7fc74affd640 1 -- 172.21.15.32:0/3969428551 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7fc75009d3e0 con 0x7fc754100fc0 2024-04-16T05:32:28.723 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:28.723 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-16T05:32:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2024-04-16T05:32:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7400a3bb0 msgr2=0x7fc7400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7400a3bb0 0x7fc7400a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc744005fd0 tx=0x7fc744005e20 comp rx=0 tx=0).stop 2024-04-16T05:32:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754100fc0 msgr2=0x7fc75406b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754100fc0 0x7fc75406b210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc75000ccb0 tx=0x7fc75000a620 comp rx=0 tx=0).stop 2024-04-16T05:32:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 shutdown_connections 2024-04-16T05:32:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc7400a3bb0 0x7fc7400a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc754102b70 0x7fc75419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7541021b0 0x7fc75419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 --2- 172.21.15.32:0/3969428551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc754100fc0 0x7fc75406b210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:28.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 >> 172.21.15.32:0/3969428551 conn(0x7fc7540fc550 msgr2=0x7fc7541053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:28.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 shutdown_connections 2024-04-16T05:32:28.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:28.726+0000 7fc75bfd4640 1 -- 172.21.15.32:0/3969428551 wait complete. 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: Deploying daemon mds.b on smithi082 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.32:6840/2946647099,v1:172.21.15.32:6841/2946647099] up:boot 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: fsmap 11 up:standby 2024-04-16T05:32:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:28 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:32:28.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:28.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: Deploying daemon mds.b on smithi082 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.32:6840/2946647099,v1:172.21.15.32:6841/2946647099] up:boot 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: fsmap 11 up:standby 2024-04-16T05:32:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:28 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: Deploying daemon mds.b on smithi082 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.32:6840/2946647099,v1:172.21.15.32:6841/2946647099] up:boot 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: fsmap 11 up:standby 2024-04-16T05:32:29.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:28 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:32:29.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:29 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3969428551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:29.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:29 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3969428551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:30.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:29 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3969428551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:30.650 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:32:30 smithi082 systemd[1]: Started Ceph mds.b for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:32:30.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:30.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.856+0000 7f7af7ba8640 1 -- 172.21.15.32:0/11267540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af01053f0 msgr2=0x7f7af01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:30.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.856+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/11267540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af01053f0 0x7f7af01077e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009900 tx=0x7f7ae402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:30.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.856+0000 7f7af7ba8640 1 -- 172.21.15.32:0/11267540 shutdown_connections 2024-04-16T05:32:30.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.856+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/11267540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af0107d20 0x7f7af010a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.856+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/11267540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af01053f0 0x7f7af01077e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.856+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/11267540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af0102ac0 0x7f7af0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.856+0000 7f7af7ba8640 1 -- 172.21.15.32:0/11267540 >> 172.21.15.32:0/11267540 conn(0x7f7af00fc4b0 msgr2=0x7f7af00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 -- 172.21.15.32:0/11267540 shutdown_connections 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 -- 172.21.15.32:0/11267540 wait complete. 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 Processor -- start 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 -- start start 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af0102ac0 0x7f7af0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af01053f0 0x7f7af01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af0107d20 0x7f7af019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7af010dd20 con 0x7f7af0107d20 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7af010dba0 con 0x7f7af0102ac0 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af7ba8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7af010dea0 con 0x7f7af01053f0 2024-04-16T05:32:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.857+0000 7f7af6145640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af0107d20 0x7f7af019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af6145640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af0107d20 0x7f7af019f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58608/0 (socket says 172.21.15.32:58608) 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af6145640 1 -- 172.21.15.32:0/3931094965 learned_addr learned my addr 172.21.15.32:0/3931094965 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af5143640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af01053f0 0x7f7af01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af5944640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af0102ac0 0x7f7af0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af6145640 1 -- 172.21.15.32:0/3931094965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af01053f0 msgr2=0x7f7af01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af6145640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af01053f0 0x7f7af01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af6145640 1 -- 172.21.15.32:0/3931094965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af0102ac0 msgr2=0x7f7af0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af6145640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af0102ac0 0x7f7af0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af6145640 1 -- 172.21.15.32:0/3931094965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af019ff90 con 0x7f7af0107d20 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af5944640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af0102ac0 0x7f7af0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af6145640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af0107d20 0x7f7af019f920 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000eb10 tx=0x7f7ae000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7aeeffd640 1 -- 172.21.15.32:0/3931094965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0004b20 con 0x7f7af0107d20 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7aeeffd640 1 -- 172.21.15.32:0/3931094965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae0004cc0 con 0x7f7af0107d20 2024-04-16T05:32:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af01a0280 con 0x7f7af0107d20 2024-04-16T05:32:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7aeeffd640 1 -- 172.21.15.32:0/3931094965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0010760 con 0x7f7af0107d20 2024-04-16T05:32:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.858+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af01a07a0 con 0x7f7af0107d20 2024-04-16T05:32:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.859+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af00fdfa0 con 0x7f7af0107d20 2024-04-16T05:32:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.860+0000 7f7aeeffd640 1 -- 172.21.15.32:0/3931094965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7ae0005000 con 0x7f7af0107d20 2024-04-16T05:32:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.861+0000 7f7aeeffd640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ad80a3bb0 0x7f7ad80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.861+0000 7f7af5944640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ad80a3bb0 0x7f7ad80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.861+0000 7f7aeeffd640 1 -- 172.21.15.32:0/3931094965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7ae00d0a40 con 0x7f7af0107d20 2024-04-16T05:32:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.862+0000 7f7af5944640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ad80a3bb0 0x7f7ad80a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80058c0 tx=0x7f7ae8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:30.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:30.865+0000 7f7aeeffd640 1 -- 172.21.15.32:0/3931094965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ae009d8c0 con 0x7f7af0107d20 2024-04-16T05:32:30.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:30 smithi032 ceph-mon[32351]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.094+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7af002d620 con 0x7f7af0107d20 2024-04-16T05:32:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.095+0000 7f7aeeffd640 1 -- 172.21.15.32:0/3931094965 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7f7ae009d1c0 con 0x7f7af0107d20 2024-04-16T05:32:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:31.094 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-16T05:32:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-16T05:32:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ad80a3bb0 msgr2=0x7f7ad80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ad80a3bb0 0x7f7ad80a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80058c0 tx=0x7f7ae8005850 comp rx=0 tx=0).stop 2024-04-16T05:32:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af0107d20 msgr2=0x7f7af019f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af0107d20 0x7f7af019f920 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000eb10 tx=0x7f7ae000c2d0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 shutdown_connections 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ad80a3bb0 0x7f7ad80a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af0107d20 0x7f7af019f920 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af01053f0 0x7f7af01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 --2- 172.21.15.32:0/3931094965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af0102ac0 0x7f7af0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.097+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 >> 172.21.15.32:0/3931094965 conn(0x7f7af00fc4b0 msgr2=0x7f7af01034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.098+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 shutdown_connections 2024-04-16T05:32:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.098+0000 7f7af7ba8640 1 -- 172.21.15.32:0/3931094965 wait complete. 2024-04-16T05:32:31.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:31.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:30 smithi134 ceph-mon[36153]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:31.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:30 smithi082 ceph-mon[36195]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 -- 172.21.15.32:0/784062520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102a50 msgr2=0x7f4350109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 --2- 172.21.15.32:0/784062520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102a50 0x7f4350109370 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4348009ca0 tx=0x7f434802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 -- 172.21.15.32:0/784062520 shutdown_connections 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 --2- 172.21.15.32:0/784062520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102a50 0x7f4350109370 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 --2- 172.21.15.32:0/784062520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4350102090 0x7f4350102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 --2- 172.21.15.32:0/784062520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4350100f40 0x7f4350101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 -- 172.21.15.32:0/784062520 >> 172.21.15.32:0/784062520 conn(0x7f43500fc4b0 msgr2=0x7f43500fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 -- 172.21.15.32:0/784062520 shutdown_connections 2024-04-16T05:32:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.215+0000 7f4358014640 1 -- 172.21.15.32:0/784062520 wait complete. 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4358014640 1 Processor -- start 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4358014640 1 -- start start 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4358014640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4350100f40 0x7f43501971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4358014640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102090 0x7f4350197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4358014640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4350102a50 0x7f435019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4358014640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f435010cbf0 con 0x7f4350102090 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4358014640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f435010ca70 con 0x7f4350100f40 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4358014640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f435010cd70 con 0x7f4350102a50 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f43555af640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102090 0x7f4350197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f43555af640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102090 0x7f4350197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58632/0 (socket says 172.21.15.32:58632) 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f43555af640 1 -- 172.21.15.32:0/601472281 learned_addr learned my addr 172.21.15.32:0/601472281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f43565b1640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4350102a50 0x7f435019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.216+0000 7f4355db0640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4350100f40 0x7f43501971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f43555af640 1 -- 172.21.15.32:0/601472281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4350102a50 msgr2=0x7f435019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f43555af640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4350102a50 0x7f435019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f43555af640 1 -- 172.21.15.32:0/601472281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4350100f40 msgr2=0x7f43501971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f43555af640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4350100f40 0x7f43501971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f43555af640 1 -- 172.21.15.32:0/601472281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435019edf0 con 0x7f4350102090 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f43555af640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102090 0x7f4350197710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f434400ead0 tx=0x7f434400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f4342ffd640 1 -- 172.21.15.32:0/601472281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434401c070 con 0x7f4350102090 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43500691f0 con 0x7f4350102090 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f4342ffd640 1 -- 172.21.15.32:0/601472281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434400bbd0 con 0x7f4350102090 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f4342ffd640 1 -- 172.21.15.32:0/601472281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344003de0 con 0x7f4350102090 2024-04-16T05:32:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.217+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4350069740 con 0x7f4350102090 2024-04-16T05:32:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.218+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4350068e20 con 0x7f4350102090 2024-04-16T05:32:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.219+0000 7f4342ffd640 1 -- 172.21.15.32:0/601472281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f434401f070 con 0x7f4350102090 2024-04-16T05:32:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.220+0000 7f4342ffd640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f433c0a3bb0 0x7f433c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.220+0000 7f4342ffd640 1 -- 172.21.15.32:0/601472281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f43440d1d70 con 0x7f4350102090 2024-04-16T05:32:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.220+0000 7f4355db0640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f433c0a3bb0 0x7f433c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.220+0000 7f4355db0640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f433c0a3bb0 0x7f433c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f434c009bd0 tx=0x7f434c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.224+0000 7f4342ffd640 1 -- 172.21.15.32:0/601472281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4344026070 con 0x7f4350102090 2024-04-16T05:32:31.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.416+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f435019efc0 con 0x7f4350102090 2024-04-16T05:32:31.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.417+0000 7f4342ffd640 1 -- 172.21.15.32:0/601472281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18420 (secure 0 0 0) 0x7f434409e5a0 con 0x7f4350102090 2024-04-16T05:32:31.416 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:31.417 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:31:47.151909+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f433c0a3bb0 msgr2=0x7f433c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f433c0a3bb0 0x7f433c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f434c009bd0 tx=0x7f434c005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102090 msgr2=0x7f4350197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102090 0x7f4350197710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f434400ead0 tx=0x7f434400ef90 comp rx=0 tx=0).stop 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 shutdown_connections 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f433c0a3bb0 0x7f433c0a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4350102a50 0x7f435019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4350102090 0x7f4350197710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 --2- 172.21.15.32:0/601472281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4350100f40 0x7f43501971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 >> 172.21.15.32:0/601472281 conn(0x7f43500fc4b0 msgr2=0x7f43501053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 shutdown_connections 2024-04-16T05:32:31.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.420+0000 7f4358014640 1 -- 172.21.15.32:0/601472281 wait complete. 2024-04-16T05:32:31.430 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T05:32:31.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:32:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 -- 172.21.15.32:0/3844693945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf81053f0 msgr2=0x7fadf81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 --2- 172.21.15.32:0/3844693945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf81053f0 0x7fadf81077e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fade0009900 tx=0x7fade002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 -- 172.21.15.32:0/3844693945 shutdown_connections 2024-04-16T05:32:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 --2- 172.21.15.32:0/3844693945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadf8107d20 0x7fadf810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 --2- 172.21.15.32:0/3844693945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf81053f0 0x7fadf81077e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 --2- 172.21.15.32:0/3844693945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fadf8102ac0 0x7fadf8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 -- 172.21.15.32:0/3844693945 >> 172.21.15.32:0/3844693945 conn(0x7fadf80fc4b0 msgr2=0x7fadf80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 -- 172.21.15.32:0/3844693945 shutdown_connections 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 -- 172.21.15.32:0/3844693945 wait complete. 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 Processor -- start 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 -- start start 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.555+0000 7fadfea4a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf8102ac0 0x7fadf8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadfea4a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fadf81053f0 0x7fadf8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadfea4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadf8107d20 0x7fadf8100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadfea4a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810dd20 con 0x7fadf8102ac0 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadfea4a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810dba0 con 0x7fadf81053f0 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadfea4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810dea0 con 0x7fadf8107d20 2024-04-16T05:32:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf8102ac0 0x7fadf8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf8102ac0 0x7fadf8102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58648/0 (socket says 172.21.15.32:58648) 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 -- 172.21.15.32:0/2962963985 learned_addr learned my addr 172.21.15.32:0/2962963985 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadeffff640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fadf81053f0 0x7fadf8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadfcfe7640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadf8107d20 0x7fadf8100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 -- 172.21.15.32:0/2962963985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadf8107d20 msgr2=0x7fadf8100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadf8107d20 0x7fadf8100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 -- 172.21.15.32:0/2962963985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fadf81053f0 msgr2=0x7fadf8100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fadf81053f0 0x7fadf8100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 -- 172.21.15.32:0/2962963985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf8101840 con 0x7fadf8102ac0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf7fff640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf8102ac0 0x7fadf8102430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fade800d950 tx=0x7fade800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf5ffb640 1 -- 172.21.15.32:0/2962963985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8017030 con 0x7fadf8102ac0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf81a24a0 con 0x7fadf8102ac0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf5ffb640 1 -- 172.21.15.32:0/2962963985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade8004020 con 0x7fadf8102ac0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadf5ffb640 1 -- 172.21.15.32:0/2962963985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80047a0 con 0x7fadf8102ac0 2024-04-16T05:32:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.556+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf81a2a10 con 0x7fadf8102ac0 2024-04-16T05:32:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.557+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf80fdfa0 con 0x7fadf8102ac0 2024-04-16T05:32:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.558+0000 7fadf5ffb640 1 -- 172.21.15.32:0/2962963985 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fade80041d0 con 0x7fadf8102ac0 2024-04-16T05:32:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.559+0000 7fadf5ffb640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fade40a3bb0 0x7fade40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.559+0000 7fadeffff640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fade40a3bb0 0x7fade40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.559+0000 7fadf5ffb640 1 -- 172.21.15.32:0/2962963985 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fade80cfa50 con 0x7fadf8102ac0 2024-04-16T05:32:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.560+0000 7fadeffff640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fade40a3bb0 0x7fade40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fade002c170 tx=0x7fade002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.564+0000 7fadf5ffb640 1 -- 172.21.15.32:0/2962963985 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fade809c830 con 0x7fadf8102ac0 2024-04-16T05:32:31.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] up:boot 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: fsmap 12 up:standby 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3931094965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:31.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:31 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/601472281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:31.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.751+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fadf8062c00 con 0x7fadf8102ac0 2024-04-16T05:32:31.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.752+0000 7fadf5ffb640 1 -- 172.21.15.32:0/2962963985 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fade809c130 con 0x7fadf8102ac0 2024-04-16T05:32:31.751 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fade40a3bb0 msgr2=0x7fade40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fade40a3bb0 0x7fade40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fade002c170 tx=0x7fade002df90 comp rx=0 tx=0).stop 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf8102ac0 msgr2=0x7fadf8102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf8102ac0 0x7fadf8102430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fade800d950 tx=0x7fade800de10 comp rx=0 tx=0).stop 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 shutdown_connections 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fade40a3bb0 0x7fade40a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadf8107d20 0x7fadf8100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fadf81053f0 0x7fadf8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 --2- 172.21.15.32:0/2962963985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf8102ac0 0x7fadf8102430 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 >> 172.21.15.32:0/2962963985 conn(0x7fadf80fc4b0 msgr2=0x7fadf8103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:31.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 shutdown_connections 2024-04-16T05:32:31.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.754+0000 7fadfea4a640 1 -- 172.21.15.32:0/2962963985 wait complete. 2024-04-16T05:32:31.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-16T05:32:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- 172.21.15.32:0/616219848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 msgr2=0x7f0050104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 --2- 172.21.15.32:0/616219848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 0x7f0050104f10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f004c009900 tx=0x7f004c030040 comp rx=0 tx=0).stop 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- 172.21.15.32:0/616219848 shutdown_connections 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 --2- 172.21.15.32:0/616219848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0050107d80 0x7f005010a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 --2- 172.21.15.32:0/616219848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0050105450 0x7f0050107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 --2- 172.21.15.32:0/616219848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 0x7f0050104f10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- 172.21.15.32:0/616219848 >> 172.21.15.32:0/616219848 conn(0x7f00500fc4b0 msgr2=0x7f00500fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- 172.21.15.32:0/616219848 shutdown_connections 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- 172.21.15.32:0/616219848 wait complete. 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 Processor -- start 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- start start 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 0x7f00501961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0050105450 0x7f00501966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0050107d80 0x7f005019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f005010bb00 con 0x7f0050102b20 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f005010b980 con 0x7f0050107d80 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.877+0000 7f0057c4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f005010bc80 con 0x7f0050105450 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 0x7f00501961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 0x7f00501961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58666/0 (socket says 172.21.15.32:58666) 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 -- 172.21.15.32:0/2570682280 learned_addr learned my addr 172.21.15.32:0/2570682280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00551e5640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0050105450 0x7f00501966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00561e7640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0050107d80 0x7f005019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 -- 172.21.15.32:0/2570682280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0050105450 msgr2=0x7f00501966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0050105450 0x7f00501966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 -- 172.21.15.32:0/2570682280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0050107d80 msgr2=0x7f005019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0050107d80 0x7f005019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 -- 172.21.15.32:0/2570682280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005019ddc0 con 0x7f0050102b20 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00561e7640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0050107d80 0x7f005019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:31.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f00559e6640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 0x7f00501961a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f004c009230 tx=0x7f004c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f0042ffd640 1 -- 172.21.15.32:0/2570682280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c047020 con 0x7f0050102b20 2024-04-16T05:32:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f0042ffd640 1 -- 172.21.15.32:0/2570682280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c038d00 con 0x7f0050102b20 2024-04-16T05:32:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005019e050 con 0x7f0050102b20 2024-04-16T05:32:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f0042ffd640 1 -- 172.21.15.32:0/2570682280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c0035f0 con 0x7f0050102b20 2024-04-16T05:32:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.878+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005019e5a0 con 0x7f0050102b20 2024-04-16T05:32:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.879+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00500fdfa0 con 0x7f0050102b20 2024-04-16T05:32:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.880+0000 7f0042ffd640 1 -- 172.21.15.32:0/2570682280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f004c04a020 con 0x7f0050102b20 2024-04-16T05:32:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.881+0000 7f0042ffd640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f003c0a3bb0 0x7f003c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.881+0000 7f00551e5640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f003c0a3bb0 0x7f003c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.882+0000 7f0042ffd640 1 -- 172.21.15.32:0/2570682280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f004c0f62a0 con 0x7f0050102b20 2024-04-16T05:32:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.882+0000 7f00551e5640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f003c0a3bb0 0x7f003c0a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00501976b0 tx=0x7f0044005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:31.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:31.886+0000 7f0042ffd640 1 -- 172.21.15.32:0/2570682280 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f004c0c3120 con 0x7f0050102b20 2024-04-16T05:32:31.919 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] up:boot 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: fsmap 12 up:standby 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3931094965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:31.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:31 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/601472281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.074+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0050062c00 con 0x7f0050102b20 2024-04-16T05:32:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.075+0000 7f0042ffd640 1 -- 172.21.15.32:0/2570682280 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f004c0c2a20 con 0x7f0050102b20 2024-04-16T05:32:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:32.074 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-16T05:32:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:32:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-16T05:32:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-16T05:32:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:32:32.074 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f003c0a3bb0 msgr2=0x7f003c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f003c0a3bb0 0x7f003c0a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00501976b0 tx=0x7f0044005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 msgr2=0x7f00501961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 0x7f00501961a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f004c009230 tx=0x7f004c004ab0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 shutdown_connections 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f003c0a3bb0 0x7f003c0a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0050107d80 0x7f005019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0050105450 0x7f00501966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 --2- 172.21.15.32:0/2570682280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0050102b20 0x7f00501961a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 >> 172.21.15.32:0/2570682280 conn(0x7f00500fc4b0 msgr2=0x7f0050103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 shutdown_connections 2024-04-16T05:32:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.077+0000 7f0057c4a640 1 -- 172.21.15.32:0/2570682280 wait complete. 2024-04-16T05:32:32.086 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-16T05:32:32.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-16T05:32:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] up:boot 2024-04-16T05:32:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: fsmap 12 up:standby 2024-04-16T05:32:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:32:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:32.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3931094965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:32.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:31 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/601472281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 -- 172.21.15.32:0/1747549931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058105410 msgr2=0x7ff058107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/1747549931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058105410 0x7ff058107800 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff048009900 tx=0x7ff04802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 -- 172.21.15.32:0/1747549931 shutdown_connections 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/1747549931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff058107d40 0x7ff05810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/1747549931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058105410 0x7ff058107800 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/1747549931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff058102ae0 0x7ff058104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 -- 172.21.15.32:0/1747549931 >> 172.21.15.32:0/1747549931 conn(0x7ff0580fc4f0 msgr2=0x7ff0580fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 -- 172.21.15.32:0/1747549931 shutdown_connections 2024-04-16T05:32:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.206+0000 7ff05f6d9640 1 -- 172.21.15.32:0/1747549931 wait complete. 2024-04-16T05:32:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05f6d9640 1 Processor -- start 2024-04-16T05:32:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05f6d9640 1 -- start start 2024-04-16T05:32:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05f6d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058102ae0 0x7ff0581983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05f6d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff058105410 0x7ff058198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05f6d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff058107d40 0x7ff05819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05f6d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff05810dcf0 con 0x7ff058102ae0 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05f6d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff05810db70 con 0x7ff058105410 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05f6d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff05810de70 con 0x7ff058107d40 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05d475640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058102ae0 0x7ff0581983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05d475640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058102ae0 0x7ff0581983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58690/0 (socket says 172.21.15.32:58690) 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05d475640 1 -- 172.21.15.32:0/330610362 learned_addr learned my addr 172.21.15.32:0/330610362 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05cc74640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff058105410 0x7ff058198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.207+0000 7ff05dc76640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff058107d40 0x7ff05819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff05d475640 1 -- 172.21.15.32:0/330610362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff058107d40 msgr2=0x7ff05819f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff05d475640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff058107d40 0x7ff05819f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff05d475640 1 -- 172.21.15.32:0/330610362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff058105410 msgr2=0x7ff058198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff05d475640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff058105410 0x7ff058198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff05d475640 1 -- 172.21.15.32:0/330610362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0581a0000 con 0x7ff058102ae0 2024-04-16T05:32:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff05d475640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058102ae0 0x7ff0581983e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff05400e9f0 tx=0x7ff05400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff04e7fc640 1 -- 172.21.15.32:0/330610362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05400cd80 con 0x7ff058102ae0 2024-04-16T05:32:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0581a02f0 con 0x7ff058102ae0 2024-04-16T05:32:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff04e7fc640 1 -- 172.21.15.32:0/330610362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff054010910 con 0x7ff058102ae0 2024-04-16T05:32:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff04e7fc640 1 -- 172.21.15.32:0/330610362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540188f0 con 0x7ff058102ae0 2024-04-16T05:32:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.208+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0581a0810 con 0x7ff058102ae0 2024-04-16T05:32:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.209+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0580fdfe0 con 0x7ff058102ae0 2024-04-16T05:32:32.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.210+0000 7ff04e7fc640 1 -- 172.21.15.32:0/330610362 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff054010ac0 con 0x7ff058102ae0 2024-04-16T05:32:32.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.211+0000 7ff04e7fc640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff0440a3bb0 0x7ff0440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.211+0000 7ff04e7fc640 1 -- 172.21.15.32:0/330610362 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff0540d02f0 con 0x7ff058102ae0 2024-04-16T05:32:32.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.211+0000 7ff05cc74640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff0440a3bb0 0x7ff0440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.212+0000 7ff05cc74640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff0440a3bb0 0x7ff0440a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff048005ab0 tx=0x7ff048005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:32.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.216+0000 7ff04e7fc640 1 -- 172.21.15.32:0/330610362 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff05409cb20 con 0x7ff058102ae0 2024-04-16T05:32:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.413+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff058062c00 con 0x7ff058102ae0 2024-04-16T05:32:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.715+0000 7ff04e7fc640 1 -- 172.21.15.32:0/330610362 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7ff0540a0e20 con 0x7ff058102ae0 2024-04-16T05:32:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff0440a3bb0 msgr2=0x7ff0440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff0440a3bb0 0x7ff0440a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff048005ab0 tx=0x7ff048005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058102ae0 msgr2=0x7ff0581983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058102ae0 0x7ff0581983e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff05400e9f0 tx=0x7ff05400eeb0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 shutdown_connections 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff0440a3bb0 0x7ff0440a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff058107d40 0x7ff05819f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff058105410 0x7ff058198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 --2- 172.21.15.32:0/330610362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058102ae0 0x7ff0581983e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 >> 172.21.15.32:0/330610362 conn(0x7ff0580fc4f0 msgr2=0x7ff058104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 shutdown_connections 2024-04-16T05:32:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.718+0000 7ff05f6d9640 1 -- 172.21.15.32:0/330610362 wait complete. 2024-04-16T05:32:32.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-16T05:32:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 -- 172.21.15.32:0/1594017893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c100f40 msgr2=0x7f556c101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1594017893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c100f40 0x7f556c101360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5560009900 tx=0x7f556002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 -- 172.21.15.32:0/1594017893 shutdown_connections 2024-04-16T05:32:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1594017893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f556c102a50 0x7f556c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1594017893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f556c102090 0x7f556c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1594017893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c100f40 0x7f556c101360 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 -- 172.21.15.32:0/1594017893 >> 172.21.15.32:0/1594017893 conn(0x7f556c0fc4b0 msgr2=0x7f556c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 -- 172.21.15.32:0/1594017893 shutdown_connections 2024-04-16T05:32:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.848+0000 7f5572d0e640 1 -- 172.21.15.32:0/1594017893 wait complete. 2024-04-16T05:32:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5572d0e640 1 Processor -- start 2024-04-16T05:32:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5572d0e640 1 -- start start 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5572d0e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f556c100f40 0x7f556c069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5572d0e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f556c102090 0x7f556c19d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5572d0e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c102a50 0x7f556c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5572d0e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f556c10cbd0 con 0x7f556c102a50 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5572d0e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f556c10ca50 con 0x7f556c100f40 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5572d0e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f556c10cd50 con 0x7f556c102090 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f55712ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c102a50 0x7f556c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f55712ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c102a50 0x7f556c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58718/0 (socket says 172.21.15.32:58718) 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f55712ab640 1 -- 172.21.15.32:0/1376268684 learned_addr learned my addr 172.21.15.32:0/1376268684 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f556bfff640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f556c102090 0x7f556c19d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f5570aaa640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f556c100f40 0x7f556c069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f55712ab640 1 -- 172.21.15.32:0/1376268684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f556c102090 msgr2=0x7f556c19d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f55712ab640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f556c102090 0x7f556c19d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f55712ab640 1 -- 172.21.15.32:0/1376268684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f556c100f40 msgr2=0x7f556c069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f55712ab640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f556c100f40 0x7f556c069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.849+0000 7f55712ab640 1 -- 172.21.15.32:0/1376268684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c1a0030 con 0x7f556c102a50 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.850+0000 7f55712ab640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c102a50 0x7f556c19f720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5564009fd0 tx=0x7f556400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.850+0000 7f5569ffb640 1 -- 172.21.15.32:0/1376268684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564017070 con 0x7f556c102a50 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.850+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c1a02c0 con 0x7f556c102a50 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.850+0000 7f5569ffb640 1 -- 172.21.15.32:0/1376268684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5564003e50 con 0x7f556c102a50 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.850+0000 7f5569ffb640 1 -- 172.21.15.32:0/1376268684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55640106c0 con 0x7f556c102a50 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.850+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c1a0810 con 0x7f556c102a50 2024-04-16T05:32:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.851+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556c108270 con 0x7f556c102a50 2024-04-16T05:32:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.852+0000 7f5569ffb640 1 -- 172.21.15.32:0/1376268684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5564003970 con 0x7f556c102a50 2024-04-16T05:32:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.853+0000 7f5569ffb640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f555c0a3bb0 0x7f555c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.853+0000 7f5569ffb640 1 -- 172.21.15.32:0/1376268684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f55640cfd70 con 0x7f556c102a50 2024-04-16T05:32:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.853+0000 7f5570aaa640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f555c0a3bb0 0x7f555c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.853+0000 7f5570aaa640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f555c0a3bb0 0x7f555c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f556002c7b0 tx=0x7f5560005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:32.861+0000 7f5569ffb640 1 -- 172.21.15.32:0/1376268684 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f556409c430 con 0x7f556c102a50 2024-04-16T05:32:32.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:32.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2962963985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:32.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2570682280' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T05:32:32.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/330610362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T05:32:32.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:32.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:32 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.063+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f556c062c00 con 0x7f556c102a50 2024-04-16T05:32:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2962963985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2570682280' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T05:32:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/330610362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T05:32:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:32 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2962963985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2570682280' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/330610362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:32 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.717+0000 7f5569ffb640 1 -- 172.21.15.32:0/1376268684 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f55640a0730 con 0x7f556c102a50 2024-04-16T05:32:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f555c0a3bb0 msgr2=0x7f555c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f555c0a3bb0 0x7f555c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f556002c7b0 tx=0x7f5560005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c102a50 msgr2=0x7f556c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c102a50 0x7f556c19f720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5564009fd0 tx=0x7f556400b260 comp rx=0 tx=0).stop 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 shutdown_connections 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f555c0a3bb0 0x7f555c0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f556c102a50 0x7f556c19f720 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f556c102090 0x7f556c19d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 --2- 172.21.15.32:0/1376268684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f556c100f40 0x7f556c069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 >> 172.21.15.32:0/1376268684 conn(0x7f556c0fc4b0 msgr2=0x7f556c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 shutdown_connections 2024-04-16T05:32:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.721+0000 7f5572d0e640 1 -- 172.21.15.32:0/1376268684 wait complete. 2024-04-16T05:32:33.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-16T05:32:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.854+0000 7f2976715640 1 -- 172.21.15.32:0/2280404640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102a50 msgr2=0x7f2970109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.854+0000 7f2976715640 1 --2- 172.21.15.32:0/2280404640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102a50 0x7f2970109370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2968009ca0 tx=0x7f296802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 -- 172.21.15.32:0/2280404640 shutdown_connections 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 --2- 172.21.15.32:0/2280404640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102a50 0x7f2970109370 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 --2- 172.21.15.32:0/2280404640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2970102090 0x7f2970102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 --2- 172.21.15.32:0/2280404640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2970100f40 0x7f2970101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 -- 172.21.15.32:0/2280404640 >> 172.21.15.32:0/2280404640 conn(0x7f29700fc4b0 msgr2=0x7f29700fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 -- 172.21.15.32:0/2280404640 shutdown_connections 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 -- 172.21.15.32:0/2280404640 wait complete. 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 Processor -- start 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 -- start start 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2970100f40 0x7f2970104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102090 0x7f2970103500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2970102a50 0x7f2970103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f297010cbf0 con 0x7f2970102090 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f297010ca70 con 0x7f2970102a50 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.855+0000 7f2976715640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f297010cd70 con 0x7f2970100f40 2024-04-16T05:32:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102090 0x7f2970103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f296ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2970100f40 0x7f2970104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102090 0x7f2970103500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58742/0 (socket says 172.21.15.32:58742) 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 -- 172.21.15.32:0/81795210 learned_addr learned my addr 172.21.15.32:0/81795210 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2974cb2640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2970102a50 0x7f2970103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 -- 172.21.15.32:0/81795210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2970100f40 msgr2=0x7f2970104ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2970100f40 0x7f2970104ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 -- 172.21.15.32:0/81795210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2970102a50 msgr2=0x7f2970103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2970102a50 0x7f2970103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 -- 172.21.15.32:0/81795210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29701042d0 con 0x7f2970102090 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2974cb2640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2970102a50 0x7f2970103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f296ffff640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2970100f40 0x7f2970104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2967fff640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102090 0x7f2970103500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f295800e9c0 tx=0x7f295800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f296dffb640 1 -- 172.21.15.32:0/81795210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295800cd80 con 0x7f2970102090 2024-04-16T05:32:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29701a24a0 con 0x7f2970102090 2024-04-16T05:32:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f296dffb640 1 -- 172.21.15.32:0/81795210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29580035f0 con 0x7f2970102090 2024-04-16T05:32:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f296dffb640 1 -- 172.21.15.32:0/81795210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958010690 con 0x7f2970102090 2024-04-16T05:32:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.856+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29701a29f0 con 0x7f2970102090 2024-04-16T05:32:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.857+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29701a2d10 con 0x7f2970102090 2024-04-16T05:32:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.860+0000 7f296dffb640 1 -- 172.21.15.32:0/81795210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f29580037a0 con 0x7f2970102090 2024-04-16T05:32:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.860+0000 7f296dffb640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f295c0a3bb0 0x7f295c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.860+0000 7f296ffff640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f295c0a3bb0 0x7f295c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.860+0000 7f296dffb640 1 -- 172.21.15.32:0/81795210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f29580d0700 con 0x7f2970102090 2024-04-16T05:32:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.861+0000 7f296ffff640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f295c0a3bb0 0x7f295c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2960009fc0 tx=0x7f2960005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:33.863+0000 7f296dffb640 1 -- 172.21.15.32:0/81795210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f295809d290 con 0x7f2970102090 2024-04-16T05:32:33.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:33 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/330610362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T05:32:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:33 smithi032 ceph-mon[32351]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T05:32:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:33 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1376268684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T05:32:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:33 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/330610362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T05:32:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:33 smithi134 ceph-mon[36153]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T05:32:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:33 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1376268684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T05:32:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.108+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f29701a3110 con 0x7f2970102090 2024-04-16T05:32:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:33 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/330610362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T05:32:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:33 smithi082 ceph-mon[36195]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T05:32:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:33 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1376268684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T05:32:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:32:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.891+0000 7f296dffb640 1 -- 172.21.15.32:0/81795210 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f295809cb90 con 0x7f2970102090 2024-04-16T05:32:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f295c0a3bb0 msgr2=0x7f295c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f295c0a3bb0 0x7f295c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2960009fc0 tx=0x7f2960005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102090 msgr2=0x7f2970103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102090 0x7f2970103500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f295800e9c0 tx=0x7f295800ee80 comp rx=0 tx=0).stop 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 shutdown_connections 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f295c0a3bb0 0x7f295c0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2970102a50 0x7f2970103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2970102090 0x7f2970103500 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 --2- 172.21.15.32:0/81795210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2970100f40 0x7f2970104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 >> 172.21.15.32:0/81795210 conn(0x7f29700fc4b0 msgr2=0x7f29700fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 shutdown_connections 2024-04-16T05:32:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:34.893+0000 7f2976715640 1 -- 172.21.15.32:0/81795210 wait complete. 2024-04-16T05:32:34.907 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-16T05:32:34.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-16T05:32:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:34 smithi032 ceph-mon[32351]: pgmap v282: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:34 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1376268684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T05:32:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:34 smithi032 ceph-mon[32351]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T05:32:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:34 smithi032 ceph-mon[32351]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:34.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:34 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/81795210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:34.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:34 smithi032 ceph-mon[32351]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T05:32:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/318368156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 msgr2=0x7f3d281036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/318368156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 0x7f3d281036c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009900 tx=0x7f3d1802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/318368156 shutdown_connections 2024-04-16T05:32:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/318368156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d28103c00 0x7f3d2810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/318368156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 0x7f3d281036c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/318368156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28102050 0x7f3d28102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/318368156 >> 172.21.15.32:0/318368156 conn(0x7f3d280fd5e0 msgr2=0x7f3d280ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/318368156 shutdown_connections 2024-04-16T05:32:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/318368156 wait complete. 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.031+0000 7f3d2f6a5640 1 Processor -- start 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2f6a5640 1 -- start start 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2f6a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d28102050 0x7f3d28198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2f6a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 0x7f3d281988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2f6a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28103c00 0x7f3d2819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2f6a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810dce0 con 0x7f3d28103240 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2f6a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810db60 con 0x7f3d28103c00 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2f6a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810de60 con 0x7f3d28102050 2024-04-16T05:32:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2cc40640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 0x7f3d281988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2cc40640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 0x7f3d281988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58756/0 (socket says 172.21.15.32:58756) 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2cc40640 1 -- 172.21.15.32:0/3869295991 learned_addr learned my addr 172.21.15.32:0/3869295991 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2dc42640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28103c00 0x7f3d2819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2d441640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d28102050 0x7f3d28198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2cc40640 1 -- 172.21.15.32:0/3869295991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d28102050 msgr2=0x7f3d28198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2cc40640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d28102050 0x7f3d28198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2cc40640 1 -- 172.21.15.32:0/3869295991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28103c00 msgr2=0x7f3d2819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2cc40640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28103c00 0x7f3d2819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.032+0000 7f3d2cc40640 1 -- 172.21.15.32:0/3869295991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2819ffb0 con 0x7f3d28103240 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.033+0000 7f3d2cc40640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 0x7f3d281988d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009230 tx=0x7f3d18005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.033+0000 7f3d1e7fc640 1 -- 172.21.15.32:0/3869295991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18041070 con 0x7f3d28103240 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.033+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281a0240 con 0x7f3d28103240 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.033+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281a0780 con 0x7f3d28103240 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.033+0000 7f3d1e7fc640 1 -- 172.21.15.32:0/3869295991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d180054e0 con 0x7f3d28103240 2024-04-16T05:32:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.033+0000 7f3d1e7fc640 1 -- 172.21.15.32:0/3869295991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18034440 con 0x7f3d28103240 2024-04-16T05:32:35.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.034+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c0052c0 con 0x7f3d28103240 2024-04-16T05:32:35.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.036+0000 7f3d1e7fc640 1 -- 172.21.15.32:0/3869295991 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3d18044070 con 0x7f3d28103240 2024-04-16T05:32:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.037+0000 7f3d1e7fc640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3d140a3c30 0x7f3d140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.037+0000 7f3d1e7fc640 1 -- 172.21.15.32:0/3869295991 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f3d180f1dd0 con 0x7f3d28103240 2024-04-16T05:32:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.037+0000 7f3d2d441640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3d140a3c30 0x7f3d140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.037+0000 7f3d2d441640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3d140a3c30 0x7f3d140a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005fd0 tx=0x7f3d2400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.042+0000 7f3d1e7fc640 1 -- 172.21.15.32:0/3869295991 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d180be210 con 0x7f3d28103240 2024-04-16T05:32:35.052 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:34 smithi134 ceph-mon[36153]: pgmap v282: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:35.052 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:34 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1376268684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T05:32:35.052 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:34 smithi134 ceph-mon[36153]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T05:32:35.052 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:34 smithi134 ceph-mon[36153]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:35.053 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:34 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/81795210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:35.053 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:34 smithi134 ceph-mon[36153]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T05:32:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:34 smithi082 ceph-mon[36195]: pgmap v282: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:34 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1376268684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T05:32:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:34 smithi082 ceph-mon[36195]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T05:32:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:34 smithi082 ceph-mon[36195]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:34 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/81795210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T05:32:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:34 smithi082 ceph-mon[36195]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T05:32:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.271+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3d0c0055b0 con 0x7f3d28103240 2024-04-16T05:32:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.733+0000 7f3d1e7fc640 1 -- 172.21.15.32:0/3869295991 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f3d180c2510 con 0x7f3d28103240 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3d140a3c30 msgr2=0x7f3d140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3d140a3c30 0x7f3d140a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005fd0 tx=0x7f3d2400b410 comp rx=0 tx=0).stop 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 msgr2=0x7f3d281988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 0x7f3d281988d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009230 tx=0x7f3d18005dd0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 shutdown_connections 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3d140a3c30 0x7f3d140a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28103c00 0x7f3d2819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d28103240 0x7f3d281988d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 --2- 172.21.15.32:0/3869295991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d28102050 0x7f3d28198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 >> 172.21.15.32:0/3869295991 conn(0x7f3d280fd5e0 msgr2=0x7f3d28106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 shutdown_connections 2024-04-16T05:32:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.736+0000 7f3d2f6a5640 1 -- 172.21.15.32:0/3869295991 wait complete. 2024-04-16T05:32:35.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 -- 172.21.15.32:0/3639558025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a54102b40 msgr2=0x7f7a54104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 --2- 172.21.15.32:0/3639558025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a54102b40 0x7f7a54104f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009860 tx=0x7f7a4402bb70 comp rx=0 tx=0).stop 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 -- 172.21.15.32:0/3639558025 shutdown_connections 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 --2- 172.21.15.32:0/3639558025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a54107da0 0x7f7a5410a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 --2- 172.21.15.32:0/3639558025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a54105470 0x7f7a54107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 --2- 172.21.15.32:0/3639558025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a54102b40 0x7f7a54104f30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 -- 172.21.15.32:0/3639558025 >> 172.21.15.32:0/3639558025 conn(0x7f7a540fc4f0 msgr2=0x7f7a540fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 -- 172.21.15.32:0/3639558025 shutdown_connections 2024-04-16T05:32:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.863+0000 7f7a589f4640 1 -- 172.21.15.32:0/3639558025 wait complete. 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a589f4640 1 Processor -- start 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a589f4640 1 -- start start 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a589f4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a54102b40 0x7f7a54198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a589f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a54105470 0x7f7a54198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a589f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a54107da0 0x7f7a5419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a589f4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410dcf0 con 0x7f7a54102b40 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a589f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410db70 con 0x7f7a54107da0 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a589f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410de70 con 0x7f7a54105470 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a5259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a54102b40 0x7f7a54198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a5259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a54102b40 0x7f7a54198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58774/0 (socket says 172.21.15.32:58774) 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a5259c640 1 -- 172.21.15.32:0/129664924 learned_addr learned my addr 172.21.15.32:0/129664924 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a49d9b640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a54105470 0x7f7a54198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a52d9d640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a54107da0 0x7f7a5419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a5259c640 1 -- 172.21.15.32:0/129664924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a54105470 msgr2=0x7f7a54198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a5259c640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a54105470 0x7f7a54198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a5259c640 1 -- 172.21.15.32:0/129664924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a54107da0 msgr2=0x7f7a5419f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a5259c640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a54107da0 0x7f7a5419f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a5259c640 1 -- 172.21.15.32:0/129664924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5419ff60 con 0x7f7a54102b40 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.864+0000 7f7a52d9d640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a54107da0 0x7f7a5419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.865+0000 7f7a5259c640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a54102b40 0x7f7a54198340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a44005940 tx=0x7f7a440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.865+0000 7f7a4bfff640 1 -- 172.21.15.32:0/129664924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44041020 con 0x7f7a54102b40 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.865+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a541a01f0 con 0x7f7a54102b40 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.865+0000 7f7a4bfff640 1 -- 172.21.15.32:0/129664924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a440323f0 con 0x7f7a54102b40 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.865+0000 7f7a4bfff640 1 -- 172.21.15.32:0/129664924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44039040 con 0x7f7a54102b40 2024-04-16T05:32:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.865+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a541a06e0 con 0x7f7a54102b40 2024-04-16T05:32:35.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.866+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a540fdfe0 con 0x7f7a54102b40 2024-04-16T05:32:35.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.867+0000 7f7a4bfff640 1 -- 172.21.15.32:0/129664924 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7a44044020 con 0x7f7a54102b40 2024-04-16T05:32:35.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.868+0000 7f7a4bfff640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a400a3bb0 0x7f7a400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:35.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.868+0000 7f7a49d9b640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a400a3bb0 0x7f7a400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.868+0000 7f7a4bfff640 1 -- 172.21.15.32:0/129664924 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f7a440f2300 con 0x7f7a54102b40 2024-04-16T05:32:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.868+0000 7f7a49d9b640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a400a3bb0 0x7f7a400a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a54199850 tx=0x7f7a3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:35.871+0000 7f7a4bfff640 1 -- 172.21.15.32:0/129664924 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a4404b020 con 0x7f7a54102b40 2024-04-16T05:32:36.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:36.065+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7a5402d620 con 0x7f7a54102b40 2024-04-16T05:32:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T05:32:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/81795210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T05:32:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T05:32:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:35 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3869295991' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/81795210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:36.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:36.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:35 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3869295991' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T05:32:36.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/81795210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:32:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:32:36.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:35 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3869295991' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: pgmap v285: 97 pgs: 17 creating+peering, 5 active+clean, 75 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3869295991' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: Cluster is now healthy 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] up:active 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:37.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:36 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/129664924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: pgmap v285: 97 pgs: 17 creating+peering, 5 active+clean, 75 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3869295991' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: Cluster is now healthy 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] up:active 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:37.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:36 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/129664924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: pgmap v285: 97 pgs: 17 creating+peering, 5 active+clean, 75 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3869295991' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: Cluster is now healthy 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] up:active 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:36 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/129664924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T05:32:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.903+0000 7f7a4bfff640 1 -- 172.21.15.32:0/129664924 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f7a440becb0 con 0x7f7a54102b40 2024-04-16T05:32:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data_ec' created 2024-04-16T05:32:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a400a3bb0 msgr2=0x7f7a400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a400a3bb0 0x7f7a400a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a54199850 tx=0x7f7a3c009260 comp rx=0 tx=0).stop 2024-04-16T05:32:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a54102b40 msgr2=0x7f7a54198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a54102b40 0x7f7a54198340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a44005940 tx=0x7f7a440042f0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 shutdown_connections 2024-04-16T05:32:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7a400a3bb0 0x7f7a400a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a54107da0 0x7f7a5419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a54105470 0x7f7a54198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 --2- 172.21.15.32:0/129664924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a54102b40 0x7f7a54198340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 >> 172.21.15.32:0/129664924 conn(0x7f7a540fc4f0 msgr2=0x7f7a54103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 shutdown_connections 2024-04-16T05:32:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:37.906+0000 7f7a589f4640 1 -- 172.21.15.32:0/129664924 wait complete. 2024-04-16T05:32:37.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-16T05:32:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 -- 172.21.15.32:0/931323132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 msgr2=0x7f8080102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 --2- 172.21.15.32:0/931323132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 0x7f8080102450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f807c009900 tx=0x7f807c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 -- 172.21.15.32:0/931323132 shutdown_connections 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 --2- 172.21.15.32:0/931323132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8080103be0 0x7f808010a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 --2- 172.21.15.32:0/931323132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080103220 0x7f80801036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 --2- 172.21.15.32:0/931323132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 0x7f8080102450 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 -- 172.21.15.32:0/931323132 >> 172.21.15.32:0/931323132 conn(0x7f80800fd5c0 msgr2=0x7f80800ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 -- 172.21.15.32:0/931323132 shutdown_connections 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 -- 172.21.15.32:0/931323132 wait complete. 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 Processor -- start 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.035+0000 7f8087e63640 1 -- start start 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8087e63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 0x7f80801983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8087e63640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8080103220 0x7f80801988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8087e63640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080103be0 0x7f808019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8087e63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f808010dd10 con 0x7f8080102030 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8087e63640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f808010db90 con 0x7f8080103220 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8087e63640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f808010de90 con 0x7f8080103be0 2024-04-16T05:32:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 0x7f80801983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 0x7f80801983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58778/0 (socket says 172.21.15.32:58778) 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 -- 172.21.15.32:0/1907969706 learned_addr learned my addr 172.21.15.32:0/1907969706 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8086400640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080103be0 0x7f808019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 -- 172.21.15.32:0/1907969706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080103be0 msgr2=0x7f808019f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080103be0 0x7f808019f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 -- 172.21.15.32:0/1907969706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8080103220 msgr2=0x7f80801988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f80853fe640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8080103220 0x7f80801988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8080103220 0x7f80801988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 -- 172.21.15.32:0/1907969706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808019ffc0 con 0x7f8080102030 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f80853fe640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8080103220 0x7f80801988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8085bff640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 0x7f80801983a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c00 tx=0x7f807c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8076ffd640 1 -- 172.21.15.32:0/1907969706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c043020 con 0x7f8080102030 2024-04-16T05:32:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8076ffd640 1 -- 172.21.15.32:0/1907969706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f807c004440 con 0x7f8080102030 2024-04-16T05:32:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80801a0250 con 0x7f8080102030 2024-04-16T05:32:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8076ffd640 1 -- 172.21.15.32:0/1907969706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c03e450 con 0x7f8080102030 2024-04-16T05:32:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.036+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a07a0 con 0x7f8080102030 2024-04-16T05:32:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.037+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8080109350 con 0x7f8080102030 2024-04-16T05:32:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.039+0000 7f8076ffd640 1 -- 172.21.15.32:0/1907969706 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f807c046020 con 0x7f8080102030 2024-04-16T05:32:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.039+0000 7f8076ffd640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f806c0a3bb0 0x7f806c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.039+0000 7f80853fe640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f806c0a3bb0 0x7f806c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.039+0000 7f8076ffd640 1 -- 172.21.15.32:0/1907969706 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f807c0f2e00 con 0x7f8080102030 2024-04-16T05:32:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.040+0000 7f80853fe640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f806c0a3bb0 0x7f806c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80801998b0 tx=0x7f8070005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.043+0000 7f8076ffd640 1 -- 172.21.15.32:0/1907969706 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f807c04d020 con 0x7f8080102030 2024-04-16T05:32:38.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.228+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f808002d620 con 0x7f8080102030 2024-04-16T05:32:38.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:37 smithi032 ceph-mon[32351]: pgmap v288: 97 pgs: 17 creating+peering, 36 active+clean, 44 unknown; 578 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-16T05:32:38.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:37 smithi032 ceph-mon[32351]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T05:32:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:37 smithi134 ceph-mon[36153]: pgmap v288: 97 pgs: 17 creating+peering, 36 active+clean, 44 unknown; 578 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-16T05:32:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:37 smithi134 ceph-mon[36153]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T05:32:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:37 smithi082 ceph-mon[36195]: pgmap v288: 97 pgs: 17 creating+peering, 36 active+clean, 44 unknown; 578 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-16T05:32:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:37 smithi082 ceph-mon[36195]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T05:32:38.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.913+0000 7f8076ffd640 1 -- 172.21.15.32:0/1907969706 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f807c0bf5c0 con 0x7f8080102030 2024-04-16T05:32:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 4 allow_ec_overwrites to true 2024-04-16T05:32:38.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f806c0a3bb0 msgr2=0x7f806c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f806c0a3bb0 0x7f806c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80801998b0 tx=0x7f8070005d30 comp rx=0 tx=0).stop 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 msgr2=0x7f80801983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 0x7f80801983a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c00 tx=0x7f807c004040 comp rx=0 tx=0).stop 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 shutdown_connections 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f806c0a3bb0 0x7f806c0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080103be0 0x7f808019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8080103220 0x7f80801988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 --2- 172.21.15.32:0/1907969706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8080102030 0x7f80801983a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 >> 172.21.15.32:0/1907969706 conn(0x7f80800fd5c0 msgr2=0x7f8080106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 shutdown_connections 2024-04-16T05:32:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:38.917+0000 7f8087e63640 1 -- 172.21.15.32:0/1907969706 wait complete. 2024-04-16T05:32:38.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-16T05:32:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 -- 172.21.15.32:0/1056499553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84101bc0 msgr2=0x7f2b84109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 --2- 172.21.15.32:0/1056499553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84101bc0 0x7f2b84109460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009930 tx=0x7f2b7c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 -- 172.21.15.32:0/1056499553 shutdown_connections 2024-04-16T05:32:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 --2- 172.21.15.32:0/1056499553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84101bc0 0x7f2b84109460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 --2- 172.21.15.32:0/1056499553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b84101200 0x7f2b84101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 --2- 172.21.15.32:0/1056499553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84102560 0x7f2b84102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 -- 172.21.15.32:0/1056499553 >> 172.21.15.32:0/1056499553 conn(0x7f2b840fbc30 msgr2=0x7f2b840fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 -- 172.21.15.32:0/1056499553 shutdown_connections 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.055+0000 7f2b898e4640 1 -- 172.21.15.32:0/1056499553 wait complete. 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b898e4640 1 Processor -- start 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b898e4640 1 -- start start 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b898e4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84101200 0x7f2b84197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b898e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b84101bc0 0x7f2b84197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b898e4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84102560 0x7f2b8419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b898e4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410cc30 con 0x7f2b84102560 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b898e4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410cab0 con 0x7f2b84101200 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b898e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410cdb0 con 0x7f2b84101bc0 2024-04-16T05:32:39.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84102560 0x7f2b8419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84102560 0x7f2b8419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57808/0 (socket says 172.21.15.32:57808) 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b837fe640 1 -- 172.21.15.32:0/3540399294 learned_addr learned my addr 172.21.15.32:0/3540399294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b82ffd640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84101200 0x7f2b84197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.056+0000 7f2b827fc640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b84101bc0 0x7f2b84197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b837fe640 1 -- 172.21.15.32:0/3540399294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b84101bc0 msgr2=0x7f2b84197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b837fe640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b84101bc0 0x7f2b84197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b837fe640 1 -- 172.21.15.32:0/3540399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84101200 msgr2=0x7f2b84197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b837fe640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84101200 0x7f2b84197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b837fe640 1 -- 172.21.15.32:0/3540399294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8419ee50 con 0x7f2b84102560 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b837fe640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84102560 0x7f2b8419e7e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009900 tx=0x7f2b7c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b888e2640 1 -- 172.21.15.32:0/3540399294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c041070 con 0x7f2b84102560 2024-04-16T05:32:39.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b888e2640 1 -- 172.21.15.32:0/3540399294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b7c0044c0 con 0x7f2b84102560 2024-04-16T05:32:39.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8419f0e0 con 0x7f2b84102560 2024-04-16T05:32:39.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b888e2640 1 -- 172.21.15.32:0/3540399294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c034440 con 0x7f2b84102560 2024-04-16T05:32:39.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.057+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8419f5d0 con 0x7f2b84102560 2024-04-16T05:32:39.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.058+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b84108340 con 0x7f2b84102560 2024-04-16T05:32:39.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.059+0000 7f2b888e2640 1 -- 172.21.15.32:0/3540399294 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2b7c044070 con 0x7f2b84102560 2024-04-16T05:32:39.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.060+0000 7f2b888e2640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2b6c0a3bb0 0x7f2b6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:39.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.060+0000 7f2b82ffd640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2b6c0a3bb0 0x7f2b6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:39.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.060+0000 7f2b888e2640 1 -- 172.21.15.32:0/3540399294 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f2b7c0f2200 con 0x7f2b84102560 2024-04-16T05:32:39.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.060+0000 7f2b82ffd640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2b6c0a3bb0 0x7f2b6c0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2b8406a3a0 tx=0x7f2b74009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:39.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.064+0000 7f2b888e2640 1 -- 172.21.15.32:0/3540399294 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b7c0be410 con 0x7f2b84102560 2024-04-16T05:32:39.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:38 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/129664924' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T05:32:39.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:38 smithi032 ceph-mon[32351]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T05:32:39.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:38 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1907969706' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T05:32:39.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.295+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f2b84062c00 con 0x7f2b84102560 2024-04-16T05:32:39.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:38 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/129664924' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T05:32:39.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:38 smithi134 ceph-mon[36153]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T05:32:39.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:38 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1907969706' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T05:32:39.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:38 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/129664924' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T05:32:39.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:38 smithi082 ceph-mon[36195]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T05:32:39.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:38 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1907969706' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T05:32:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.914+0000 7f2b888e2640 1 -- 172.21.15.32:0/3540399294 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f2b7c0c2710 con 0x7f2b84102560 2024-04-16T05:32:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:added data pool 4 to fsmap 2024-04-16T05:32:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2b6c0a3bb0 msgr2=0x7f2b6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2b6c0a3bb0 0x7f2b6c0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2b8406a3a0 tx=0x7f2b74009670 comp rx=0 tx=0).stop 2024-04-16T05:32:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84102560 msgr2=0x7f2b8419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84102560 0x7f2b8419e7e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009900 tx=0x7f2b7c0040c0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 shutdown_connections 2024-04-16T05:32:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2b6c0a3bb0 0x7f2b6c0a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b84102560 0x7f2b8419e7e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b84101bc0 0x7f2b84197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 --2- 172.21.15.32:0/3540399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84101200 0x7f2b84197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 >> 172.21.15.32:0/3540399294 conn(0x7f2b840fbc30 msgr2=0x7f2b84105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.917+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 shutdown_connections 2024-04-16T05:32:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:39.918+0000 7f2b898e4640 1 -- 172.21.15.32:0/3540399294 wait complete. 2024-04-16T05:32:39.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 -- 172.21.15.32:0/3810000916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 msgr2=0x7f98e4107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 --2- 172.21.15.32:0/3810000916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 0x7f98e4107800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009900 tx=0x7f98cc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 -- 172.21.15.32:0/3810000916 shutdown_connections 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 --2- 172.21.15.32:0/3810000916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98e4107d40 0x7f98e410a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 --2- 172.21.15.32:0/3810000916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 0x7f98e4107800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 --2- 172.21.15.32:0/3810000916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e4102ae0 0x7f98e4104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 -- 172.21.15.32:0/3810000916 >> 172.21.15.32:0/3810000916 conn(0x7f98e40fc4f0 msgr2=0x7f98e40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 -- 172.21.15.32:0/3810000916 shutdown_connections 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.043+0000 7f98e9500640 1 -- 172.21.15.32:0/3810000916 wait complete. 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e9500640 1 Processor -- start 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e9500640 1 -- start start 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e9500640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e4102ae0 0x7f98e4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e9500640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 0x7f98e4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e9500640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98e4107d40 0x7f98e419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e9500640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f98e410dcf0 con 0x7f98e4105410 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e9500640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98e410db70 con 0x7f98e4102ae0 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e9500640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98e410de70 con 0x7f98e4107d40 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 0x7f98e4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 0x7f98e4198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57836/0 (socket says 172.21.15.32:57836) 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.044+0000 7f98e27fc640 1 -- 172.21.15.32:0/2976182955 learned_addr learned my addr 172.21.15.32:0/2976182955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e37fe640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98e4107d40 0x7f98e419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e2ffd640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e4102ae0 0x7f98e4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e27fc640 1 -- 172.21.15.32:0/2976182955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98e4107d40 msgr2=0x7f98e419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e27fc640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98e4107d40 0x7f98e419f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e27fc640 1 -- 172.21.15.32:0/2976182955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e4102ae0 msgr2=0x7f98e4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e27fc640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e4102ae0 0x7f98e4198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e27fc640 1 -- 172.21.15.32:0/2976182955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e419ff40 con 0x7f98e4105410 2024-04-16T05:32:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e27fc640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 0x7f98e4198860 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f98cc005ab0 tx=0x7f98cc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98dbfff640 1 -- 172.21.15.32:0/2976182955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc041070 con 0x7f98e4105410 2024-04-16T05:32:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98dbfff640 1 -- 172.21.15.32:0/2976182955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98cc0054e0 con 0x7f98e4105410 2024-04-16T05:32:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e41a01d0 con 0x7f98e4105410 2024-04-16T05:32:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98dbfff640 1 -- 172.21.15.32:0/2976182955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc034440 con 0x7f98e4105410 2024-04-16T05:32:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.045+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41a0710 con 0x7f98e4105410 2024-04-16T05:32:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.046+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e40fdfe0 con 0x7f98e4105410 2024-04-16T05:32:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.047+0000 7f98dbfff640 1 -- 172.21.15.32:0/2976182955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f98cc044070 con 0x7f98e4105410 2024-04-16T05:32:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.048+0000 7f98dbfff640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f98d00a3c30 0x7f98d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.048+0000 7f98e2ffd640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f98d00a3c30 0x7f98d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.048+0000 7f98dbfff640 1 -- 172.21.15.32:0/2976182955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f98cc0f24b0 con 0x7f98e4105410 2024-04-16T05:32:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.048+0000 7f98e2ffd640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f98d00a3c30 0x7f98d00a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98d4008d20 tx=0x7f98d40082b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.051+0000 7f98dbfff640 1 -- 172.21.15.32:0/2976182955 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98cc0bec50 con 0x7f98e4105410 2024-04-16T05:32:40.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:39 smithi032 ceph-mon[32351]: pgmap v290: 129 pgs: 12 creating+peering, 89 active+clean, 28 unknown; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 9 op/s 2024-04-16T05:32:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:39 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1907969706' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T05:32:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:39 smithi032 ceph-mon[32351]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T05:32:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:39 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3540399294' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T05:32:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:39 smithi032 ceph-mon[32351]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:39 smithi032 ceph-mon[32351]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T05:32:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.281+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98e402d620 con 0x7f98e4105410 2024-04-16T05:32:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.282+0000 7f98dbfff640 1 -- 172.21.15.32:0/2976182955 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11803 (secure 0 0 0) 0x7f98cc0be550 con 0x7f98e4105410 2024-04-16T05:32:40.280 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:40.281 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:32:39.296753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24494,"qdb_cluster":[24494]},"id":1}]} 2024-04-16T05:32:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2024-04-16T05:32:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f98d00a3c30 msgr2=0x7f98d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f98d00a3c30 0x7f98d00a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98d4008d20 tx=0x7f98d40082b0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 msgr2=0x7f98e4198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 0x7f98e4198860 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f98cc005ab0 tx=0x7f98cc005dc0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 shutdown_connections 2024-04-16T05:32:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f98d00a3c30 0x7f98d00a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98e4107d40 0x7f98e419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e4105410 0x7f98e4198860 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 --2- 172.21.15.32:0/2976182955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e4102ae0 0x7f98e4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 >> 172.21.15.32:0/2976182955 conn(0x7f98e40fc4f0 msgr2=0x7f98e4103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 shutdown_connections 2024-04-16T05:32:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.284+0000 7f98e9500640 1 -- 172.21.15.32:0/2976182955 wait complete. 2024-04-16T05:32:40.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:40.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:39 smithi134 ceph-mon[36153]: pgmap v290: 129 pgs: 12 creating+peering, 89 active+clean, 28 unknown; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 9 op/s 2024-04-16T05:32:40.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:39 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1907969706' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T05:32:40.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:39 smithi134 ceph-mon[36153]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T05:32:40.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:39 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3540399294' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T05:32:40.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:39 smithi134 ceph-mon[36153]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:40.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:39 smithi134 ceph-mon[36153]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T05:32:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:39 smithi082 ceph-mon[36195]: pgmap v290: 129 pgs: 12 creating+peering, 89 active+clean, 28 unknown; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 9 op/s 2024-04-16T05:32:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:39 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1907969706' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T05:32:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:39 smithi082 ceph-mon[36195]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T05:32:40.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:39 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3540399294' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T05:32:40.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:39 smithi082 ceph-mon[36195]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T05:32:40.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:39 smithi082 ceph-mon[36195]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T05:32:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 -- 172.21.15.32:0/1271292443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb650102aa0 msgr2=0x7fb6501093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 --2- 172.21.15.32:0/1271292443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb650102aa0 0x7fb6501093c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb648009900 tx=0x7fb64802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 -- 172.21.15.32:0/1271292443 shutdown_connections 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 --2- 172.21.15.32:0/1271292443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb650102aa0 0x7fb6501093c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 --2- 172.21.15.32:0/1271292443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6501020e0 0x7fb650102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 --2- 172.21.15.32:0/1271292443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb650100f10 0x7fb650101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 -- 172.21.15.32:0/1271292443 >> 172.21.15.32:0/1271292443 conn(0x7fb6500fc4f0 msgr2=0x7fb6500fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 -- 172.21.15.32:0/1271292443 shutdown_connections 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.401+0000 7fb65831c640 1 -- 172.21.15.32:0/1271292443 wait complete. 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb65831c640 1 Processor -- start 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb65831c640 1 -- start start 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb65831c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb650100f10 0x7fb650198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb65831c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6501020e0 0x7fb650198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb65831c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb650102aa0 0x7fb65019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb65831c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb65010cbe0 con 0x7fb6501020e0 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb65831c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb65010ca60 con 0x7fb650100f10 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb65831c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb65010cd60 con 0x7fb650102aa0 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6558b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6501020e0 0x7fb650198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6558b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6501020e0 0x7fb650198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57844/0 (socket says 172.21.15.32:57844) 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6558b7640 1 -- 172.21.15.32:0/2521697126 learned_addr learned my addr 172.21.15.32:0/2521697126 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6568b9640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb650102aa0 0x7fb65019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6560b8640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb650100f10 0x7fb650198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6558b7640 1 -- 172.21.15.32:0/2521697126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb650102aa0 msgr2=0x7fb65019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6558b7640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb650102aa0 0x7fb65019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6558b7640 1 -- 172.21.15.32:0/2521697126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb650100f10 msgr2=0x7fb650198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6558b7640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb650100f10 0x7fb650198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.402+0000 7fb6558b7640 1 -- 172.21.15.32:0/2521697126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65019ff50 con 0x7fb6501020e0 2024-04-16T05:32:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.403+0000 7fb6558b7640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6501020e0 0x7fb650198870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb64000c970 tx=0x7fb64000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.403+0000 7fb6477fe640 1 -- 172.21.15.32:0/2521697126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640007bb0 con 0x7fb6501020e0 2024-04-16T05:32:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.403+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6501a0240 con 0x7fb6501020e0 2024-04-16T05:32:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.403+0000 7fb6477fe640 1 -- 172.21.15.32:0/2521697126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb640007d50 con 0x7fb6501020e0 2024-04-16T05:32:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.403+0000 7fb6477fe640 1 -- 172.21.15.32:0/2521697126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64000f450 con 0x7fb6501020e0 2024-04-16T05:32:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.403+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6501a0740 con 0x7fb6501020e0 2024-04-16T05:32:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.404+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6501a0a00 con 0x7fb6501020e0 2024-04-16T05:32:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.405+0000 7fb6477fe640 1 -- 172.21.15.32:0/2521697126 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb640004fc0 con 0x7fb6501020e0 2024-04-16T05:32:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.406+0000 7fb6477fe640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb63c0a3bb0 0x7fb63c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.406+0000 7fb6560b8640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb63c0a3bb0 0x7fb63c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.406+0000 7fb6477fe640 1 -- 172.21.15.32:0/2521697126 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb6400d08e0 con 0x7fb6501020e0 2024-04-16T05:32:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.406+0000 7fb6560b8640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb63c0a3bb0 0x7fb63c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb650100c30 tx=0x7fb64c008230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.410+0000 7fb6477fe640 1 -- 172.21.15.32:0/2521697126 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb640014030 con 0x7fb6501020e0 2024-04-16T05:32:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.598+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6501a0bd0 con 0x7fb6501020e0 2024-04-16T05:32:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.599+0000 7fb6477fe640 1 -- 172.21.15.32:0/2521697126 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7fb64009d060 con 0x7fb6501020e0 2024-04-16T05:32:40.598 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:40.599 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.601+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb63c0a3bb0 msgr2=0x7fb63c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.601+0000 7fb65831c640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb63c0a3bb0 0x7fb63c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb650100c30 tx=0x7fb64c008230 comp rx=0 tx=0).stop 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.601+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6501020e0 msgr2=0x7fb650198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6501020e0 0x7fb650198870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb64000c970 tx=0x7fb64000ce30 comp rx=0 tx=0).stop 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 shutdown_connections 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb63c0a3bb0 0x7fb63c0a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb650102aa0 0x7fb65019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6501020e0 0x7fb650198870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 --2- 172.21.15.32:0/2521697126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb650100f10 0x7fb650198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 >> 172.21.15.32:0/2521697126 conn(0x7fb6500fc4f0 msgr2=0x7fb65006a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 shutdown_connections 2024-04-16T05:32:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.602+0000 7fb65831c640 1 -- 172.21.15.32:0/2521697126 wait complete. 2024-04-16T05:32:40.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.827+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/4194083114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102090 msgr2=0x7fdbc8102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.827+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/4194083114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102090 0x7fdbc8102510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009900 tx=0x7fdbbc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/4194083114 shutdown_connections 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/4194083114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc8102a50 0x7fdbc8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/4194083114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102090 0x7fdbc8102510 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/4194083114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbc8100f40 0x7fdbc8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/4194083114 >> 172.21.15.32:0/4194083114 conn(0x7fdbc80fc4b0 msgr2=0x7fdbc80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/4194083114 shutdown_connections 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/4194083114 wait complete. 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 Processor -- start 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 -- start start 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbc8100f40 0x7fdbc81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc8102090 0x7fdbc81977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102a50 0x7fdbc819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810cbd0 con 0x7fdbc8102a50 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810ca50 con 0x7fdbc8102090 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbcfd8f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810cd50 con 0x7fdbc8100f40 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.828+0000 7fdbce32c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102a50 0x7fdbc819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbce32c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102a50 0x7fdbc819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57870/0 (socket says 172.21.15.32:57870) 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbce32c640 1 -- 172.21.15.32:0/810634321 learned_addr learned my addr 172.21.15.32:0/810634321 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbcdb2b640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbc8100f40 0x7fdbc81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbce32c640 1 -- 172.21.15.32:0/810634321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbc8100f40 msgr2=0x7fdbc81972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbce32c640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbc8100f40 0x7fdbc81972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbcd32a640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc8102090 0x7fdbc81977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbce32c640 1 -- 172.21.15.32:0/810634321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc8102090 msgr2=0x7fdbc81977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbce32c640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc8102090 0x7fdbc81977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbce32c640 1 -- 172.21.15.32:0/810634321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc819eec0 con 0x7fdbc8102a50 2024-04-16T05:32:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbce32c640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102a50 0x7fdbc819e850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009fd0 tx=0x7fdbb800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbc6ffd640 1 -- 172.21.15.32:0/810634321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8017070 con 0x7fdbc8102a50 2024-04-16T05:32:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbc6ffd640 1 -- 172.21.15.32:0/810634321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb8003e50 con 0x7fdbc8102a50 2024-04-16T05:32:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc819f1b0 con 0x7fdbc8102a50 2024-04-16T05:32:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbc6ffd640 1 -- 172.21.15.32:0/810634321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80106c0 con 0x7fdbc8102a50 2024-04-16T05:32:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.829+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc819f6b0 con 0x7fdbc8102a50 2024-04-16T05:32:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.830+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc81082d0 con 0x7fdbc8102a50 2024-04-16T05:32:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.831+0000 7fdbc6ffd640 1 -- 172.21.15.32:0/810634321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdbb8003970 con 0x7fdbc8102a50 2024-04-16T05:32:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.832+0000 7fdbc6ffd640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdbb00a3bb0 0x7fdbb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.832+0000 7fdbcdb2b640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdbb00a3bb0 0x7fdbb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.832+0000 7fdbc6ffd640 1 -- 172.21.15.32:0/810634321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fdbb80d0430 con 0x7fdbc8102a50 2024-04-16T05:32:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.832+0000 7fdbcdb2b640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdbb00a3bb0 0x7fdbb00a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0005d00 tx=0x7fdbc0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:40.836+0000 7fdbc6ffd640 1 -- 172.21.15.32:0/810634321 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbb809cb70 con 0x7fdbc8102a50 2024-04-16T05:32:40.923 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:32:40 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b[66419]: 2024-04-16T05:32:40.673+0000 7f2baedcc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-16T05:32:41.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.027+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbc802d7f0 con 0x7fdbc8102a50 2024-04-16T05:32:41.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.029+0000 7fdbc6ffd640 1 -- 172.21.15.32:0/810634321 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7fdbb809c470 con 0x7fdbc8102a50 2024-04-16T05:32:41.027 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:41.028 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdbb00a3bb0 msgr2=0x7fdbb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdbb00a3bb0 0x7fdbb00a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0005d00 tx=0x7fdbc0005c50 comp rx=0 tx=0).stop 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102a50 msgr2=0x7fdbc819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102a50 0x7fdbc819e850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009fd0 tx=0x7fdbb800b260 comp rx=0 tx=0).stop 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 shutdown_connections 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdbb00a3bb0 0x7fdbb00a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbc8102a50 0x7fdbc819e850 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc8102090 0x7fdbc81977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 --2- 172.21.15.32:0/810634321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbc8100f40 0x7fdbc81972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 >> 172.21.15.32:0/810634321 conn(0x7fdbc80fc4b0 msgr2=0x7fdbc81054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 shutdown_connections 2024-04-16T05:32:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.031+0000 7fdbcfd8f640 1 -- 172.21.15.32:0/810634321 wait complete. 2024-04-16T05:32:41.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 -- 172.21.15.32:0/1540851780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103c20 msgr2=0x7f877010a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 --2- 172.21.15.32:0/1540851780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103c20 0x7f877010a4b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8768009ca0 tx=0x7f876802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 -- 172.21.15.32:0/1540851780 shutdown_connections 2024-04-16T05:32:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 --2- 172.21.15.32:0/1540851780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103c20 0x7f877010a4b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 --2- 172.21.15.32:0/1540851780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8770103260 0x7f87701036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 --2- 172.21.15.32:0/1540851780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8770102070 0x7f8770102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 -- 172.21.15.32:0/1540851780 >> 172.21.15.32:0/1540851780 conn(0x7f87700fd5e0 msgr2=0x7f87700ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 -- 172.21.15.32:0/1540851780 shutdown_connections 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 -- 172.21.15.32:0/1540851780 wait complete. 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 Processor -- start 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.163+0000 7f8778060640 1 -- start start 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8778060640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8770102070 0x7f8770198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8778060640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103260 0x7f8770198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8778060640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8770103c20 0x7f877019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8778060640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f877010dd00 con 0x7f8770103260 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8778060640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f877010db80 con 0x7f8770103c20 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8778060640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f877010de80 con 0x7f8770102070 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103260 0x7f8770198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103260 0x7f8770198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57884/0 (socket says 172.21.15.32:57884) 2024-04-16T05:32:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 -- 172.21.15.32:0/269803918 learned_addr learned my addr 172.21.15.32:0/269803918 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87765fd640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8770103c20 0x7f877019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8775dfc640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8770102070 0x7f8770198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 -- 172.21.15.32:0/269803918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8770102070 msgr2=0x7f8770198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8770102070 0x7f8770198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 -- 172.21.15.32:0/269803918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8770103c20 msgr2=0x7f877019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8770103c20 0x7f877019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 -- 172.21.15.32:0/269803918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877019ff40 con 0x7f8770103260 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f87755fb640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103260 0x7f8770198860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f876000c890 tx=0x7f876000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8766ffd640 1 -- 172.21.15.32:0/269803918 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760004220 con 0x7f8770103260 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8766ffd640 1 -- 172.21.15.32:0/269803918 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87600043c0 con 0x7f8770103260 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87701a0230 con 0x7f8770103260 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8766ffd640 1 -- 172.21.15.32:0/269803918 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87600049e0 con 0x7f8770103260 2024-04-16T05:32:41.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.164+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87701a07d0 con 0x7f8770103260 2024-04-16T05:32:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.165+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8770109330 con 0x7f8770103260 2024-04-16T05:32:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.167+0000 7f8766ffd640 1 -- 172.21.15.32:0/269803918 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f87600079b0 con 0x7f8770103260 2024-04-16T05:32:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.167+0000 7f8766ffd640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f875c0a3c30 0x7f875c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.167+0000 7f8775dfc640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f875c0a3c30 0x7f875c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.167+0000 7f8766ffd640 1 -- 172.21.15.32:0/269803918 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f87600d0640 con 0x7f8770103260 2024-04-16T05:32:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.168+0000 7f8775dfc640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f875c0a3c30 0x7f875c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87701030c0 tx=0x7f876c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.171+0000 7f8766ffd640 1 -- 172.21.15.32:0/269803918 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8760014030 con 0x7f8770103260 2024-04-16T05:32:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:40 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3540399294' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T05:32:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:40 smithi032 ceph-mon[32351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:40 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2976182955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:41.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:40 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2521697126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:41.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:40 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3540399294' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T05:32:41.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:40 smithi134 ceph-mon[36153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:41.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:40 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2976182955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:41.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:40 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2521697126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.395+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f877002d890 con 0x7f8770103260 2024-04-16T05:32:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.396+0000 7f8766ffd640 1 -- 172.21.15.32:0/269803918 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11803 (secure 0 0 0) 0x7f876009ce60 con 0x7f8770103260 2024-04-16T05:32:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:41.398 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:32:39.296753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24494,"qdb_cluster":[24494]},"id":1}]} 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f875c0a3c30 msgr2=0x7f875c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f875c0a3c30 0x7f875c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87701030c0 tx=0x7f876c005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103260 msgr2=0x7f8770198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103260 0x7f8770198860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f876000c890 tx=0x7f876000cd50 comp rx=0 tx=0).stop 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 shutdown_connections 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f875c0a3c30 0x7f875c0a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8770103c20 0x7f877019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8770103260 0x7f8770198860 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 --2- 172.21.15.32:0/269803918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8770102070 0x7f8770198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 >> 172.21.15.32:0/269803918 conn(0x7f87700fd5e0 msgr2=0x7f8770106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 shutdown_connections 2024-04-16T05:32:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.398+0000 7f8778060640 1 -- 172.21.15.32:0/269803918 wait complete. 2024-04-16T05:32:41.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:40 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3540399294' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T05:32:41.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:40 smithi082 ceph-mon[36195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:41.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:40 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2976182955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:41.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:40 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2521697126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:41.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 -- 172.21.15.32:0/1029723749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102b50 msgr2=0x7efee81093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 --2- 172.21.15.32:0/1029723749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102b50 0x7efee81093e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efee0009ca0 tx=0x7efee002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 -- 172.21.15.32:0/1029723749 shutdown_connections 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 --2- 172.21.15.32:0/1029723749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102b50 0x7efee81093e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 --2- 172.21.15.32:0/1029723749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee8102190 0x7efee8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 --2- 172.21.15.32:0/1029723749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee8101480 0x7efee81018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 -- 172.21.15.32:0/1029723749 >> 172.21.15.32:0/1029723749 conn(0x7efee80fbc70 msgr2=0x7efee80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 -- 172.21.15.32:0/1029723749 shutdown_connections 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.514+0000 7efeedae0640 1 -- 172.21.15.32:0/1029723749 wait complete. 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efeedae0640 1 Processor -- start 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efeedae0640 1 -- start start 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efeedae0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee8101480 0x7efee8100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efeedae0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102190 0x7efee819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efeedae0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee8102b50 0x7efee819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efeedae0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efee810cc20 con 0x7efee8102190 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efeedae0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efee810caa0 con 0x7efee8102b50 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efeedae0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efee810cda0 con 0x7efee8101480 2024-04-16T05:32:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efee6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102190 0x7efee819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efee6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102190 0x7efee819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57904/0 (socket says 172.21.15.32:57904) 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efee6ffd640 1 -- 172.21.15.32:0/693768835 learned_addr learned my addr 172.21.15.32:0/693768835 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.515+0000 7efee7fff640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee8102b50 0x7efee819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee77fe640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee8101480 0x7efee8100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee6ffd640 1 -- 172.21.15.32:0/693768835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee8101480 msgr2=0x7efee8100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee6ffd640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee8101480 0x7efee8100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee6ffd640 1 -- 172.21.15.32:0/693768835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee8102b50 msgr2=0x7efee819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee6ffd640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee8102b50 0x7efee819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee6ffd640 1 -- 172.21.15.32:0/693768835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee819ff00 con 0x7efee8102190 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee6ffd640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102190 0x7efee819d000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efedc00d950 tx=0x7efedc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee4ff9640 1 -- 172.21.15.32:0/693768835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc017070 con 0x7efee8102190 2024-04-16T05:32:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee4ff9640 1 -- 172.21.15.32:0/693768835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efedc009c30 con 0x7efee8102190 2024-04-16T05:32:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efee4ff9640 1 -- 172.21.15.32:0/693768835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc00f6b0 con 0x7efee8102190 2024-04-16T05:32:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee81a01f0 con 0x7efee8102190 2024-04-16T05:32:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.516+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee81a06f0 con 0x7efee8102190 2024-04-16T05:32:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.517+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee81082c0 con 0x7efee8102190 2024-04-16T05:32:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.518+0000 7efee4ff9640 1 -- 172.21.15.32:0/693768835 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efedc0109d0 con 0x7efee8102190 2024-04-16T05:32:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.519+0000 7efee4ff9640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efed00a3bb0 0x7efed00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.519+0000 7efee77fe640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efed00a3bb0 0x7efed00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.519+0000 7efee4ff9640 1 -- 172.21.15.32:0/693768835 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efedc0d0180 con 0x7efee8102190 2024-04-16T05:32:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.519+0000 7efee77fe640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efed00a3bb0 0x7efed00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efed8005fd0 tx=0x7efed800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.523+0000 7efee4ff9640 1 -- 172.21.15.32:0/693768835 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efedc009de0 con 0x7efee8102190 2024-04-16T05:32:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.710+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efee8062c30 con 0x7efee8102190 2024-04-16T05:32:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.712+0000 7efee4ff9640 1 -- 172.21.15.32:0/693768835 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7efedc09c270 con 0x7efee8102190 2024-04-16T05:32:41.710 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:41.711 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.713+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efed00a3bb0 msgr2=0x7efed00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.713+0000 7efeedae0640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efed00a3bb0 0x7efed00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efed8005fd0 tx=0x7efed800b410 comp rx=0 tx=0).stop 2024-04-16T05:32:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.713+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102190 msgr2=0x7efee819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.713+0000 7efeedae0640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102190 0x7efee819d000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efedc00d950 tx=0x7efedc00de10 comp rx=0 tx=0).stop 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.714+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 shutdown_connections 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.714+0000 7efeedae0640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7efed00a3bb0 0x7efed00a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.714+0000 7efeedae0640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee8102b50 0x7efee819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.714+0000 7efeedae0640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee8102190 0x7efee819d000 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.714+0000 7efeedae0640 1 --2- 172.21.15.32:0/693768835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee8101480 0x7efee8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.714+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 >> 172.21.15.32:0/693768835 conn(0x7efee80fbc70 msgr2=0x7efee8105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.714+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 shutdown_connections 2024-04-16T05:32:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.714+0000 7efeedae0640 1 -- 172.21.15.32:0/693768835 wait complete. 2024-04-16T05:32:41.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-16T05:32:34.109354+0000', 'modified': '2024-04-16T05:32:39.296753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6839/2453097092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2453097092}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2453097092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24494, 'qdb_cluster': [24494]} 2024-04-16T05:32:41.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-16T05:32:41.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T05:32:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.833+0000 7fc117247640 1 -- 172.21.15.32:0/1976648441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1101024f0 msgr2=0x7fc110102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.833+0000 7fc117247640 1 --2- 172.21.15.32:0/1976648441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1101024f0 0x7fc110102910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc100009900 tx=0x7fc10002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 -- 172.21.15.32:0/1976648441 shutdown_connections 2024-04-16T05:32:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 --2- 172.21.15.32:0/1976648441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110101be0 0x7fc110109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 --2- 172.21.15.32:0/1976648441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc110101220 0x7fc1101016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 --2- 172.21.15.32:0/1976648441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1101024f0 0x7fc110102910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 -- 172.21.15.32:0/1976648441 >> 172.21.15.32:0/1976648441 conn(0x7fc1100fbc30 msgr2=0x7fc1100fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 -- 172.21.15.32:0/1976648441 shutdown_connections 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 -- 172.21.15.32:0/1976648441 wait complete. 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 Processor -- start 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 -- start start 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110101220 0x7fc1101983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110101be0 0x7fc110198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1101024f0 0x7fc11019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc11010dd10 con 0x7fc110101be0 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc11010db90 con 0x7fc110101220 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc117247640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc11010de90 con 0x7fc1101024f0 2024-04-16T05:32:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc10ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110101be0 0x7fc110198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc10ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110101be0 0x7fc110198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57912/0 (socket says 172.21.15.32:57912) 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.834+0000 7fc10ffff640 1 -- 172.21.15.32:0/1228493592 learned_addr learned my addr 172.21.15.32:0/1228493592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc114fe3640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110101220 0x7fc1101983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc1157e4640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1101024f0 0x7fc11019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10ffff640 1 -- 172.21.15.32:0/1228493592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1101024f0 msgr2=0x7fc11019f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10ffff640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1101024f0 0x7fc11019f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10ffff640 1 -- 172.21.15.32:0/1228493592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110101220 msgr2=0x7fc1101983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10ffff640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110101220 0x7fc1101983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10ffff640 1 -- 172.21.15.32:0/1228493592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11019ffe0 con 0x7fc110101be0 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc1157e4640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1101024f0 0x7fc11019f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10ffff640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110101be0 0x7fc110198900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc10400e9f0 tx=0x7fc10400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10dffb640 1 -- 172.21.15.32:0/1228493592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10400cd80 con 0x7fc110101be0 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc117247640 1 -- 172.21.15.32:0/1228493592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1101a02d0 con 0x7fc110101be0 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10dffb640 1 -- 172.21.15.32:0/1228493592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc104005500 con 0x7fc110101be0 2024-04-16T05:32:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc10dffb640 1 -- 172.21.15.32:0/1228493592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104010690 con 0x7fc110101be0 2024-04-16T05:32:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.835+0000 7fc117247640 1 -- 172.21.15.32:0/1228493592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1101a0820 con 0x7fc110101be0 2024-04-16T05:32:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.837+0000 7fc10dffb640 1 -- 172.21.15.32:0/1228493592 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc1040046c0 con 0x7fc110101be0 2024-04-16T05:32:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc119e2d740 1 Processor -- start 2024-04-16T05:32:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc119e2d740 1 -- start start 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc10dffb640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0f80a3bb0 0x7fc0f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfc1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfc6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfc8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc119e2d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56365bed4240 con 0x56365bfc68c0 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc119e2d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56365bf319f0 con 0x56365bfc1750 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc119e2d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56365beec9e0 con 0x56365bfc0d60 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0f80a3bb0 0x7fc0f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc10dffb640 1 -- 172.21.15.32:0/1228493592 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc1040d0c00 con 0x7fc110101be0 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfc8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfc8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57922/0 (socket says 172.21.15.32:57922) 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 -- 172.21.15.32:0/3035668584 learned_addr learned my addr 172.21.15.32:0/3035668584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc114fe3640 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfc1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc10ffff640 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfc6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 -- 172.21.15.32:0/3035668584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 msgr2=0x56365bfc1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfc1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 -- 172.21.15.32:0/3035668584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 msgr2=0x56365bfc6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfc6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc1157e4640 1 -- 172.21.15.32:0/3035668584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365bedfbc0 con 0x56365bfc68c0 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.838+0000 7fc114fe3640 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfc1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc1157e4640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0f80a3bb0 0x7fc0f80a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc108009fd0 tx=0x7fc108009dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc1157e4640 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfc8d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc108011810 tx=0x7fc108014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edee3622ac9313aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc0f6ffd640 1 -- 172.21.15.32:0/3035668584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10800fc00 con 0x56365bfc68c0 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc0f6ffd640 1 -- 172.21.15.32:0/3035668584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10800fda0 con 0x56365bfc68c0 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc0f6ffd640 1 -- 172.21.15.32:0/3035668584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108005500 con 0x56365bfc68c0 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc119e2d740 1 -- 172.21.15.32:0/3035668584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 msgr2=0x56365bfc8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc119e2d740 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfc8d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc108011810 tx=0x7fc108014f90 comp rx=0 tx=0).stop 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc119e2d740 1 -- 172.21.15.32:0/3035668584 shutdown_connections 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc119e2d740 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfc8d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc119e2d740 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfc6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc119e2d740 1 --2- 172.21.15.32:0/3035668584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfc1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.839+0000 7fc119e2d740 1 -- 172.21.15.32:0/3035668584 >> 172.21.15.32:0/3035668584 conn(0x56365bfbc2d0 msgr2=0x56365bfbc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 -- 172.21.15.32:0/3035668584 shutdown_connections 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 -- 172.21.15.32:0/3035668584 wait complete. 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 -- start start 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 Processor -- start 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfcaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfcb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfd2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56365bf2b0f0 con 0x56365bfc68c0 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56365bf25dd0 con 0x56365bfc1750 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56365bf33cd0 con 0x56365bfc0d60 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc119e2d740 10 client.0 _ll_register_callbacks cb 0x56365bf1c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:32:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc1157e4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfd2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc1157e4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfd2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57938/0 (socket says 172.21.15.32:57938) 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.840+0000 7fc1157e4640 1 -- 172.21.15.32:0/2531886970 learned_addr learned my addr 172.21.15.32:0/2531886970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc114fe3640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfcaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc10ffff640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfcb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc1157e4640 1 -- 172.21.15.32:0/2531886970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 msgr2=0x56365bfcaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc1157e4640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfcaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc1157e4640 1 -- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 msgr2=0x56365bfcb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc1157e4640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfcb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc1157e4640 1 -- 172.21.15.32:0/2531886970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365bc392d0 con 0x56365bfc68c0 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc114fe3640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfcaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc10ffff640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfcb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc1157e4640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfd2460 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc108014410 tx=0x7fc108014440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554138ee84fdde7d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc0effff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc0effff640 1 -- 172.21.15.32:0/2531886970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108034060 con 0x56365bfc68c0 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc0effff640 1 -- 172.21.15.32:0/2531886970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc1080054e0 con 0x56365bfc68c0 2024-04-16T05:32:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc119e2d740 1 -- 172.21.15.32:0/2531886970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10800fda0 con 0x56365bfc68c0 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc0effff640 1 -- 172.21.15.32:0/2531886970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc10803b050 con 0x56365bfc68c0 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc119e2d740 10 client.14721 fetch_fsmap learned FSMap version 17 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc119e2d740 1 -- 172.21.15.32:0/2531886970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56365bd036b0 con 0x56365bfc68c0 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.841+0000 7fc0effff640 1 -- 172.21.15.32:0/2531886970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10800fda0 con 0x56365bfc68c0 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc0effff640 1 -- 172.21.15.32:0/2531886970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fc10802e030 con 0x56365bfc68c0 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc119e2d740 10 client.14721 fetch_fsmap finished waiting for FSMap version 17 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc119e2d740 10 client.14721 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc119e2d740 20 client.14721 populate_metadata read hostname 'smithi032' 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc119e2d740 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56365bfd7b00 0x56365bfd8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc119e2d740 4 client.14721 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc119e2d740 1 -- 172.21.15.32:0/2531886970 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56365bead4c0 con 0x56365bfd7b00 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc114fe3640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56365bfd7b00 0x56365bfd8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc114fe3640 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56365bfd7b00 0x56365bfd8f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bd0a1894cfa799e server_cookie=ae143833aa9d5923 in_seq=0 out_seq=0 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.842+0000 7fc0effff640 10 client.14721 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:32:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc0effff640 1 -- 172.21.15.32:0/2531886970 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56365bead4c0 con 0x56365bfd7b00 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc0effff640 10 client.14721 handle_command_reply: tid=0 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 client.14721 shutdown 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 -- 172.21.15.32:0/2531886970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 msgr2=0x56365bfd2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfd2460 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc108014410 tx=0x7fc108014440 comp rx=0 tx=0).stop 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 -- 172.21.15.32:0/2531886970 shutdown_connections 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc68c0 0x56365bfd2460 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc1750 0x56365bfcb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56365bfd7b00 0x56365bfd8f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 --2- 172.21.15.32:0/2531886970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc0d60 0x56365bfcaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc0effff640 0 client.14721 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 -- 172.21.15.32:0/2531886970 >> 172.21.15.32:0/2531886970 conn(0x56365bfbc2d0 msgr2=0x56365bfbb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 -- 172.21.15.32:0/2531886970 shutdown_connections 2024-04-16T05:32:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.843+0000 7fc119e2d740 1 -- 172.21.15.32:0/2531886970 wait complete. 2024-04-16T05:32:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.844+0000 7fc119e2d740 20 client.14721 trim_cache size 0 max 16384 2024-04-16T05:32:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc119e2d740 1 Processor -- start 2024-04-16T05:32:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc119e2d740 1 -- start start 2024-04-16T05:32:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc0d60 0x56365bfbc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc1750 0x56365bfd2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc68c0 0x56365bfbaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc119e2d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56365bfd72b0 con 0x56365bfc0d60 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc119e2d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56365bfd7980 con 0x56365bfc68c0 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc119e2d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56365bfd8050 con 0x56365bfc1750 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc114fe3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc0d60 0x56365bfbc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc114fe3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc0d60 0x56365bfbc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57952/0 (socket says 172.21.15.32:57952) 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc114fe3640 1 -- 172.21.15.32:0/3663253079 learned_addr learned my addr 172.21.15.32:0/3663253079 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc10ffff640 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc1750 0x56365bfd2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc114fe3640 1 -- 172.21.15.32:0/3663253079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc1750 msgr2=0x56365bfd2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc114fe3640 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc1750 0x56365bfd2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc114fe3640 1 -- 172.21.15.32:0/3663253079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc68c0 msgr2=0x56365bfbaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc1157e4640 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc68c0 0x56365bfbaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc114fe3640 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc68c0 0x56365bfbaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.862+0000 7fc114fe3640 1 -- 172.21.15.32:0/3663253079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365bfc1b30 con 0x56365bfc0d60 2024-04-16T05:32:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc114fe3640 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc0d60 0x56365bfbc6b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc10002c7b0 tx=0x7fc100005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9963ad5058bd2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc0f67fc640 1 -- 172.21.15.32:0/3663253079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10004c070 con 0x56365bfc0d60 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc0f67fc640 1 -- 172.21.15.32:0/3663253079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc100047bd0 con 0x56365bfc0d60 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc0f67fc640 1 -- 172.21.15.32:0/3663253079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100045070 con 0x56365bfc0d60 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 -- 172.21.15.32:0/3663253079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc0d60 msgr2=0x56365bfbc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc0d60 0x56365bfbc6b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc10002c7b0 tx=0x7fc100005e70 comp rx=0 tx=0).stop 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 -- 172.21.15.32:0/3663253079 shutdown_connections 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc68c0 0x56365bfbaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc1750 0x56365bfd2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 --2- 172.21.15.32:0/3663253079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc0d60 0x56365bfbc6b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 -- 172.21.15.32:0/3663253079 >> 172.21.15.32:0/3663253079 conn(0x56365bfbc2d0 msgr2=0x56365bfd6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 -- 172.21.15.32:0/3663253079 shutdown_connections 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.863+0000 7fc119e2d740 1 -- 172.21.15.32:0/3663253079 wait complete. 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 1 -- start start 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 1 Processor -- start 2024-04-16T05:32:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc0d60 0x56365bfcdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc1750 0x56365bfce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc68c0 0x56365bfd10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56365bfd8df0 con 0x56365bfc1750 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56365bfd8720 con 0x56365bfc0d60 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56365bfd94c0 con 0x56365bfc68c0 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc119e2d740 10 client.0 _ll_register_callbacks cb 0x7fc1040056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc10ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc1750 0x56365bfce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc10ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc1750 0x56365bfce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57956/0 (socket says 172.21.15.32:57956) 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc10ffff640 1 -- 172.21.15.32:0/678985134 learned_addr learned my addr 172.21.15.32:0/678985134 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc114fe3640 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc0d60 0x56365bfcdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc1157e4640 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc68c0 0x56365bfd10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.864+0000 7fc10ffff640 1 -- 172.21.15.32:0/678985134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc68c0 msgr2=0x56365bfd10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc10ffff640 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc68c0 0x56365bfd10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc10ffff640 1 -- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc0d60 msgr2=0x56365bfcdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:32:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc10ffff640 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc0d60 0x56365bfcdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc10ffff640 1 -- 172.21.15.32:0/678985134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365bfd16a0 con 0x56365bfc1750 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc114fe3640 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc0d60 0x56365bfcdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc10ffff640 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc1750 0x56365bfce090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc10400e9c0 tx=0x7fc10409d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238875cf99bbe43a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc0effff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc0effff640 1 -- 172.21.15.32:0/678985134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1040a58b0 con 0x56365bfc1750 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc0effff640 1 -- 172.21.15.32:0/678985134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc104087040 con 0x56365bfc1750 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 1 -- 172.21.15.32:0/678985134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc100047bd0 con 0x56365bfc1750 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc0effff640 1 -- 172.21.15.32:0/678985134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc104016df0 con 0x56365bfc1750 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 10 client.14727 fetch_fsmap learned FSMap version 17 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 1 -- 172.21.15.32:0/678985134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56365c03dc40 con 0x56365bfc1750 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc0effff640 1 -- 172.21.15.32:0/678985134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc100047bd0 con 0x56365bfc1750 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc0effff640 1 -- 172.21.15.32:0/678985134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fc10409c090 con 0x56365bfc1750 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 10 client.14727 fetch_fsmap finished waiting for FSMap version 17 2024-04-16T05:32:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 10 client.14727 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 20 client.14727 populate_metadata read hostname 'smithi032' 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56365c042840 0x56365c041550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 4 client.14727 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.865+0000 7fc119e2d740 1 -- 172.21.15.32:0/678985134 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "status"}) -- 0x56365bf344e0 con 0x56365c042840 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc114fe3640 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56365c042840 0x56365c041550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc0effff640 10 client.14727 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc114fe3640 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56365c042840 0x56365c041550 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7527550b7de5383d server_cookie=281e672fa797ceae in_seq=0 out_seq=0 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc0effff640 1 -- 172.21.15.32:0/678985134 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56365bf344e0 con 0x56365c042840 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc0effff640 10 client.14727 handle_command_reply: tid=0 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 client.14727 shutdown 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 -- 172.21.15.32:0/678985134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc1750 msgr2=0x56365bfce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc1750 0x56365bfce090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc10400e9c0 tx=0x7fc10409d340 comp rx=0 tx=0).stop 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 -- 172.21.15.32:0/678985134 shutdown_connections 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56365c042840 0x56365c041550 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56365bfc68c0 0x56365bfd10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56365bfc1750 0x56365bfce090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 --2- 172.21.15.32:0/678985134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365bfc0d60 0x56365bfcdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc0effff640 0 client.14727 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 -- 172.21.15.32:0/678985134 >> 172.21.15.32:0/678985134 conn(0x56365bfbc2d0 msgr2=0x56365bfbc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 -- 172.21.15.32:0/678985134 shutdown_connections 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.866+0000 7fc119e2d740 1 -- 172.21.15.32:0/678985134 wait complete. 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc119e2d740 20 client.14727 trim_cache size 0 max 16384 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:32:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24494, 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 6.9741324679999996, 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 17, 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 89, 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.320365465 2024-04-16T05:32:41.873 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:32:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 -- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0f80a3bb0 msgr2=0x7fc0f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0f80a3bb0 0x7fc0f80a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc108009fd0 tx=0x7fc108009dc0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 -- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110101be0 msgr2=0x7fc110198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110101be0 0x7fc110198900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc10400e9f0 tx=0x7fc10400edf0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 -- 172.21.15.32:0/1228493592 shutdown_connections 2024-04-16T05:32:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0f80a3bb0 0x7fc0f80a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1101024f0 0x7fc11019f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110101be0 0x7fc110198900 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 --2- 172.21.15.32:0/1228493592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110101220 0x7fc1101983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 -- 172.21.15.32:0/1228493592 >> 172.21.15.32:0/1228493592 conn(0x7fc1100fbc30 msgr2=0x7fc110105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 -- 172.21.15.32:0/1228493592 shutdown_connections 2024-04-16T05:32:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:41.867+0000 7fc117247640 1 -- 172.21.15.32:0/1228493592 wait complete. 2024-04-16T05:32:41.876 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T05:32:41.877 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:32:41.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph fs ls 2024-04-16T05:32:41.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:41 smithi032 ceph-mon[32351]: pgmap v293: 129 pgs: 4 creating+peering, 104 active+clean, 21 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-04-16T05:32:41.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:41 smithi032 ceph-mon[32351]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:32:41.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:41 smithi032 ceph-mon[32351]: Cluster is now healthy 2024-04-16T05:32:41.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:41 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/810634321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:41.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:41 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/269803918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:41.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:41 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/693768835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:42.121 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:32:42.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:41 smithi134 ceph-mon[36153]: pgmap v293: 129 pgs: 4 creating+peering, 104 active+clean, 21 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-04-16T05:32:42.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:41 smithi134 ceph-mon[36153]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:32:42.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:41 smithi134 ceph-mon[36153]: Cluster is now healthy 2024-04-16T05:32:42.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:41 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/810634321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:42.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:41 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/269803918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:42.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:41 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/693768835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:42.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:41 smithi082 ceph-mon[36195]: pgmap v293: 129 pgs: 4 creating+peering, 104 active+clean, 21 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-04-16T05:32:42.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:41 smithi082 ceph-mon[36195]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T05:32:42.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:41 smithi082 ceph-mon[36195]: Cluster is now healthy 2024-04-16T05:32:42.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:41 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/810634321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:42.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:41 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/269803918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:42.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:41 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/693768835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:43.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 -- 172.21.15.32:0/2356351304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 msgr2=0x7f6c64102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:43.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/2356351304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 0x7f6c64102fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009990 tx=0x7f6c50009ca0 comp rx=0 tx=0).stop 2024-04-16T05:32:43.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 -- 172.21.15.32:0/2356351304 shutdown_connections 2024-04-16T05:32:43.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/2356351304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64103500 0x7f6c6410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:43.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/2356351304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64108b90 0x7f6c64108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:43.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/2356351304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 0x7f6c64102fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 -- 172.21.15.32:0/2356351304 >> 172.21.15.32:0/2356351304 conn(0x7f6c64079c50 msgr2=0x7f6c6407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 -- 172.21.15.32:0/2356351304 shutdown_connections 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 -- 172.21.15.32:0/2356351304 wait complete. 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 Processor -- start 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.974+0000 7f6c6abc3700 1 -- start start 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6abc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 0x7f6c64199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6abc3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64103500 0x7f6c64199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6abc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64108b90 0x7f6c6419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6abc3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64112ee0 con 0x7f6c64102b40 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6abc3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64112d60 con 0x7f6c64108b90 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6abc3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64113060 con 0x7f6c64103500 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 0x7f6c64199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 0x7f6c64199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57972/0 (socket says 172.21.15.32:57972) 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 -- 172.21.15.32:0/4143627378 learned_addr learned my addr 172.21.15.32:0/4143627378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c69160700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64108b90 0x7f6c6419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c63fff700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64103500 0x7f6c64199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 -- 172.21.15.32:0/4143627378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64103500 msgr2=0x7f6c64199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64103500 0x7f6c64199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 -- 172.21.15.32:0/4143627378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64108b90 msgr2=0x7f6c6419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64108b90 0x7f6c6419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 -- 172.21.15.32:0/4143627378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6419e420 con 0x7f6c64102b40 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c69160700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64108b90 0x7f6c6419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c63fff700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64103500 0x7f6c64199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6895f700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 0x7f6c64199410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800b530 tx=0x7f6c5800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c61ffb700 1 -- 172.21.15.32:0/4143627378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58015020 con 0x7f6c64102b40 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6419e710 con 0x7f6c64102b40 2024-04-16T05:32:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c61ffb700 1 -- 172.21.15.32:0/4143627378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c58011d10 con 0x7f6c64102b40 2024-04-16T05:32:43.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.975+0000 7f6c61ffb700 1 -- 172.21.15.32:0/4143627378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58010e50 con 0x7f6c64102b40 2024-04-16T05:32:43.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.976+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c641a5f90 con 0x7f6c64102b40 2024-04-16T05:32:43.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.976+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c640697e0 con 0x7f6c64102b40 2024-04-16T05:32:43.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.978+0000 7f6c61ffb700 1 -- 172.21.15.32:0/4143627378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c58018020 con 0x7f6c64102b40 2024-04-16T05:32:43.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.978+0000 7f6c61ffb700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c540a3de0 0x7f6c540a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.978+0000 7f6c61ffb700 1 -- 172.21.15.32:0/4143627378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6c580cff20 con 0x7f6c64102b40 2024-04-16T05:32:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.978+0000 7f6c63fff700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c540a3de0 0x7f6c540a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.979+0000 7f6c63fff700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c540a3de0 0x7f6c540a62a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000b5c0 tx=0x7f6c50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:43.983+0000 7f6c61ffb700 1 -- 172.21.15.32:0/4143627378 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c5801f020 con 0x7f6c64102b40 2024-04-16T05:32:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.249+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6c6419a7a0 con 0x7f6c64102b40 2024-04-16T05:32:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.249+0000 7f6c61ffb700 1 -- 172.21.15.32:0/4143627378 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f6c6419a7a0 con 0x7f6c64102b40 2024-04-16T05:32:44.248 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T05:32:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c540a3de0 msgr2=0x7f6c540a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c540a3de0 0x7f6c540a62a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000b5c0 tx=0x7f6c50005fb0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 msgr2=0x7f6c64199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 0x7f6c64199410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800b530 tx=0x7f6c5800b8f0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 shutdown_connections 2024-04-16T05:32:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6c540a3de0 0x7f6c540a62a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64108b90 0x7f6c6419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c64102b40 0x7f6c64199410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 --2- 172.21.15.32:0/4143627378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64103500 0x7f6c64199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 >> 172.21.15.32:0/4143627378 conn(0x7f6c64079c50 msgr2=0x7f6c64101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 shutdown_connections 2024-04-16T05:32:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:44.251+0000 7f6c6abc3700 1 -- 172.21.15.32:0/4143627378 wait complete. 2024-04-16T05:32:44.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:44 smithi134 ceph-mon[36153]: pgmap v294: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-16T05:32:44.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:44 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4143627378' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:32:44.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:44 smithi082 ceph-mon[36195]: pgmap v294: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-16T05:32:44.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:44 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4143627378' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:32:44.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:44 smithi032 ceph-mon[32351]: pgmap v294: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-16T05:32:44.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:44 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4143627378' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:32:45.101 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-16T05:32:45.101 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T05:32:45.101 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-16T05:32:45.101 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-16T05:32:45.102 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:32:45.102 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-16T05:32:45.102 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T05:32:45.102 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T05:32:45.102 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T05:32:45.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:45.102 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-16T05:32:45.120 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42070sec preferred_lft 42070sec 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:32:45.121 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-16T05:32:45.122 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T05:32:45.122 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:32:45.122 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:32:45.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-04-16T05:32:45.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-04-16T05:32:45.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-04-16T05:32:45.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T05:32:45.122 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:32:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:32:45.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:32:45.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:45.337 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T05:32:45.366 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-16T05:32:45.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:45.368 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-16T05:32:45.444 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-16T05:32:45.444 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T05:32:45.444 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-16T05:32:45.444 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-16T05:32:45.444 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T05:32:45.445 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:32:45.445 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:32:45.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-16T05:32:45.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-16T05:32:45.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-16T05:32:45.445 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:32:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:32:45.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:32:45.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:45.817 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-16T05:32:45.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:45.833 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-04-16T05:32:45.888 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:32:45.888 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:32:45.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:32:45.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-16T05:32:45.888 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:32:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:32:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:32:46.013 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-16T05:32:46.013 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:32:46.013 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:32:46.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-16T05:32:46.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T05:32:46.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-16T05:32:46.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-16T05:32:46.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-16T05:32:46.014 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:32:46.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:32:46.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:32:46.159 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:32:46.159 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:32:46.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-16T05:32:46.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T05:32:46.160 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:32:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:32:46.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:32:46.243 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-16T05:32:46.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T05:32:46.244 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:32:46.272 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T05:32:46.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T05:32:46.272 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:32:46.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-16T05:32:46.396 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T05:32:46.452 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-16T05:32:46.452 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/selinux 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-16T05:32:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-16T05:32:46.454 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-16T05:32:46.454 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T05:32:46.454 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-16T05:32:46.454 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-16T05:32:46.454 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T05:32:46.454 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T05:32:46.454 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T05:32:46.454 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-16T05:32:46.455 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-16T05:32:46.455 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay 2024-04-16T05:32:46.455 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/151e394c32b678b770367ca0ce9c11eb06186b0555f13061f3e035e9b1975a93/merged 2024-04-16T05:32:46.455 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/fe617c3f11e0f43ba4a41c0dfed02c9e1094eea0328988af7a2dbfcea7272f90/merged 2024-04-16T05:32:46.455 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/0 2024-04-16T05:32:46.455 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T05:32:46.455 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/d6a10c2c46636c58162d3b56b65696338ea5afcdcb6afde8d998c4c9f9247ab2/merged 2024-04-16T05:32:46.455 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/097b3c8986d8b3628ab5f27a14306e3021921f34009af5beebc5b1172c747252/merged 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/7836d69616b1ffb19045f98c7b0a58d85a59717fd76a1f94a8f2c0c26acf93ed/merged 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/701ccdb50ea0b6be47e143bc81dd57041f5c17d116339c3bd208a0a43255f245/merged 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/b9576f07d5134488d6e6d231d3af13c56e98f83ed10aacded8a8772584a17a12/merged 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/1361fa01d9936332082f14acaacff0f63828520b4fca8d1f7011aa203e2120c5/merged 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/feee35ca6436607927759f05875f131a64d9073d08f4f309d31acddfa842a737/merged 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/a4acdd463a4189a4cfa3e354cd7b9dccc39d6d1ac0b171fc5e2ea1ada6588dee/merged 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:32:46.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:46.456 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-16T05:32:46.507 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-16T05:32:46.507 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-16T05:32:46.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-16T05:32:46.578 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T05:32:46.634 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/selinux 2024-04-16T05:32:46.635 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/151e394c32b678b770367ca0ce9c11eb06186b0555f13061f3e035e9b1975a93/merged 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/fe617c3f11e0f43ba4a41c0dfed02c9e1094eea0328988af7a2dbfcea7272f90/merged 2024-04-16T05:32:46.636 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/0 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/d6a10c2c46636c58162d3b56b65696338ea5afcdcb6afde8d998c4c9f9247ab2/merged 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/097b3c8986d8b3628ab5f27a14306e3021921f34009af5beebc5b1172c747252/merged 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/7836d69616b1ffb19045f98c7b0a58d85a59717fd76a1f94a8f2c0c26acf93ed/merged 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/701ccdb50ea0b6be47e143bc81dd57041f5c17d116339c3bd208a0a43255f245/merged 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/b9576f07d5134488d6e6d231d3af13c56e98f83ed10aacded8a8772584a17a12/merged 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/1361fa01d9936332082f14acaacff0f63828520b4fca8d1f7011aa203e2120c5/merged 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/feee35ca6436607927759f05875f131a64d9073d08f4f309d31acddfa842a737/merged 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/containers/storage/overlay/a4acdd463a4189a4cfa3e354cd7b9dccc39d6d1ac0b171fc5e2ea1ada6588dee/merged 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:32:46.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:46.638 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-16T05:32:46.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:2024-04-16T05:32:46.659+0000 7f8188c0e480 -1 init, newargv = 0x55a171108f10 newargc=15 2024-04-16T05:32:46.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[76725]: starting ceph client 2024-04-16T05:32:46.663 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[76725]: starting fuse 2024-04-16T05:32:46.673 INFO:teuthology.orchestra.run.smithi032.stdout:119 2024-04-16T05:32:46.673 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-16T05:32:46.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-16T05:32:46.673 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-16T05:32:46.673 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-16T05:32:46.673 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-16T05:32:46.674 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.admin")) 2024-04-16T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-16T05:32:46.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:46 smithi032 ceph-mon[32351]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 292 B/s wr, 3 op/s 2024-04-16T05:32:46.796 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.admin.76725.asok 2024-04-16T05:32:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:32:46.806 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76725.asok 2024-04-16T05:32:46.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:46.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76725.asok status 2024-04-16T05:32:46.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:46 smithi082 ceph-mon[36195]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 292 B/s wr, 3 op/s 2024-04-16T05:32:46.949 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:32:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-16T05:32:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a87fcff6340baeedd43af126e793f3241b6400ea", 2024-04-16T05:32:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3101-ga87fcff6 (a87fcff6340baeedd43af126e793f3241b6400ea) squid (dev)", 2024-04-16T05:32:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "admin", 2024-04-16T05:32:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-16T05:32:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-16T05:32:46.949 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "76725", 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 14739, 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 14739 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1368089156 2024-04-16T05:32:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1368089156 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.14739 192.168.0.1:0/1368089156", 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/1368089156", 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 17, 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 89, 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-16T05:32:46.951 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:32:46.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:46.958 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-16T05:32:46.958 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:32:46.974 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-16T05:32:46.974 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:32:46.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:46.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:32:47.043 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-16T05:32:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:46 smithi134 ceph-mon[36153]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 292 B/s wr, 3 op/s 2024-04-16T05:32:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T05:32:47.154 INFO:teuthology.orchestra.run.smithi032.stdout:# file: . 2024-04-16T05:32:47.155 INFO:teuthology.orchestra.run.smithi032.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-16T05:32:47.155 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:32:47.160 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:32:47.161 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:32:47.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/3830153856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d781032e0 msgr2=0x7f5d78103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/3830153856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d781032e0 0x7f5d78103760 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009930 tx=0x7f5d6c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/3830153856 shutdown_connections 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/3830153856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d78103ca0 0x7f5d7810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/3830153856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d781032e0 0x7f5d78103760 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/3830153856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78100310 0x7f5d78100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/3830153856 >> 172.21.15.32:0/3830153856 conn(0x7f5d780fbc40 msgr2=0x7f5d780fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/3830153856 shutdown_connections 2024-04-16T05:32:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.323+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/3830153856 wait complete. 2024-04-16T05:32:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7d0ba640 1 Processor -- start 2024-04-16T05:32:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7d0ba640 1 -- start start 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7d0ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78100310 0x7f5d78198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7d0ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d781032e0 0x7f5d781988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7d0ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103ca0 0x7f5d7819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7d0ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7810dd00 con 0x7f5d78100310 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7d0ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7810db80 con 0x7f5d78103ca0 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7d0ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7810de80 con 0x7f5d781032e0 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d76d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78100310 0x7f5d78198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d76d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78100310 0x7f5d78198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57994/0 (socket says 172.21.15.32:57994) 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d76d9d640 1 -- 172.21.15.32:0/804418007 learned_addr learned my addr 172.21.15.32:0/804418007 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7759e640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103ca0 0x7f5d7819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.324+0000 7f5d7659c640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d781032e0 0x7f5d781988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d76d9d640 1 -- 172.21.15.32:0/804418007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d781032e0 msgr2=0x7f5d781988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d76d9d640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d781032e0 0x7f5d781988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d76d9d640 1 -- 172.21.15.32:0/804418007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103ca0 msgr2=0x7f5d7819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d76d9d640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103ca0 0x7f5d7819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d76d9d640 1 -- 172.21.15.32:0/804418007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7819ffb0 con 0x7f5d78100310 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d76d9d640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78100310 0x7f5d78198390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800e9c0 tx=0x7f5d6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d5ffff640 1 -- 172.21.15.32:0/804418007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800cd80 con 0x7f5d78100310 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d781a02a0 con 0x7f5d78100310 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d5ffff640 1 -- 172.21.15.32:0/804418007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d680035f0 con 0x7f5d78100310 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d5ffff640 1 -- 172.21.15.32:0/804418007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68010690 con 0x7f5d78100310 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.325+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d781a0810 con 0x7f5d78100310 2024-04-16T05:32:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.326+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d78109530 con 0x7f5d78100310 2024-04-16T05:32:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.327+0000 7f5d5ffff640 1 -- 172.21.15.32:0/804418007 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d680037a0 con 0x7f5d78100310 2024-04-16T05:32:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.328+0000 7f5d5ffff640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5d600a3bb0 0x7f5d600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.328+0000 7f5d5ffff640 1 -- 172.21.15.32:0/804418007 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5d680d0a20 con 0x7f5d78100310 2024-04-16T05:32:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.328+0000 7f5d7659c640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5d600a3bb0 0x7f5d600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.328+0000 7f5d7659c640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5d600a3bb0 0x7f5d600a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c02efd0 tx=0x7f5d6c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.332+0000 7f5d5ffff640 1 -- 172.21.15.32:0/804418007 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d6809d1c0 con 0x7f5d78100310 2024-04-16T05:32:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.522+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d78062c00 con 0x7f5d78100310 2024-04-16T05:32:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.524+0000 7f5d5ffff640 1 -- 172.21.15.32:0/804418007 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7f5d6809cac0 con 0x7f5d78100310 2024-04-16T05:32:47.523 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:47.523 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5d600a3bb0 msgr2=0x7f5d600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5d600a3bb0 0x7f5d600a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c02efd0 tx=0x7f5d6c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78100310 msgr2=0x7f5d78198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78100310 0x7f5d78198390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800e9c0 tx=0x7f5d6800ee80 comp rx=0 tx=0).stop 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 shutdown_connections 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5d600a3bb0 0x7f5d600a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103ca0 0x7f5d7819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d781032e0 0x7f5d781988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 --2- 172.21.15.32:0/804418007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78100310 0x7f5d78198390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 >> 172.21.15.32:0/804418007 conn(0x7f5d780fbc40 msgr2=0x7f5d78106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 shutdown_connections 2024-04-16T05:32:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.526+0000 7f5d7d0ba640 1 -- 172.21.15.32:0/804418007 wait complete. 2024-04-16T05:32:47.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:32:47.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:32:47.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 -- 172.21.15.32:0/1102988721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd8107d20 msgr2=0x7f5fd810a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/1102988721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd8107d20 0x7f5fd810a170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009900 tx=0x7f5fd002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:47.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 -- 172.21.15.32:0/1102988721 shutdown_connections 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/1102988721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd8107d20 0x7f5fd810a170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/1102988721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd81053f0 0x7f5fd81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/1102988721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8102ac0 0x7f5fd8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 -- 172.21.15.32:0/1102988721 >> 172.21.15.32:0/1102988721 conn(0x7f5fd80fc4b0 msgr2=0x7f5fd80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 -- 172.21.15.32:0/1102988721 shutdown_connections 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 -- 172.21.15.32:0/1102988721 wait complete. 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 Processor -- start 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 -- start start 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8102ac0 0x7f5fd8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.648+0000 7f5fe00f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd81053f0 0x7f5fd8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fe00f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8107d20 0x7f5fd8100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fe00f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd810dd20 con 0x7f5fd81053f0 2024-04-16T05:32:47.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fe00f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd810dba0 con 0x7f5fd8107d20 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fe00f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd810dea0 con 0x7f5fd8102ac0 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd81053f0 0x7f5fd8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd81053f0 0x7f5fd8100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58014/0 (socket says 172.21.15.32:58014) 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 -- 172.21.15.32:0/2476236109 learned_addr learned my addr 172.21.15.32:0/2476236109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fde68f640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8107d20 0x7f5fd8100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 -- 172.21.15.32:0/2476236109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8102ac0 msgr2=0x7f5fd8102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdde8e640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8102ac0 0x7f5fd8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8102ac0 0x7f5fd8102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 -- 172.21.15.32:0/2476236109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8107d20 msgr2=0x7f5fd8100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8107d20 0x7f5fd8100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 -- 172.21.15.32:0/2476236109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd8101840 con 0x7f5fd81053f0 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fdd68d640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd81053f0 0x7f5fd8100a70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00e9f0 tx=0x7f5fcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:47.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fd6ffd640 1 -- 172.21.15.32:0/2476236109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00ccc0 con 0x7f5fd81053f0 2024-04-16T05:32:47.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fd6ffd640 1 -- 172.21.15.32:0/2476236109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fcc005500 con 0x7f5fd81053f0 2024-04-16T05:32:47.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd81a2480 con 0x7f5fd81053f0 2024-04-16T05:32:47.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fd6ffd640 1 -- 172.21.15.32:0/2476236109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc010690 con 0x7f5fd81053f0 2024-04-16T05:32:47.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.649+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd81a29d0 con 0x7f5fd81053f0 2024-04-16T05:32:47.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.650+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd80fdfa0 con 0x7f5fd81053f0 2024-04-16T05:32:47.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.651+0000 7f5fd6ffd640 1 -- 172.21.15.32:0/2476236109 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5fcc0046c0 con 0x7f5fd81053f0 2024-04-16T05:32:47.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.652+0000 7f5fd6ffd640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5fc00a3c30 0x7f5fc00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.652+0000 7f5fdde8e640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5fc00a3c30 0x7f5fc00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.652+0000 7f5fd6ffd640 1 -- 172.21.15.32:0/2476236109 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5fcc0d0a90 con 0x7f5fd81053f0 2024-04-16T05:32:47.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.652+0000 7f5fdde8e640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5fc00a3c30 0x7f5fc00a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005fd0 tx=0x7f5fc8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:47.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.657+0000 7f5fd6ffd640 1 -- 172.21.15.32:0/2476236109 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fcc09d1d0 con 0x7f5fd81053f0 2024-04-16T05:32:47.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:47 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/804418007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:47.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.844+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fd8062c00 con 0x7f5fd81053f0 2024-04-16T05:32:47.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.844+0000 7f5fd6ffd640 1 -- 172.21.15.32:0/2476236109 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f5fcc09cad0 con 0x7f5fd81053f0 2024-04-16T05:32:47.843 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5fc00a3c30 msgr2=0x7f5fc00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5fc00a3c30 0x7f5fc00a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005fd0 tx=0x7f5fc8009500 comp rx=0 tx=0).stop 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd81053f0 msgr2=0x7f5fd8100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd81053f0 0x7f5fd8100a70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00e9f0 tx=0x7f5fcc00eeb0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 shutdown_connections 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5fc00a3c30 0x7f5fc00a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8107d20 0x7f5fd8100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd81053f0 0x7f5fd8100a70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 --2- 172.21.15.32:0/2476236109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8102ac0 0x7f5fd8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 >> 172.21.15.32:0/2476236109 conn(0x7f5fd80fc4b0 msgr2=0x7f5fd8103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 shutdown_connections 2024-04-16T05:32:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.846+0000 7f5fe00f2640 1 -- 172.21.15.32:0/2476236109 wait complete. 2024-04-16T05:32:47.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:32:47.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:47 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/804418007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 -- 172.21.15.32:0/2051073837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102090 msgr2=0x7f89a8102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 --2- 172.21.15.32:0/2051073837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102090 0x7f89a8102510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8990009900 tx=0x7f899002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 -- 172.21.15.32:0/2051073837 shutdown_connections 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 --2- 172.21.15.32:0/2051073837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89a8102a50 0x7f89a8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 --2- 172.21.15.32:0/2051073837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102090 0x7f89a8102510 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 --2- 172.21.15.32:0/2051073837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89a8100f40 0x7f89a8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 -- 172.21.15.32:0/2051073837 >> 172.21.15.32:0/2051073837 conn(0x7f89a80fc4b0 msgr2=0x7f89a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 -- 172.21.15.32:0/2051073837 shutdown_connections 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 -- 172.21.15.32:0/2051073837 wait complete. 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 Processor -- start 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.966+0000 7f89ae73d640 1 -- start start 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89ae73d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89a8100f40 0x7f89a8069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89ae73d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89a8102090 0x7f89a819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89ae73d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102a50 0x7f89a819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89ae73d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89a810cbd0 con 0x7f89a8102a50 2024-04-16T05:32:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89ae73d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f89a810ca50 con 0x7f89a8102090 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89ae73d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89a810cd50 con 0x7f89a8100f40 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102a50 0x7f89a819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102a50 0x7f89a819f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58034/0 (socket says 172.21.15.32:58034) 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 -- 172.21.15.32:0/4120067931 learned_addr learned my addr 172.21.15.32:0/4120067931 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f899ffff640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89a8102090 0x7f89a819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89a7fff640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89a8100f40 0x7f89a8069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 -- 172.21.15.32:0/4120067931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89a8100f40 msgr2=0x7f89a8069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89a8100f40 0x7f89a8069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 -- 172.21.15.32:0/4120067931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89a8102090 msgr2=0x7f89a819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89a8102090 0x7f89a819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 -- 172.21.15.32:0/4120067931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a819ff30 con 0x7f89a8102a50 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89a7fff640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89a8100f40 0x7f89a8069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89accda640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102a50 0x7f89a819f620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f89a000cce0 tx=0x7f89a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89a5ffb640 1 -- 172.21.15.32:0/4120067931 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a000f450 con 0x7f89a8102a50 2024-04-16T05:32:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89a5ffb640 1 -- 172.21.15.32:0/4120067931 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a000fad0 con 0x7f89a8102a50 2024-04-16T05:32:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a81a01c0 con 0x7f89a8102a50 2024-04-16T05:32:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.967+0000 7f89a5ffb640 1 -- 172.21.15.32:0/4120067931 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0018c10 con 0x7f89a8102a50 2024-04-16T05:32:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.968+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a81a0690 con 0x7f89a8102a50 2024-04-16T05:32:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.968+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a8108250 con 0x7f89a8102a50 2024-04-16T05:32:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.970+0000 7f89a5ffb640 1 -- 172.21.15.32:0/4120067931 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f89a000f5f0 con 0x7f89a8102a50 2024-04-16T05:32:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.970+0000 7f89a5ffb640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f89940a3bb0 0x7f89940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.970+0000 7f89a7fff640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f89940a3bb0 0x7f89940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.970+0000 7f89a5ffb640 1 -- 172.21.15.32:0/4120067931 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f89a00d0d80 con 0x7f89a8102a50 2024-04-16T05:32:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.971+0000 7f89a7fff640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f89940a3bb0 0x7f89940a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f899800abe0 tx=0x7f899800a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:47.974+0000 7f89a5ffb640 1 -- 172.21.15.32:0/4120067931 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89a0014030 con 0x7f89a8102a50 2024-04-16T05:32:48.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:47 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/804418007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.160+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89a802d7b0 con 0x7f89a8102a50 2024-04-16T05:32:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.161+0000 7f89a5ffb640 1 -- 172.21.15.32:0/4120067931 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f89a009d5b0 con 0x7f89a8102a50 2024-04-16T05:32:48.159 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:32:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.162+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f89940a3bb0 msgr2=0x7f89940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.162+0000 7f89ae73d640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f89940a3bb0 0x7f89940a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f899800abe0 tx=0x7f899800a5c0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.162+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102a50 msgr2=0x7f89a819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.162+0000 7f89ae73d640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102a50 0x7f89a819f620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f89a000cce0 tx=0x7f89a000a620 comp rx=0 tx=0).stop 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.162+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 shutdown_connections 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.162+0000 7f89ae73d640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f89940a3bb0 0x7f89940a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.162+0000 7f89ae73d640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a8102a50 0x7f89a819f620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.162+0000 7f89ae73d640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89a8102090 0x7f89a819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.163+0000 7f89ae73d640 1 --2- 172.21.15.32:0/4120067931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89a8100f40 0x7f89a8069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.163+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 >> 172.21.15.32:0/4120067931 conn(0x7f89a80fc4b0 msgr2=0x7f89a8105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.163+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 shutdown_connections 2024-04-16T05:32:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.163+0000 7f89ae73d640 1 -- 172.21.15.32:0/4120067931 wait complete. 2024-04-16T05:32:48.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 -- 172.21.15.32:0/1385233525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a8101210 msgr2=0x7f04a8101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1385233525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a8101210 0x7f04a8101690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0498009900 tx=0x7f049802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 -- 172.21.15.32:0/1385233525 shutdown_connections 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1385233525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8101bd0 0x7f04a8109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1385233525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a8101210 0x7f04a8101690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1385233525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a81024e0 0x7f04a8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 -- 172.21.15.32:0/1385233525 >> 172.21.15.32:0/1385233525 conn(0x7f04a80fbc40 msgr2=0x7f04a80fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 -- 172.21.15.32:0/1385233525 shutdown_connections 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.282+0000 7f04af9cd640 1 -- 172.21.15.32:0/1385233525 wait complete. 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04af9cd640 1 Processor -- start 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04af9cd640 1 -- start start 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04af9cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8101210 0x7f04a81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04af9cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8101bd0 0x7f04a8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04af9cd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a81024e0 0x7f04a819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04af9cd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04a810cbe0 con 0x7f04a81024e0 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04af9cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f04a810ca60 con 0x7f04a8101210 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04af9cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04a810cd60 con 0x7f04a8101bd0 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04adf6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a81024e0 0x7f04a819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04adf6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a81024e0 0x7f04a819f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58052/0 (socket says 172.21.15.32:58052) 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04adf6a640 1 -- 172.21.15.32:0/1516489932 learned_addr learned my addr 172.21.15.32:0/1516489932 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04ad769640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8101210 0x7f04a81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04acf68640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8101bd0 0x7f04a8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04adf6a640 1 -- 172.21.15.32:0/1516489932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8101bd0 msgr2=0x7f04a8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04adf6a640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8101bd0 0x7f04a8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04adf6a640 1 -- 172.21.15.32:0/1516489932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8101210 msgr2=0x7f04a81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04adf6a640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8101210 0x7f04a81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.283+0000 7f04adf6a640 1 -- 172.21.15.32:0/1516489932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a81a0000 con 0x7f04a81024e0 2024-04-16T05:32:48.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.284+0000 7f04adf6a640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a81024e0 0x7f04a819f990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04a000eb10 tx=0x7f04a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.284+0000 7f049e7fc640 1 -- 172.21.15.32:0/1516489932 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0004b20 con 0x7f04a81024e0 2024-04-16T05:32:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.284+0000 7f049e7fc640 1 -- 172.21.15.32:0/1516489932 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04a0004cc0 con 0x7f04a81024e0 2024-04-16T05:32:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.284+0000 7f049e7fc640 1 -- 172.21.15.32:0/1516489932 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0010760 con 0x7f04a81024e0 2024-04-16T05:32:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.284+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a81a02f0 con 0x7f04a81024e0 2024-04-16T05:32:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.284+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a81a0810 con 0x7f04a81024e0 2024-04-16T05:32:48.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.285+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a81a0ad0 con 0x7f04a81024e0 2024-04-16T05:32:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.286+0000 7f049e7fc640 1 -- 172.21.15.32:0/1516489932 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f04a0005020 con 0x7f04a81024e0 2024-04-16T05:32:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.286+0000 7f049e7fc640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f04940a3c30 0x7f04940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.287+0000 7f049e7fc640 1 -- 172.21.15.32:0/1516489932 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f04a00d0be0 con 0x7f04a81024e0 2024-04-16T05:32:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.287+0000 7f04ad769640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f04940a3c30 0x7f04940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.287+0000 7f04ad769640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f04940a3c30 0x7f04940a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04a4005fd0 tx=0x7f04a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.291+0000 7f049e7fc640 1 -- 172.21.15.32:0/1516489932 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04a009d380 con 0x7f04a81024e0 2024-04-16T05:32:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.477+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04a81a0ca0 con 0x7f04a81024e0 2024-04-16T05:32:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.477+0000 7f049e7fc640 1 -- 172.21.15.32:0/1516489932 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f04a009cc80 con 0x7f04a81024e0 2024-04-16T05:32:48.477 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f04940a3c30 msgr2=0x7f04940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f04940a3c30 0x7f04940a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04a4005fd0 tx=0x7f04a400b390 comp rx=0 tx=0).stop 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a81024e0 msgr2=0x7f04a819f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a81024e0 0x7f04a819f990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04a000eb10 tx=0x7f04a000c2d0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 shutdown_connections 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f04940a3c30 0x7f04940a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a81024e0 0x7f04a819f990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8101bd0 0x7f04a8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 --2- 172.21.15.32:0/1516489932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8101210 0x7f04a81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 >> 172.21.15.32:0/1516489932 conn(0x7f04a80fbc40 msgr2=0x7f04a80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 shutdown_connections 2024-04-16T05:32:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.479+0000 7f04af9cd640 1 -- 172.21.15.32:0/1516489932 wait complete. 2024-04-16T05:32:48.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 -- 172.21.15.32:0/1212787200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e81033a0 msgr2=0x7fd5e8103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1212787200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e81033a0 0x7fd5e8103820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009930 tx=0x7fd5d802bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 -- 172.21.15.32:0/1212787200 shutdown_connections 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1212787200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e8103d60 0x7fd5e810a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1212787200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e81033a0 0x7fd5e8103820 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1212787200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e8100330 0x7fd5e8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 -- 172.21.15.32:0/1212787200 >> 172.21.15.32:0/1212787200 conn(0x7fd5e80fbc40 msgr2=0x7fd5e80fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 -- 172.21.15.32:0/1212787200 shutdown_connections 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.599+0000 7fd5f0055640 1 -- 172.21.15.32:0/1212787200 wait complete. 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5f0055640 1 Processor -- start 2024-04-16T05:32:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5f0055640 1 -- start start 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5f0055640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e8100330 0x7fd5e81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5f0055640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e81033a0 0x7fd5e8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5f0055640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e8103d60 0x7fd5e819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5f0055640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810dcf0 con 0x7fd5e8103d60 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5f0055640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810db70 con 0x7fd5e81033a0 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5f0055640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810de70 con 0x7fd5e8100330 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ee5f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e8103d60 0x7fd5e819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ee5f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e8103d60 0x7fd5e819f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52364/0 (socket says 172.21.15.32:52364) 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ee5f2640 1 -- 172.21.15.32:0/1596001776 learned_addr learned my addr 172.21.15.32:0/1596001776 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5eddf1640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e8100330 0x7fd5e81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ed5f0640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e81033a0 0x7fd5e8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ee5f2640 1 -- 172.21.15.32:0/1596001776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e8100330 msgr2=0x7fd5e81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ee5f2640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e8100330 0x7fd5e81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ee5f2640 1 -- 172.21.15.32:0/1596001776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e81033a0 msgr2=0x7fd5e8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ee5f2640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e81033a0 0x7fd5e8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ee5f2640 1 -- 172.21.15.32:0/1596001776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e81a0010 con 0x7fd5e8103d60 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5ed5f0640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e81033a0 0x7fd5e8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.600+0000 7fd5eddf1640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e8100330 0x7fd5e81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.601+0000 7fd5ee5f2640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e8103d60 0x7fd5e819f9a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009fd0 tx=0x7fd5e000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.601+0000 7fd5deffd640 1 -- 172.21.15.32:0/1596001776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0017070 con 0x7fd5e8103d60 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.601+0000 7fd5deffd640 1 -- 172.21.15.32:0/1596001776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e0003e50 con 0x7fd5e8103d60 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.601+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e81a0300 con 0x7fd5e8103d60 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.601+0000 7fd5deffd640 1 -- 172.21.15.32:0/1596001776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00106c0 con 0x7fd5e8103d60 2024-04-16T05:32:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.601+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81a0800 con 0x7fd5e8103d60 2024-04-16T05:32:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.602+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e8109540 con 0x7fd5e8103d60 2024-04-16T05:32:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.603+0000 7fd5deffd640 1 -- 172.21.15.32:0/1596001776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd5e0003970 con 0x7fd5e8103d60 2024-04-16T05:32:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.603+0000 7fd5deffd640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd5d40a3bb0 0x7fd5d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.604+0000 7fd5eddf1640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd5d40a3bb0 0x7fd5d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.604+0000 7fd5deffd640 1 -- 172.21.15.32:0/1596001776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd5e00d04e0 con 0x7fd5e8103d60 2024-04-16T05:32:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.604+0000 7fd5eddf1640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd5d40a3bb0 0x7fd5d40a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd5e806a380 tx=0x7fd5e400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:48.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.608+0000 7fd5deffd640 1 -- 172.21.15.32:0/1596001776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5e009cc20 con 0x7fd5e8103d60 2024-04-16T05:32:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.796+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd5e8062c00 con 0x7fd5e8103d60 2024-04-16T05:32:48.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.796+0000 7fd5deffd640 1 -- 172.21.15.32:0/1596001776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fd5e009c520 con 0x7fd5e8103d60 2024-04-16T05:32:48.796 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:32:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd5d40a3bb0 msgr2=0x7fd5d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd5d40a3bb0 0x7fd5d40a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd5e806a380 tx=0x7fd5e400a560 comp rx=0 tx=0).stop 2024-04-16T05:32:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e8103d60 msgr2=0x7fd5e819f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e8103d60 0x7fd5e819f9a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009fd0 tx=0x7fd5e000b260 comp rx=0 tx=0).stop 2024-04-16T05:32:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 shutdown_connections 2024-04-16T05:32:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd5d40a3bb0 0x7fd5d40a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e8103d60 0x7fd5e819f9a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e81033a0 0x7fd5e8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 --2- 172.21.15.32:0/1596001776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e8100330 0x7fd5e81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.798+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 >> 172.21.15.32:0/1596001776 conn(0x7fd5e80fbc40 msgr2=0x7fd5e8106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:48.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.799+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 shutdown_connections 2024-04-16T05:32:48.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.799+0000 7fd5f0055640 1 -- 172.21.15.32:0/1596001776 wait complete. 2024-04-16T05:32:48.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:48 smithi032 ceph-mon[32351]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-16T05:32:48.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:48 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2476236109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:48.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:48 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4120067931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:32:48.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:48 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1516489932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:32:48.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T05:32:48.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:48 smithi082 ceph-mon[36195]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-16T05:32:48.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:48 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2476236109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:48.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:48 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4120067931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:32:48.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:48 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1516489932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:32:48.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 -- 172.21.15.32:0/3197321174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 msgr2=0x7fc3301036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 --2- 172.21.15.32:0/3197321174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 0x7fc3301036c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc324009900 tx=0x7fc32402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:48.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 -- 172.21.15.32:0/3197321174 shutdown_connections 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 --2- 172.21.15.32:0/3197321174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330103c00 0x7fc33010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 --2- 172.21.15.32:0/3197321174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 0x7fc3301036c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 --2- 172.21.15.32:0/3197321174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc330102050 0x7fc330102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 -- 172.21.15.32:0/3197321174 >> 172.21.15.32:0/3197321174 conn(0x7fc3300fd5e0 msgr2=0x7fc3300ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 -- 172.21.15.32:0/3197321174 shutdown_connections 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 -- 172.21.15.32:0/3197321174 wait complete. 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 Processor -- start 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.917+0000 7fc335dd5640 1 -- start start 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc335dd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330102050 0x7fc33019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc335dd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 0x7fc33019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc335dd5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc330103c00 0x7fc3301a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc335dd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010dd20 con 0x7fc330103240 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc335dd5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010dba0 con 0x7fc330103c00 2024-04-16T05:32:48.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc335dd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010dea0 con 0x7fc330102050 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 0x7fc33019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 0x7fc33019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52380/0 (socket says 172.21.15.32:52380) 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 -- 172.21.15.32:0/1185485375 learned_addr learned my addr 172.21.15.32:0/1185485375 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32ffff640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc330103c00 0x7fc3301a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32f7fe640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330102050 0x7fc33019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 -- 172.21.15.32:0/1185485375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330102050 msgr2=0x7fc33019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330102050 0x7fc33019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 -- 172.21.15.32:0/1185485375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc330103c00 msgr2=0x7fc3301a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc330103c00 0x7fc3301a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 -- 172.21.15.32:0/1185485375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3301a21d0 con 0x7fc330103240 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32ffff640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc330103c00 0x7fc3301a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32effd640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 0x7fc33019aaf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc324009230 tx=0x7fc324004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32cff9640 1 -- 172.21.15.32:0/1185485375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324041070 con 0x7fc330103240 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32cff9640 1 -- 172.21.15.32:0/1185485375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3240054e0 con 0x7fc330103240 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3301a2460 con 0x7fc330103240 2024-04-16T05:32:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.918+0000 7fc32cff9640 1 -- 172.21.15.32:0/1185485375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3240346a0 con 0x7fc330103240 2024-04-16T05:32:48.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.919+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3301a2950 con 0x7fc330103240 2024-04-16T05:32:48.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.919+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3301a2bb0 con 0x7fc330103240 2024-04-16T05:32:48.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.920+0000 7fc32cff9640 1 -- 172.21.15.32:0/1185485375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc324044070 con 0x7fc330103240 2024-04-16T05:32:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.921+0000 7fc32cff9640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc3180a3bb0 0x7fc3180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.921+0000 7fc32cff9640 1 -- 172.21.15.32:0/1185485375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc3240f2480 con 0x7fc330103240 2024-04-16T05:32:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.921+0000 7fc32f7fe640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc3180a3bb0 0x7fc3180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.922+0000 7fc32f7fe640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc3180a3bb0 0x7fc3180a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc320005fd0 tx=0x7fc320005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:48.926+0000 7fc32cff9640 1 -- 172.21.15.32:0/1185485375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3240bec20 con 0x7fc330103240 2024-04-16T05:32:49.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.124+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc3301a2d80 con 0x7fc330103240 2024-04-16T05:32:49.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.125+0000 7fc32cff9640 1 -- 172.21.15.32:0/1185485375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7fc3240be520 con 0x7fc330103240 2024-04-16T05:32:49.124 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2557553342 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:49.124 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1590295441 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:49.124 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1590295441 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:49.124 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2929016634 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:49.125 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/26695350 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:49.125 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3665169244 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:49.125 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/3517252554 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:49.125 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1771946956 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:49.125 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/3517252554 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:49.125 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1006904605 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:listed 10 entries 2024-04-16T05:32:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc3180a3bb0 msgr2=0x7fc3180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc3180a3bb0 0x7fc3180a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc320005fd0 tx=0x7fc320005ce0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 msgr2=0x7fc33019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 0x7fc33019aaf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc324009230 tx=0x7fc324004040 comp rx=0 tx=0).stop 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 shutdown_connections 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc3180a3bb0 0x7fc3180a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc330103c00 0x7fc3301a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc330103240 0x7fc33019aaf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.126+0000 7fc335dd5640 1 --2- 172.21.15.32:0/1185485375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330102050 0x7fc33019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.127+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 >> 172.21.15.32:0/1185485375 conn(0x7fc3300fd5e0 msgr2=0x7fc330074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.127+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 shutdown_connections 2024-04-16T05:32:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.127+0000 7fc335dd5640 1 -- 172.21.15.32:0/1185485375 wait complete. 2024-04-16T05:32:49.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:48 smithi134 ceph-mon[36153]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-16T05:32:49.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:48 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2476236109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:49.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:48 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4120067931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:32:49.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:48 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1516489932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:32:49.137 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:32:49.137 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:32:49.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2379613049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0100f00 msgr2=0x7fb6f0101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2379613049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0100f00 0x7fb6f0101340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009900 tx=0x7fb6e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2379613049 shutdown_connections 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2379613049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0102a90 0x7fb6f01093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2379613049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f01020d0 0x7fb6f0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2379613049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0100f00 0x7fb6f0101340 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2379613049 >> 172.21.15.32:0/2379613049 conn(0x7fb6f00fc4e0 msgr2=0x7fb6f00fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2379613049 shutdown_connections 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2379613049 wait complete. 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.300+0000 7fb6f6e74640 1 Processor -- start 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f6e74640 1 -- start start 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f6e74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f0100f00 0x7fb6f019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f6e74640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f01020d0 0x7fb6f019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f6e74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0102a90 0x7fb6f01a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f6e74640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010cbd0 con 0x7fb6f0102a90 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f6e74640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010ca50 con 0x7fb6f01020d0 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f6e74640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010cd50 con 0x7fb6f0100f00 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0102a90 0x7fb6f01a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0102a90 0x7fb6f01a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52398/0 (socket says 172.21.15.32:52398) 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 -- 172.21.15.32:0/2791542211 learned_addr learned my addr 172.21.15.32:0/2791542211 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f4c10640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f0100f00 0x7fb6f019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6effff640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f01020d0 0x7fb6f019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 -- 172.21.15.32:0/2791542211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f0100f00 msgr2=0x7fb6f019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f0100f00 0x7fb6f019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 -- 172.21.15.32:0/2791542211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f01020d0 msgr2=0x7fb6f019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f01020d0 0x7fb6f019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 -- 172.21.15.32:0/2791542211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f01a2190 con 0x7fb6f0102a90 2024-04-16T05:32:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f4c10640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f0100f00 0x7fb6f019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6f5411640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0102a90 0x7fb6f01a1b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800cce0 tx=0x7fb6e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.301+0000 7fb6edffb640 1 -- 172.21.15.32:0/2791542211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800f450 con 0x7fb6f0102a90 2024-04-16T05:32:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.302+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01a2480 con 0x7fb6f0102a90 2024-04-16T05:32:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.302+0000 7fb6edffb640 1 -- 172.21.15.32:0/2791542211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e800fad0 con 0x7fb6f0102a90 2024-04-16T05:32:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.302+0000 7fb6edffb640 1 -- 172.21.15.32:0/2791542211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8018c10 con 0x7fb6f0102a90 2024-04-16T05:32:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.302+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a29d0 con 0x7fb6f0102a90 2024-04-16T05:32:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.303+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f01a2c90 con 0x7fb6f0102a90 2024-04-16T05:32:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.304+0000 7fb6edffb640 1 -- 172.21.15.32:0/2791542211 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6e800f5f0 con 0x7fb6f0102a90 2024-04-16T05:32:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.304+0000 7fb6edffb640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.304+0000 7fb6edffb640 1 -- 172.21.15.32:0/2791542211 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb6e80d0da0 con 0x7fb6f0102a90 2024-04-16T05:32:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.304+0000 7fb6f4c10640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.305+0000 7fb6f4c10640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb6e402c7b0 tx=0x7fb6e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:49.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.309+0000 7fb6edffb640 1 -- 172.21.15.32:0/2791542211 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6e8014030 con 0x7fb6f0102a90 2024-04-16T05:32:49.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.497+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6f01a2f00 con 0x7fb6f0102a90 2024-04-16T05:32:49.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.498+0000 7fb6edffb640 1 -- 172.21.15.32:0/2791542211 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7fb6e809d540 con 0x7fb6f0102a90 2024-04-16T05:32:49.498 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:49.498 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6e00a3bb0 msgr2=0x7fb6e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb6e402c7b0 tx=0x7fb6e4005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0102a90 msgr2=0x7fb6f01a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0102a90 0x7fb6f01a1b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800cce0 tx=0x7fb6e800a620 comp rx=0 tx=0).stop 2024-04-16T05:32:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 shutdown_connections 2024-04-16T05:32:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f0102a90 0x7fb6f01a1b20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f01020d0 0x7fb6f019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 --2- 172.21.15.32:0/2791542211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f0100f00 0x7fb6f019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 >> 172.21.15.32:0/2791542211 conn(0x7fb6f00fc4e0 msgr2=0x7fb6f00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 shutdown_connections 2024-04-16T05:32:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.500+0000 7fb6f6e74640 1 -- 172.21.15.32:0/2791542211 wait complete. 2024-04-16T05:32:49.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:32:49.512 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/400141507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 msgr2=0x7fdc2810a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/400141507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 0x7fdc2810a1b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009ca0 tx=0x7fdc2002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/400141507 shutdown_connections 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/400141507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 0x7fdc2810a1b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/400141507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28105490 0x7fdc28107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/400141507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc28102b60 0x7fdc28104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/400141507 >> 172.21.15.32:0/400141507 conn(0x7fdc280fc4f0 msgr2=0x7fdc280fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/400141507 shutdown_connections 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.621+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/400141507 wait complete. 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2ccd9640 1 Processor -- start 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2ccd9640 1 -- start start 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2ccd9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc28102b60 0x7fdc281982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2ccd9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28105490 0x7fdc28198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2ccd9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 0x7fdc2819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2ccd9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2810dcf0 con 0x7fdc28107dc0 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2ccd9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2810db70 con 0x7fdc28105490 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2ccd9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2810de70 con 0x7fdc28102b60 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 0x7fdc2819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 0x7fdc2819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52412/0 (socket says 172.21.15.32:52412) 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 -- 172.21.15.32:0/1164611596 learned_addr learned my addr 172.21.15.32:0/1164611596 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc2659c640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc28102b60 0x7fdc281982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc1ffff640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28105490 0x7fdc28198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 -- 172.21.15.32:0/1164611596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc28102b60 msgr2=0x7fdc281982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc28102b60 0x7fdc281982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 -- 172.21.15.32:0/1164611596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28105490 msgr2=0x7fdc28198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28105490 0x7fdc28198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 -- 172.21.15.32:0/1164611596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2819ff10 con 0x7fdc28107dc0 2024-04-16T05:32:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.622+0000 7fdc26d9d640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 0x7fdc2819f8a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdc2002c700 tx=0x7fdc20032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.623+0000 7fdc1f7fe640 1 -- 172.21.15.32:0/1164611596 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20039040 con 0x7fdc28107dc0 2024-04-16T05:32:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.623+0000 7fdc1f7fe640 1 -- 172.21.15.32:0/1164611596 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc20044070 con 0x7fdc28107dc0 2024-04-16T05:32:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.623+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc281a01a0 con 0x7fdc28107dc0 2024-04-16T05:32:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.623+0000 7fdc1f7fe640 1 -- 172.21.15.32:0/1164611596 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200042a0 con 0x7fdc28107dc0 2024-04-16T05:32:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.623+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc281a0690 con 0x7fdc28107dc0 2024-04-16T05:32:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.624+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc280fdfe0 con 0x7fdc28107dc0 2024-04-16T05:32:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.625+0000 7fdc1f7fe640 1 -- 172.21.15.32:0/1164611596 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdc20003970 con 0x7fdc28107dc0 2024-04-16T05:32:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.626+0000 7fdc1f7fe640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdc140a3bb0 0x7fdc140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.626+0000 7fdc2659c640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdc140a3bb0 0x7fdc140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.626+0000 7fdc1f7fe640 1 -- 172.21.15.32:0/1164611596 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fdc200f2b70 con 0x7fdc28107dc0 2024-04-16T05:32:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.626+0000 7fdc2659c640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdc140a3bb0 0x7fdc140a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc180058c0 tx=0x7fdc1800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.631+0000 7fdc1f7fe640 1 -- 172.21.15.32:0/1164611596 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc200bf3a0 con 0x7fdc28107dc0 2024-04-16T05:32:49.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.817+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc28062c00 con 0x7fdc28107dc0 2024-04-16T05:32:49.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.818+0000 7fdc1f7fe640 1 -- 172.21.15.32:0/1164611596 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fdc200beca0 con 0x7fdc28107dc0 2024-04-16T05:32:49.817 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:32:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdc140a3bb0 msgr2=0x7fdc140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdc140a3bb0 0x7fdc140a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc180058c0 tx=0x7fdc1800a2d0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 msgr2=0x7fdc2819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 0x7fdc2819f8a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdc2002c700 tx=0x7fdc20032f20 comp rx=0 tx=0).stop 2024-04-16T05:32:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 shutdown_connections 2024-04-16T05:32:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fdc140a3bb0 0x7fdc140a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc28107dc0 0x7fdc2819f8a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28105490 0x7fdc28198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 --2- 172.21.15.32:0/1164611596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc28102b60 0x7fdc281982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 >> 172.21.15.32:0/1164611596 conn(0x7fdc280fc4f0 msgr2=0x7fdc281034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 shutdown_connections 2024-04-16T05:32:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.820+0000 7fdc2ccd9640 1 -- 172.21.15.32:0/1164611596 wait complete. 2024-04-16T05:32:49.830 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:49 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1596001776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:32:49.830 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:49 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1185485375' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:32:49.830 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:49 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2791542211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:49.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:32:49.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:49 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1596001776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:32:49.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:49 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1185485375' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:32:49.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:49 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2791542211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:49.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.940+0000 7f52d457c640 1 -- 172.21.15.32:0/3488905142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc107d80 msgr2=0x7f52cc10a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.940+0000 7f52d457c640 1 --2- 172.21.15.32:0/3488905142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc107d80 0x7f52cc10a170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009ca0 tx=0x7f52c402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 -- 172.21.15.32:0/3488905142 shutdown_connections 2024-04-16T05:32:49.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 --2- 172.21.15.32:0/3488905142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc107d80 0x7f52cc10a170 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 --2- 172.21.15.32:0/3488905142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52cc105450 0x7f52cc107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 --2- 172.21.15.32:0/3488905142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52cc102b20 0x7f52cc104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 -- 172.21.15.32:0/3488905142 >> 172.21.15.32:0/3488905142 conn(0x7f52cc0fc4b0 msgr2=0x7f52cc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 -- 172.21.15.32:0/3488905142 shutdown_connections 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 -- 172.21.15.32:0/3488905142 wait complete. 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 Processor -- start 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 -- start start 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52cc102b20 0x7f52cc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc105450 0x7f52cc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52cc107d80 0x7f52cc19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc10dcd0 con 0x7f52cc105450 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc10db50 con 0x7f52cc102b20 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d457c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc10de50 con 0x7f52cc107d80 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d1b17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc105450 0x7f52cc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d1b17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc105450 0x7f52cc1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52430/0 (socket says 172.21.15.32:52430) 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.941+0000 7f52d1b17640 1 -- 172.21.15.32:0/1369473188 learned_addr learned my addr 172.21.15.32:0/1369473188 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d2318640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52cc102b20 0x7f52cc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d1b17640 1 -- 172.21.15.32:0/1369473188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52cc107d80 msgr2=0x7f52cc19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d1b17640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52cc107d80 0x7f52cc19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d1b17640 1 -- 172.21.15.32:0/1369473188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52cc102b20 msgr2=0x7f52cc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d1b17640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52cc102b20 0x7f52cc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d1b17640 1 -- 172.21.15.32:0/1369473188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc19ff90 con 0x7f52cc105450 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d1b17640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc105450 0x7f52cc1988b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c970 tx=0x7f52bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52c37fe640 1 -- 172.21.15.32:0/1369473188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc007bb0 con 0x7f52cc105450 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc1a0280 con 0x7f52cc105450 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52c37fe640 1 -- 172.21.15.32:0/1369473188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52bc007d50 con 0x7f52cc105450 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52c37fe640 1 -- 172.21.15.32:0/1369473188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0187a0 con 0x7f52cc105450 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.942+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc1a07d0 con 0x7f52cc105450 2024-04-16T05:32:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.943+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc0fdfa0 con 0x7f52cc105450 2024-04-16T05:32:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.944+0000 7f52c37fe640 1 -- 172.21.15.32:0/1369473188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f52bc00f450 con 0x7f52cc105450 2024-04-16T05:32:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.945+0000 7f52c37fe640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52b80a3bb0 0x7f52b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.945+0000 7f52c37fe640 1 -- 172.21.15.32:0/1369473188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f52bc0d0780 con 0x7f52cc105450 2024-04-16T05:32:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.945+0000 7f52d2318640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52b80a3bb0 0x7f52b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.945+0000 7f52d2318640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52b80a3bb0 0x7f52b80a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:49.949+0000 7f52c37fe640 1 -- 172.21.15.32:0/1369473188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52bc014030 con 0x7f52cc105450 2024-04-16T05:32:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:49 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1596001776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:32:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:49 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1185485375' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:32:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:49 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2791542211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:50.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.137+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52cc062c00 con 0x7f52cc105450 2024-04-16T05:32:50.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.137+0000 7f52c37fe640 1 -- 172.21.15.32:0/1369473188 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f52bc09c870 con 0x7f52cc105450 2024-04-16T05:32:50.137 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52b80a3bb0 msgr2=0x7f52b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52b80a3bb0 0x7f52b80a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c8005850 comp rx=0 tx=0).stop 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc105450 msgr2=0x7f52cc1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc105450 0x7f52cc1988b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c970 tx=0x7f52bc00ce30 comp rx=0 tx=0).stop 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 shutdown_connections 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52b80a3bb0 0x7f52b80a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52cc107d80 0x7f52cc19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc105450 0x7f52cc1988b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 --2- 172.21.15.32:0/1369473188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52cc102b20 0x7f52cc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 >> 172.21.15.32:0/1369473188 conn(0x7f52cc0fc4b0 msgr2=0x7f52cc103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:50.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 shutdown_connections 2024-04-16T05:32:50.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.139+0000 7f52d457c640 1 -- 172.21.15.32:0/1369473188 wait complete. 2024-04-16T05:32:50.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:32:50.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 -- 172.21.15.32:0/4173635646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88102a50 msgr2=0x7fba88109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 --2- 172.21.15.32:0/4173635646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88102a50 0x7fba88109370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba80009ca0 tx=0x7fba8002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 -- 172.21.15.32:0/4173635646 shutdown_connections 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 --2- 172.21.15.32:0/4173635646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88102a50 0x7fba88109370 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 --2- 172.21.15.32:0/4173635646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba88102090 0x7fba88102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 --2- 172.21.15.32:0/4173635646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba88100f40 0x7fba88101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 -- 172.21.15.32:0/4173635646 >> 172.21.15.32:0/4173635646 conn(0x7fba880fc4b0 msgr2=0x7fba880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 -- 172.21.15.32:0/4173635646 shutdown_connections 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.259+0000 7fba8ec74640 1 -- 172.21.15.32:0/4173635646 wait complete. 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ec74640 1 Processor -- start 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ec74640 1 -- start start 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ec74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88100f40 0x7fba881982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ec74640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba88102090 0x7fba88198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ec74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba88102a50 0x7fba8819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ec74640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba8810cbd0 con 0x7fba88100f40 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ec74640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fba8810ca50 con 0x7fba88102090 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ec74640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fba8810cd50 con 0x7fba88102a50 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ca10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88100f40 0x7fba881982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ca10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88100f40 0x7fba881982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52448/0 (socket says 172.21.15.32:52448) 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8ca10640 1 -- 172.21.15.32:0/3177201281 learned_addr learned my addr 172.21.15.32:0/3177201281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba87fff640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba88102090 0x7fba88198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.260+0000 7fba8d211640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba88102a50 0x7fba8819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8ca10640 1 -- 172.21.15.32:0/3177201281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba88102a50 msgr2=0x7fba8819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8ca10640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba88102a50 0x7fba8819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8ca10640 1 -- 172.21.15.32:0/3177201281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba88102090 msgr2=0x7fba88198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8ca10640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba88102090 0x7fba88198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8ca10640 1 -- 172.21.15.32:0/3177201281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8819fee0 con 0x7fba88100f40 2024-04-16T05:32:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8d211640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba88102a50 0x7fba8819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8ca10640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88100f40 0x7fba881982c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00d950 tx=0x7fba7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba85ffb640 1 -- 172.21.15.32:0/3177201281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c017070 con 0x7fba88100f40 2024-04-16T05:32:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba881a01d0 con 0x7fba88100f40 2024-04-16T05:32:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba85ffb640 1 -- 172.21.15.32:0/3177201281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba7c009c30 con 0x7fba88100f40 2024-04-16T05:32:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba85ffb640 1 -- 172.21.15.32:0/3177201281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00f6b0 con 0x7fba88100f40 2024-04-16T05:32:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.261+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba881a0720 con 0x7fba88100f40 2024-04-16T05:32:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.262+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba881a0980 con 0x7fba88100f40 2024-04-16T05:32:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.263+0000 7fba85ffb640 1 -- 172.21.15.32:0/3177201281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fba7c0109d0 con 0x7fba88100f40 2024-04-16T05:32:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.264+0000 7fba85ffb640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fba780a3bb0 0x7fba780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.264+0000 7fba87fff640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fba780a3bb0 0x7fba780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.264+0000 7fba85ffb640 1 -- 172.21.15.32:0/3177201281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fba7c0d02d0 con 0x7fba88100f40 2024-04-16T05:32:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.264+0000 7fba87fff640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fba780a3bb0 0x7fba780a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba881997d0 tx=0x7fba74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:50.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.268+0000 7fba85ffb640 1 -- 172.21.15.32:0/3177201281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba7c009de0 con 0x7fba88100f40 2024-04-16T05:32:50.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.456+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba881a0b50 con 0x7fba88100f40 2024-04-16T05:32:50.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.457+0000 7fba85ffb640 1 -- 172.21.15.32:0/3177201281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fba7c09ca10 con 0x7fba88100f40 2024-04-16T05:32:50.456 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:32:50.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fba780a3bb0 msgr2=0x7fba780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fba780a3bb0 0x7fba780a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba881997d0 tx=0x7fba74005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88100f40 msgr2=0x7fba881982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88100f40 0x7fba881982c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00d950 tx=0x7fba7c00de10 comp rx=0 tx=0).stop 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 shutdown_connections 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fba780a3bb0 0x7fba780a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba88102a50 0x7fba8819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba88102090 0x7fba88198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 --2- 172.21.15.32:0/3177201281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba88100f40 0x7fba881982c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 >> 172.21.15.32:0/3177201281 conn(0x7fba880fc4b0 msgr2=0x7fba880fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 shutdown_connections 2024-04-16T05:32:50.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.459+0000 7fba8ec74640 1 -- 172.21.15.32:0/3177201281 wait complete. 2024-04-16T05:32:50.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:32:50.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 -- 172.21.15.32:0/340722164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30100f40 msgr2=0x7f4e30101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 --2- 172.21.15.32:0/340722164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30100f40 0x7f4e30101360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009900 tx=0x7f4e2002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 -- 172.21.15.32:0/340722164 shutdown_connections 2024-04-16T05:32:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 --2- 172.21.15.32:0/340722164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e30102a50 0x7f4e30109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 --2- 172.21.15.32:0/340722164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e30102090 0x7f4e30102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 --2- 172.21.15.32:0/340722164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30100f40 0x7f4e30101360 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 -- 172.21.15.32:0/340722164 >> 172.21.15.32:0/340722164 conn(0x7f4e300fc4b0 msgr2=0x7f4e300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 -- 172.21.15.32:0/340722164 shutdown_connections 2024-04-16T05:32:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.579+0000 7f4e36c63640 1 -- 172.21.15.32:0/340722164 wait complete. 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e36c63640 1 Processor -- start 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e36c63640 1 -- start start 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e36c63640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e30100f40 0x7f4e30069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e36c63640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e30102090 0x7f4e3019d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e36c63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30102a50 0x7f4e3019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e36c63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3010cbf0 con 0x7f4e30102a50 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e36c63640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3010ca70 con 0x7f4e30100f40 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e36c63640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3010cd70 con 0x7f4e30102090 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e35200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30102a50 0x7f4e3019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e35200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30102a50 0x7f4e3019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52460/0 (socket says 172.21.15.32:52460) 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e35200640 1 -- 172.21.15.32:0/2091308883 learned_addr learned my addr 172.21.15.32:0/2091308883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e2ffff640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e30102090 0x7f4e3019d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e349ff640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e30100f40 0x7f4e30069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e35200640 1 -- 172.21.15.32:0/2091308883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e30102090 msgr2=0x7f4e3019d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e35200640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e30102090 0x7f4e3019d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e35200640 1 -- 172.21.15.32:0/2091308883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e30100f40 msgr2=0x7f4e30069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e35200640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e30100f40 0x7f4e30069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e35200640 1 -- 172.21.15.32:0/2091308883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e301a0050 con 0x7f4e30102a50 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.580+0000 7f4e349ff640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e30100f40 0x7f4e30069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.581+0000 7f4e35200640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30102a50 0x7f4e3019f740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800eaa0 tx=0x7f4e2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.581+0000 7f4e2dffb640 1 -- 172.21.15.32:0/2091308883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800cdb0 con 0x7f4e30102a50 2024-04-16T05:32:50.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.581+0000 7f4e2dffb640 1 -- 172.21.15.32:0/2091308883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e28005020 con 0x7f4e30102a50 2024-04-16T05:32:50.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.581+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e301a02e0 con 0x7f4e30102a50 2024-04-16T05:32:50.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.581+0000 7f4e2dffb640 1 -- 172.21.15.32:0/2091308883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280116b0 con 0x7f4e30102a50 2024-04-16T05:32:50.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.581+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e301970d0 con 0x7f4e30102a50 2024-04-16T05:32:50.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.582+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e30108270 con 0x7f4e30102a50 2024-04-16T05:32:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.583+0000 7f4e2dffb640 1 -- 172.21.15.32:0/2091308883 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4e28005560 con 0x7f4e30102a50 2024-04-16T05:32:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.584+0000 7f4e2dffb640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e1c0a7fc0 0x7f4e1c0aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.584+0000 7f4e2dffb640 1 -- 172.21.15.32:0/2091308883 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4e280d0880 con 0x7f4e30102a50 2024-04-16T05:32:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.584+0000 7f4e349ff640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e1c0a7fc0 0x7f4e1c0aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.584+0000 7f4e349ff640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e1c0a7fc0 0x7f4e1c0aa480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e20006010 tx=0x7f4e2002c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:50.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.588+0000 7f4e2dffb640 1 -- 172.21.15.32:0/2091308883 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e28016030 con 0x7f4e30102a50 2024-04-16T05:32:50.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:50 smithi032 ceph-mon[32351]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2024-04-16T05:32:50.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:50 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1164611596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:50.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:50 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1369473188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:32:50.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:50 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3177201281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:32:50.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.777+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4e30062cd0 con 0x7f4e30102a50 2024-04-16T05:32:50.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.777+0000 7f4e2dffb640 1 -- 172.21.15.32:0/2091308883 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f4e2809d020 con 0x7f4e30102a50 2024-04-16T05:32:50.777 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:32:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e1c0a7fc0 msgr2=0x7f4e1c0aa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e1c0a7fc0 0x7f4e1c0aa480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e20006010 tx=0x7f4e2002c8a0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30102a50 msgr2=0x7f4e3019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30102a50 0x7f4e3019f740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800eaa0 tx=0x7f4e2800ef60 comp rx=0 tx=0).stop 2024-04-16T05:32:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 shutdown_connections 2024-04-16T05:32:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e1c0a7fc0 0x7f4e1c0aa480 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e30102a50 0x7f4e3019f740 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e30102090 0x7f4e3019d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 --2- 172.21.15.32:0/2091308883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e30100f40 0x7f4e30069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.779+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 >> 172.21.15.32:0/2091308883 conn(0x7f4e300fc4b0 msgr2=0x7f4e30105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.780+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 shutdown_connections 2024-04-16T05:32:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.780+0000 7f4e36c63640 1 -- 172.21.15.32:0/2091308883 wait complete. 2024-04-16T05:32:50.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T05:32:50.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 -- 172.21.15.32:0/642616690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32301024e0 msgr2=0x7f3230102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 --2- 172.21.15.32:0/642616690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32301024e0 0x7f3230102900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f322c009ca0 tx=0x7f322c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 -- 172.21.15.32:0/642616690 shutdown_connections 2024-04-16T05:32:50.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 --2- 172.21.15.32:0/642616690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3230101bd0 0x7f3230109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 --2- 172.21.15.32:0/642616690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3230101210 0x7f3230101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 --2- 172.21.15.32:0/642616690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32301024e0 0x7f3230102900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 -- 172.21.15.32:0/642616690 >> 172.21.15.32:0/642616690 conn(0x7f32300fbc40 msgr2=0x7f32300fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 -- 172.21.15.32:0/642616690 shutdown_connections 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.897+0000 7f32377e9640 1 -- 172.21.15.32:0/642616690 wait complete. 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f32377e9640 1 Processor -- start 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f32377e9640 1 -- start start 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f32377e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3230101210 0x7f32301983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f32377e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3230101bd0 0x7f3230198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f32377e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32301024e0 0x7f323019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f32377e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f323010cbc0 con 0x7f3230101bd0 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f32377e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f323010ca40 con 0x7f32301024e0 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f32377e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f323010cd40 con 0x7f3230101210 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3234d84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3230101bd0 0x7f3230198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3234d84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3230101bd0 0x7f3230198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52480/0 (socket says 172.21.15.32:52480) 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3234d84640 1 -- 172.21.15.32:0/583698313 learned_addr learned my addr 172.21.15.32:0/583698313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3235585640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3230101210 0x7f32301983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3235d86640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32301024e0 0x7f323019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3234d84640 1 -- 172.21.15.32:0/583698313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3230101210 msgr2=0x7f32301983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3234d84640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3230101210 0x7f32301983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3234d84640 1 -- 172.21.15.32:0/583698313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32301024e0 msgr2=0x7f323019f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3234d84640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32301024e0 0x7f323019f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.898+0000 7f3234d84640 1 -- 172.21.15.32:0/583698313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323019ffb0 con 0x7f3230101bd0 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.899+0000 7f3234d84640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3230101bd0 0x7f3230198920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f322000d950 tx=0x7f322000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.899+0000 7f32267fc640 1 -- 172.21.15.32:0/583698313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220017070 con 0x7f3230101bd0 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.899+0000 7f32267fc640 1 -- 172.21.15.32:0/583698313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3220009c30 con 0x7f3230101bd0 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.899+0000 7f32267fc640 1 -- 172.21.15.32:0/583698313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322000f6b0 con 0x7f3230101bd0 2024-04-16T05:32:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.899+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32301a02a0 con 0x7f3230101bd0 2024-04-16T05:32:50.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.899+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301a07f0 con 0x7f3230101bd0 2024-04-16T05:32:50.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.900+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32301a0a50 con 0x7f3230101bd0 2024-04-16T05:32:50.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:50 smithi082 ceph-mon[36195]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2024-04-16T05:32:50.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:50 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1164611596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:50.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:50 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1369473188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:32:50.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:50 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3177201281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:32:50.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.901+0000 7f32267fc640 1 -- 172.21.15.32:0/583698313 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f32200109d0 con 0x7f3230101bd0 2024-04-16T05:32:50.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.901+0000 7f32267fc640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f321c0a3bb0 0x7f321c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:50.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.902+0000 7f3235585640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f321c0a3bb0 0x7f321c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:50.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.902+0000 7f32267fc640 1 -- 172.21.15.32:0/583698313 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f32200d01e0 con 0x7f3230101bd0 2024-04-16T05:32:50.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.902+0000 7f3235585640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f321c0a3bb0 0x7f321c0a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f322c02c700 tx=0x7f322c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:50.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:50.906+0000 7f32267fc640 1 -- 172.21.15.32:0/583698313 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3220009de0 con 0x7f3230101bd0 2024-04-16T05:32:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.105+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f32301a0c20 con 0x7f3230101bd0 2024-04-16T05:32:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.105+0000 7f32267fc640 1 -- 172.21.15.32:0/583698313 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7f322009ca20 con 0x7f3230101bd0 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2557553342 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1590295441 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1590295441 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2929016634 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/26695350 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3665169244 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/3517252554 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1771946956 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/3517252554 2024-04-17T05:25:04.672077+0000 2024-04-16T05:32:51.107 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1006904605 2024-04-17T05:25:32.817074+0000 2024-04-16T05:32:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:listed 10 entries 2024-04-16T05:32:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f321c0a3bb0 msgr2=0x7f321c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f321c0a3bb0 0x7f321c0a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f322c02c700 tx=0x7f322c032590 comp rx=0 tx=0).stop 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3230101bd0 msgr2=0x7f3230198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3230101bd0 0x7f3230198920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f322000d950 tx=0x7f322000de10 comp rx=0 tx=0).stop 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 shutdown_connections 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f321c0a3bb0 0x7f321c0a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32301024e0 0x7f323019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3230101bd0 0x7f3230198920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 --2- 172.21.15.32:0/583698313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3230101210 0x7f32301983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 >> 172.21.15.32:0/583698313 conn(0x7f32300fbc40 msgr2=0x7f32300fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 shutdown_connections 2024-04-16T05:32:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.108+0000 7f32377e9640 1 -- 172.21.15.32:0/583698313 wait complete. 2024-04-16T05:32:51.118 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-16T05:32:51.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:51.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-16T05:32:51.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:50 smithi134 ceph-mon[36153]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2024-04-16T05:32:51.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:50 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1164611596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:32:51.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:50 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1369473188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:32:51.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:50 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3177201281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:32:51.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[76725]: fuse finished with error 0 and tester_r 0 2024-04-16T05:32:51.147 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T05:32:51.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:32:51.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:32:51.232 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-16T05:32:51.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:51.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 -- 172.21.15.32:0/1392575096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103c00 msgr2=0x7fe94810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 --2- 172.21.15.32:0/1392575096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103c00 0x7fe94810a490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe940009ca0 tx=0x7fe94002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 -- 172.21.15.32:0/1392575096 shutdown_connections 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 --2- 172.21.15.32:0/1392575096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103c00 0x7fe94810a490 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 --2- 172.21.15.32:0/1392575096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe948103240 0x7fe9481036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 --2- 172.21.15.32:0/1392575096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe948102050 0x7fe948102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 -- 172.21.15.32:0/1392575096 >> 172.21.15.32:0/1392575096 conn(0x7fe9480fd5e0 msgr2=0x7fe9480ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 -- 172.21.15.32:0/1392575096 shutdown_connections 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.403+0000 7fe94edf2640 1 -- 172.21.15.32:0/1392575096 wait complete. 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94edf2640 1 Processor -- start 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94edf2640 1 -- start start 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94edf2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe948102050 0x7fe948075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94edf2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103240 0x7fe9480743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94edf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe948103c00 0x7fe948074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94edf2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe94810dd20 con 0x7fe948103240 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94edf2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe94810dba0 con 0x7fe948102050 2024-04-16T05:32:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94edf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe94810dea0 con 0x7fe948103c00 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe947fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103240 0x7fe9480743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe947fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103240 0x7fe9480743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52500/0 (socket says 172.21.15.32:52500) 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe947fff640 1 -- 172.21.15.32:0/3254264127 learned_addr learned my addr 172.21.15.32:0/3254264127 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94cb8e640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe948102050 0x7fe948075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe94d38f640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe948103c00 0x7fe948074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe947fff640 1 -- 172.21.15.32:0/3254264127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe948103c00 msgr2=0x7fe948074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe947fff640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe948103c00 0x7fe948074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe947fff640 1 -- 172.21.15.32:0/3254264127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe948102050 msgr2=0x7fe948075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe947fff640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe948102050 0x7fe948075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.404+0000 7fe947fff640 1 -- 172.21.15.32:0/3254264127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9480751c0 con 0x7fe948103240 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.405+0000 7fe947fff640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103240 0x7fe9480743f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe93400b4a0 tx=0x7fe93400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.405+0000 7fe945ffb640 1 -- 172.21.15.32:0/3254264127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934014070 con 0x7fe948103240 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.405+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9481a2480 con 0x7fe948103240 2024-04-16T05:32:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.405+0000 7fe945ffb640 1 -- 172.21.15.32:0/3254264127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe934010af0 con 0x7fe948103240 2024-04-16T05:32:51.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.405+0000 7fe945ffb640 1 -- 172.21.15.32:0/3254264127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934017d10 con 0x7fe948103240 2024-04-16T05:32:51.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.405+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9481a2980 con 0x7fe948103240 2024-04-16T05:32:51.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.406+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe948109310 con 0x7fe948103240 2024-04-16T05:32:51.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.407+0000 7fe945ffb640 1 -- 172.21.15.32:0/3254264127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe934017070 con 0x7fe948103240 2024-04-16T05:32:51.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.408+0000 7fe945ffb640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9380a3bb0 0x7fe9380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.408+0000 7fe94cb8e640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9380a3bb0 0x7fe9380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.408+0000 7fe945ffb640 1 -- 172.21.15.32:0/3254264127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe9340cfa20 con 0x7fe948103240 2024-04-16T05:32:51.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.408+0000 7fe94cb8e640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9380a3bb0 0x7fe9380a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9481030a0 tx=0x7fe93c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.412+0000 7fe945ffb640 1 -- 172.21.15.32:0/3254264127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe93409bb10 con 0x7fe948103240 2024-04-16T05:32:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.603+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe948062c00 con 0x7fe948103240 2024-04-16T05:32:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.605+0000 7fe945ffb640 1 -- 172.21.15.32:0/3254264127 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7fe93409fe10 con 0x7fe948103240 2024-04-16T05:32:51.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:51.605 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9380a3bb0 msgr2=0x7fe9380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9380a3bb0 0x7fe9380a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9481030a0 tx=0x7fe93c00a2d0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103240 msgr2=0x7fe9480743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103240 0x7fe9480743f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe93400b4a0 tx=0x7fe93400b960 comp rx=0 tx=0).stop 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 shutdown_connections 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9380a3bb0 0x7fe9380a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe948103c00 0x7fe948074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe948103240 0x7fe9480743f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 --2- 172.21.15.32:0/3254264127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe948102050 0x7fe948075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 >> 172.21.15.32:0/3254264127 conn(0x7fe9480fd5e0 msgr2=0x7fe9481064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 shutdown_connections 2024-04-16T05:32:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.607+0000 7fe94edf2640 1 -- 172.21.15.32:0/3254264127 wait complete. 2024-04-16T05:32:51.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 -- 172.21.15.32:0/1448462713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d41020e0 msgr2=0x7f51d4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 --2- 172.21.15.32:0/1448462713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d41020e0 0x7f51d4102560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009900 tx=0x7f51bc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 -- 172.21.15.32:0/1448462713 shutdown_connections 2024-04-16T05:32:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 --2- 172.21.15.32:0/1448462713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d4102aa0 0x7f51d41093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 --2- 172.21.15.32:0/1448462713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d41020e0 0x7f51d4102560 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 --2- 172.21.15.32:0/1448462713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d4100f10 0x7f51d4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 -- 172.21.15.32:0/1448462713 >> 172.21.15.32:0/1448462713 conn(0x7f51d40fc4f0 msgr2=0x7f51d40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 -- 172.21.15.32:0/1448462713 shutdown_connections 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 -- 172.21.15.32:0/1448462713 wait complete. 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 Processor -- start 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.726+0000 7f51d9796640 1 -- start start 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d9796640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4100f10 0x7f51d4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d9796640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d41020e0 0x7f51d4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d9796640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d4102aa0 0x7f51d419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d9796640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51d410cc10 con 0x7f51d4100f10 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d9796640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f51d410ca90 con 0x7f51d41020e0 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d9796640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51d410cd90 con 0x7f51d4102aa0 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4100f10 0x7f51d4197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4100f10 0x7f51d4197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52506/0 (socket says 172.21.15.32:52506) 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 -- 172.21.15.32:0/404267292 learned_addr learned my addr 172.21.15.32:0/404267292 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 -- 172.21.15.32:0/404267292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d4102aa0 msgr2=0x7f51d419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d4102aa0 0x7f51d419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 -- 172.21.15.32:0/404267292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d41020e0 msgr2=0x7f51d4197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d27fc640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d41020e0 0x7f51d4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d41020e0 0x7f51d4197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 -- 172.21.15.32:0/404267292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d419ee40 con 0x7f51d4100f10 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d2ffd640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4100f10 0x7f51d4197220 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51c400d770 tx=0x7f51c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51cbfff640 1 -- 172.21.15.32:0/404267292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4017070 con 0x7f51d4100f10 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d419f130 con 0x7f51d4100f10 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51cbfff640 1 -- 172.21.15.32:0/404267292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c4009c30 con 0x7f51d4100f10 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51cbfff640 1 -- 172.21.15.32:0/404267292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4003dd0 con 0x7f51d4100f10 2024-04-16T05:32:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.727+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d419f680 con 0x7f51d4100f10 2024-04-16T05:32:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.728+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d41082a0 con 0x7f51d4100f10 2024-04-16T05:32:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.730+0000 7f51cbfff640 1 -- 172.21.15.32:0/404267292 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f51c4003fb0 con 0x7f51d4100f10 2024-04-16T05:32:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.730+0000 7f51cbfff640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f51c00a3bb0 0x7f51c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.730+0000 7f51d27fc640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f51c00a3bb0 0x7f51c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.730+0000 7f51cbfff640 1 -- 172.21.15.32:0/404267292 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f51c40d0480 con 0x7f51d4100f10 2024-04-16T05:32:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.731+0000 7f51d27fc640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f51c00a3bb0 0x7f51c00a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0096c0 tx=0x7f51bc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.734+0000 7f51cbfff640 1 -- 172.21.15.32:0/404267292 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51c4009de0 con 0x7f51d4100f10 2024-04-16T05:32:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.922+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51d402d620 con 0x7f51d4100f10 2024-04-16T05:32:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.923+0000 7f51cbfff640 1 -- 172.21.15.32:0/404267292 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7f51c409cc20 con 0x7f51d4100f10 2024-04-16T05:32:51.923 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:51.923 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f51c00a3bb0 msgr2=0x7f51c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f51c00a3bb0 0x7f51c00a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0096c0 tx=0x7f51bc02df90 comp rx=0 tx=0).stop 2024-04-16T05:32:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4100f10 msgr2=0x7f51d4197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4100f10 0x7f51d4197220 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51c400d770 tx=0x7f51c400dc30 comp rx=0 tx=0).stop 2024-04-16T05:32:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 shutdown_connections 2024-04-16T05:32:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f51c00a3bb0 0x7f51c00a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d4102aa0 0x7f51d419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d41020e0 0x7f51d4197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 --2- 172.21.15.32:0/404267292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4100f10 0x7f51d4197220 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 >> 172.21.15.32:0/404267292 conn(0x7f51d40fc4f0 msgr2=0x7f51d4105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 shutdown_connections 2024-04-16T05:32:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:51.925+0000 7f51d9796640 1 -- 172.21.15.32:0/404267292 wait complete. 2024-04-16T05:32:51.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:51 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2091308883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:32:51.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:51 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/583698313' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:32:51.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:51 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3254264127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:51.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-16T05:32:52.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.046+0000 7f46f3b1c640 1 -- 172.21.15.32:0/497172625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 msgr2=0x7f46ec109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.046+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/497172625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 0x7f46ec109370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009ca0 tx=0x7f46dc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.046+0000 7f46f3b1c640 1 -- 172.21.15.32:0/497172625 shutdown_connections 2024-04-16T05:32:52.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.046+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/497172625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 0x7f46ec109370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/497172625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46ec102090 0x7f46ec102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/497172625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec100f40 0x7f46ec101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 -- 172.21.15.32:0/497172625 >> 172.21.15.32:0/497172625 conn(0x7f46ec0fc4b0 msgr2=0x7f46ec0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 -- 172.21.15.32:0/497172625 shutdown_connections 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 -- 172.21.15.32:0/497172625 wait complete. 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 Processor -- start 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 -- start start 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46ec100f40 0x7f46ec198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec102090 0x7f46ec198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 0x7f46ec19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10dd30 con 0x7f46ec102a50 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10dbb0 con 0x7f46ec100f40 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.047+0000 7f46f3b1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10deb0 con 0x7f46ec102090 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 0x7f46ec19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 0x7f46ec19f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52522/0 (socket says 172.21.15.32:52522) 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 -- 172.21.15.32:0/451912544 learned_addr learned my addr 172.21.15.32:0/451912544 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f10b7640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec102090 0x7f46ec198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f18b8640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46ec100f40 0x7f46ec198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 -- 172.21.15.32:0/451912544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec102090 msgr2=0x7f46ec198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec102090 0x7f46ec198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 -- 172.21.15.32:0/451912544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46ec100f40 msgr2=0x7f46ec198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46ec100f40 0x7f46ec198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 -- 172.21.15.32:0/451912544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec19ff50 con 0x7f46ec102a50 2024-04-16T05:32:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f18b8640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46ec100f40 0x7f46ec198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:52.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f20b9640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 0x7f46ec19f8e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f46dc02c700 tx=0x7f46dc032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:52.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46eaffd640 1 -- 172.21.15.32:0/451912544 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc039040 con 0x7f46ec102a50 2024-04-16T05:32:52.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46eaffd640 1 -- 172.21.15.32:0/451912544 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46dc044070 con 0x7f46ec102a50 2024-04-16T05:32:52.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec1a01e0 con 0x7f46ec102a50 2024-04-16T05:32:52.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46eaffd640 1 -- 172.21.15.32:0/451912544 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0042a0 con 0x7f46ec102a50 2024-04-16T05:32:52.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.048+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1a0720 con 0x7f46ec102a50 2024-04-16T05:32:52.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.049+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec1082d0 con 0x7f46ec102a50 2024-04-16T05:32:52.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.050+0000 7f46eaffd640 1 -- 172.21.15.32:0/451912544 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f46dc003970 con 0x7f46ec102a50 2024-04-16T05:32:52.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.051+0000 7f46eaffd640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46d40a3c30 0x7f46d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.051+0000 7f46f18b8640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46d40a3c30 0x7f46d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.051+0000 7f46eaffd640 1 -- 172.21.15.32:0/451912544 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f46dc0f2bc0 con 0x7f46ec102a50 2024-04-16T05:32:52.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.052+0000 7f46f18b8640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46d40a3c30 0x7f46d40a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009b40 tx=0x7f46e4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:52.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.056+0000 7f46eaffd640 1 -- 172.21.15.32:0/451912544 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46dc0bf390 con 0x7f46ec102a50 2024-04-16T05:32:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:51 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2091308883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:32:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:51 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/583698313' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:32:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:51 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3254264127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:51 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2091308883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:32:52.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:51 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/583698313' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:32:52.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:51 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3254264127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.285+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f46ec062c00 con 0x7f46ec102a50 2024-04-16T05:32:52.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.659+0000 7f46eaffd640 1 -- 172.21.15.32:0/451912544 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f46dc0bec90 con 0x7f46ec102a50 2024-04-16T05:32:52.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46d40a3c30 msgr2=0x7f46d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46d40a3c30 0x7f46d40a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009b40 tx=0x7f46e4005e50 comp rx=0 tx=0).stop 2024-04-16T05:32:52.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 msgr2=0x7f46ec19f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 0x7f46ec19f8e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f46dc02c700 tx=0x7f46dc032f20 comp rx=0 tx=0).stop 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 shutdown_connections 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46d40a3c30 0x7f46d40a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec102a50 0x7f46ec19f8e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec102090 0x7f46ec198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 --2- 172.21.15.32:0/451912544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46ec100f40 0x7f46ec198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 >> 172.21.15.32:0/451912544 conn(0x7f46ec0fc4b0 msgr2=0x7f46ec1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 shutdown_connections 2024-04-16T05:32:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.662+0000 7f46f3b1c640 1 -- 172.21.15.32:0/451912544 wait complete. 2024-04-16T05:32:52.674 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-16T05:32:52.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-16T05:32:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3420472642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103c00 msgr2=0x7f7bb810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3420472642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103c00 0x7f7bb810a490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009ca0 tx=0x7f7bb002c100 comp rx=0 tx=0).stop 2024-04-16T05:32:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3420472642 shutdown_connections 2024-04-16T05:32:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3420472642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103c00 0x7f7bb810a490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3420472642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8103240 0x7f7bb81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3420472642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb8102050 0x7f7bb8102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3420472642 >> 172.21.15.32:0/3420472642 conn(0x7f7bb80fd5e0 msgr2=0x7f7bb80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:52.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3420472642 shutdown_connections 2024-04-16T05:32:52.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.787+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3420472642 wait complete. 2024-04-16T05:32:52.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bbd1f4640 1 Processor -- start 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bbd1f4640 1 -- start start 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bbd1f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8102050 0x7f7bb81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bbd1f4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103240 0x7f7bb81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bbd1f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb8103c00 0x7f7bb819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bbd1f4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810dce0 con 0x7f7bb8103240 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bbd1f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810db60 con 0x7f7bb8103c00 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bbd1f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810de60 con 0x7f7bb8102050 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bb659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103240 0x7f7bb81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bb659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103240 0x7f7bb81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52550/0 (socket says 172.21.15.32:52550) 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.788+0000 7f7bb659c640 1 -- 172.21.15.32:0/3115763799 learned_addr learned my addr 172.21.15.32:0/3115763799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bb759e640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb8103c00 0x7f7bb819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bb6d9d640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8102050 0x7f7bb81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bb659c640 1 -- 172.21.15.32:0/3115763799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8102050 msgr2=0x7f7bb81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bb659c640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8102050 0x7f7bb81982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bb659c640 1 -- 172.21.15.32:0/3115763799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb8103c00 msgr2=0x7f7bb819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bb659c640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb8103c00 0x7f7bb819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bb659c640 1 -- 172.21.15.32:0/3115763799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb819fec0 con 0x7f7bb8103240 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bb659c640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103240 0x7f7bb81987e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000d770 tx=0x7f7ba000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7baffff640 1 -- 172.21.15.32:0/3115763799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0017070 con 0x7f7bb8103240 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb81a01b0 con 0x7f7bb8103240 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7baffff640 1 -- 172.21.15.32:0/3115763799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba0005500 con 0x7f7bb8103240 2024-04-16T05:32:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7baffff640 1 -- 172.21.15.32:0/3115763799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0004e00 con 0x7f7bb8103240 2024-04-16T05:32:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.789+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb81a0700 con 0x7f7bb8103240 2024-04-16T05:32:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.790+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb8109310 con 0x7f7bb8103240 2024-04-16T05:32:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.791+0000 7f7baffff640 1 -- 172.21.15.32:0/3115763799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7ba0004fa0 con 0x7f7bb8103240 2024-04-16T05:32:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.792+0000 7f7baffff640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.792+0000 7f7bb6d9d640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.792+0000 7f7baffff640 1 -- 172.21.15.32:0/3115763799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7ba00d0530 con 0x7f7bb8103240 2024-04-16T05:32:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.792+0000 7f7bb6d9d640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80058c0 tx=0x7f7ba8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:52.796+0000 7f7baffff640 1 -- 172.21.15.32:0/3115763799 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ba009cc70 con 0x7f7bb8103240 2024-04-16T05:32:52.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:52 smithi032 ceph-mon[32351]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-04-16T05:32:52.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:52 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/404267292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:52.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:52 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/451912544' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T05:32:53.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.022+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f7bb802d820 con 0x7f7bb8103240 2024-04-16T05:32:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:52 smithi134 ceph-mon[36153]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-04-16T05:32:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:52 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/404267292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:52 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/451912544' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T05:32:53.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:52 smithi082 ceph-mon[36195]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-04-16T05:32:53.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:52 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/404267292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:53.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:52 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/451912544' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T05:32:53.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.665+0000 7f7baffff640 1 -- 172.21.15.32:0/3115763799 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f7ba009c570 con 0x7f7bb8103240 2024-04-16T05:32:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ba40a3bb0 msgr2=0x7f7ba40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80058c0 tx=0x7f7ba8005850 comp rx=0 tx=0).stop 2024-04-16T05:32:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103240 msgr2=0x7f7bb81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103240 0x7f7bb81987e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000d770 tx=0x7f7ba000dc30 comp rx=0 tx=0).stop 2024-04-16T05:32:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 shutdown_connections 2024-04-16T05:32:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb8103c00 0x7f7bb819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8103240 0x7f7bb81987e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 --2- 172.21.15.32:0/3115763799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8102050 0x7f7bb81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 >> 172.21.15.32:0/3115763799 conn(0x7f7bb80fd5e0 msgr2=0x7f7bb81064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 shutdown_connections 2024-04-16T05:32:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.669+0000 7f7bbd1f4640 1 -- 172.21.15.32:0/3115763799 wait complete. 2024-04-16T05:32:53.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-16T05:32:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 -- 172.21.15.32:0/3274867537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0102050 msgr2=0x7fc9d0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 --2- 172.21.15.32:0/3274867537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0102050 0x7fc9d0102470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009900 tx=0x7fc9c402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 -- 172.21.15.32:0/3274867537 shutdown_connections 2024-04-16T05:32:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 --2- 172.21.15.32:0/3274867537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0103c00 0x7fc9d010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 --2- 172.21.15.32:0/3274867537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d0103240 0x7fc9d01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 --2- 172.21.15.32:0/3274867537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0102050 0x7fc9d0102470 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 -- 172.21.15.32:0/3274867537 >> 172.21.15.32:0/3274867537 conn(0x7fc9d00fd5e0 msgr2=0x7fc9d00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 -- 172.21.15.32:0/3274867537 shutdown_connections 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.800+0000 7fc9d700c640 1 -- 172.21.15.32:0/3274867537 wait complete. 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d700c640 1 Processor -- start 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d700c640 1 -- start start 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d700c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0102050 0x7fc9d0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d700c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d0103240 0x7fc9d01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d700c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0103c00 0x7fc9d019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d700c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d010dca0 con 0x7fc9d0103c00 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d700c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d010db20 con 0x7fc9d0103240 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d700c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d010de20 con 0x7fc9d0102050 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0103c00 0x7fc9d019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0103c00 0x7fc9d019f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52566/0 (socket says 172.21.15.32:52566) 2024-04-16T05:32:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 -- 172.21.15.32:0/1996578376 learned_addr learned my addr 172.21.15.32:0/1996578376 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d4da8640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0102050 0x7fc9d0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9cffff640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d0103240 0x7fc9d01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 -- 172.21.15.32:0/1996578376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0102050 msgr2=0x7fc9d0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0102050 0x7fc9d0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 -- 172.21.15.32:0/1996578376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d0103240 msgr2=0x7fc9d01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d0103240 0x7fc9d01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 -- 172.21.15.32:0/1996578376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d019ffb0 con 0x7fc9d0103c00 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9cffff640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d0103240 0x7fc9d01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.801+0000 7fc9d55a9640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0103c00 0x7fc9d019f940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800eaa0 tx=0x7fc9c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.802+0000 7fc9cdffb640 1 -- 172.21.15.32:0/1996578376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c800cdb0 con 0x7fc9d0103c00 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.802+0000 7fc9cdffb640 1 -- 172.21.15.32:0/1996578376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c8005020 con 0x7fc9d0103c00 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.802+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d01a02a0 con 0x7fc9d0103c00 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.802+0000 7fc9cdffb640 1 -- 172.21.15.32:0/1996578376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80116b0 con 0x7fc9d0103c00 2024-04-16T05:32:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.802+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a07a0 con 0x7fc9d0103c00 2024-04-16T05:32:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.803+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d0109410 con 0x7fc9d0103c00 2024-04-16T05:32:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.804+0000 7fc9cdffb640 1 -- 172.21.15.32:0/1996578376 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc9c8005560 con 0x7fc9d0103c00 2024-04-16T05:32:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.804+0000 7fc9cdffb640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc9c00a3bb0 0x7fc9c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.805+0000 7fc9cdffb640 1 -- 172.21.15.32:0/1996578376 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc9c80d0880 con 0x7fc9d0103c00 2024-04-16T05:32:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.805+0000 7fc9d4da8640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc9c00a3bb0 0x7fc9c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.805+0000 7fc9d4da8640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc9c00a3bb0 0x7fc9c00a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000c00 tx=0x7fc9c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:53.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:53.810+0000 7fc9cdffb640 1 -- 172.21.15.32:0/1996578376 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9c8016030 con 0x7fc9d0103c00 2024-04-16T05:32:53.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:53 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/451912544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T05:32:53.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:53 smithi032 ceph-mon[32351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:53.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:53 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3115763799' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T05:32:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.038+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc9d0062c00 con 0x7fc9d0103c00 2024-04-16T05:32:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:53 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/451912544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T05:32:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:53 smithi134 ceph-mon[36153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:53 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3115763799' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T05:32:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:53 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/451912544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T05:32:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:53 smithi082 ceph-mon[36195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:53 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3115763799' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T05:32:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.673+0000 7fc9cdffb640 1 -- 172.21.15.32:0/1996578376 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fc9c809d020 con 0x7fc9d0103c00 2024-04-16T05:32:54.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.676+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc9c00a3bb0 msgr2=0x7fc9c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.676+0000 7fc9d700c640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc9c00a3bb0 0x7fc9c00a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000c00 tx=0x7fc9c4005fd0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.676+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0103c00 msgr2=0x7fc9d019f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.676+0000 7fc9d700c640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0103c00 0x7fc9d019f940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800eaa0 tx=0x7fc9c800ef60 comp rx=0 tx=0).stop 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.677+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 shutdown_connections 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.677+0000 7fc9d700c640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc9c00a3bb0 0x7fc9c00a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.677+0000 7fc9d700c640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d0103c00 0x7fc9d019f940 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.677+0000 7fc9d700c640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d0103240 0x7fc9d01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.677+0000 7fc9d700c640 1 --2- 172.21.15.32:0/1996578376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0102050 0x7fc9d0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.677+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 >> 172.21.15.32:0/1996578376 conn(0x7fc9d00fd5e0 msgr2=0x7fc9d01065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.677+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 shutdown_connections 2024-04-16T05:32:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.677+0000 7fc9d700c640 1 -- 172.21.15.32:0/1996578376 wait complete. 2024-04-16T05:32:54.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 -- 172.21.15.32:0/2426994610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 msgr2=0x7f6f4010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 --2- 172.21.15.32:0/2426994610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 0x7f6f4010a170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009ca0 tx=0x7f6f3802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 -- 172.21.15.32:0/2426994610 shutdown_connections 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 --2- 172.21.15.32:0/2426994610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 0x7f6f4010a170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 --2- 172.21.15.32:0/2426994610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f40105450 0x7f6f40107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 --2- 172.21.15.32:0/2426994610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f40102b20 0x7f6f40104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 -- 172.21.15.32:0/2426994610 >> 172.21.15.32:0/2426994610 conn(0x7f6f400fc4b0 msgr2=0x7f6f400fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 -- 172.21.15.32:0/2426994610 shutdown_connections 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 -- 172.21.15.32:0/2426994610 wait complete. 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 Processor -- start 2024-04-16T05:32:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.809+0000 7f6f48552640 1 -- start start 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f48552640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f40102b20 0x7f6f401982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f48552640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f40105450 0x7f6f40198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f48552640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 0x7f6f4019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f48552640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4010dcd0 con 0x7f6f40107d80 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f48552640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4010db50 con 0x7f6f40105450 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f48552640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4010de50 con 0x7f6f40102b20 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 0x7f6f4019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 0x7f6f4019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52596/0 (socket says 172.21.15.32:52596) 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 -- 172.21.15.32:0/2121376787 learned_addr learned my addr 172.21.15.32:0/2121376787 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f45aed640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f40105450 0x7f6f40198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f462ee640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f40102b20 0x7f6f401982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 -- 172.21.15.32:0/2121376787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f40102b20 msgr2=0x7f6f401982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f40102b20 0x7f6f401982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 -- 172.21.15.32:0/2121376787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f40105450 msgr2=0x7f6f40198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f40105450 0x7f6f40198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 -- 172.21.15.32:0/2121376787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4019fee0 con 0x7f6f40107d80 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f45aed640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f40105450 0x7f6f40198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f46aef640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 0x7f6f4019f870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f38037040 tx=0x7f6f38032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f377fe640 1 -- 172.21.15.32:0/2121376787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38041020 con 0x7f6f40107d80 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f377fe640 1 -- 172.21.15.32:0/2121376787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f38034d90 con 0x7f6f40107d80 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f377fe640 1 -- 172.21.15.32:0/2121376787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38039040 con 0x7f6f40107d80 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f401a0170 con 0x7f6f40107d80 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.810+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f401a0660 con 0x7f6f40107d80 2024-04-16T05:32:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.811+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f400fdfa0 con 0x7f6f40107d80 2024-04-16T05:32:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.813+0000 7f6f377fe640 1 -- 172.21.15.32:0/2121376787 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6f38044020 con 0x7f6f40107d80 2024-04-16T05:32:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.814+0000 7f6f377fe640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6f2c0a3bb0 0x7f6f2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.814+0000 7f6f377fe640 1 -- 172.21.15.32:0/2121376787 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6f380f29f0 con 0x7f6f40107d80 2024-04-16T05:32:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.814+0000 7f6f462ee640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6f2c0a3bb0 0x7f6f2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.814+0000 7f6f462ee640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6f2c0a3bb0 0x7f6f2c0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c005e10 tx=0x7f6f3c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:54.818+0000 7f6f377fe640 1 -- 172.21.15.32:0/2121376787 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f3804b020 con 0x7f6f40107d80 2024-04-16T05:32:54.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:54 smithi032 ceph-mon[32351]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T05:32:54.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:54 smithi032 ceph-mon[32351]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T05:32:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:54 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3115763799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T05:32:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:54 smithi032 ceph-mon[32351]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T05:32:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:54 smithi032 ceph-mon[32351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:54 smithi032 ceph-mon[32351]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-16T05:32:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:54 smithi032 ceph-mon[32351]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-16T05:32:54.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:54 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1996578376' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T05:32:55.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.050+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6f4002d620 con 0x7f6f40107d80 2024-04-16T05:32:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:54 smithi134 ceph-mon[36153]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T05:32:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:54 smithi134 ceph-mon[36153]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T05:32:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:54 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3115763799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T05:32:55.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:54 smithi134 ceph-mon[36153]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T05:32:55.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:54 smithi134 ceph-mon[36153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:55.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:54 smithi134 ceph-mon[36153]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-16T05:32:55.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:54 smithi134 ceph-mon[36153]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-16T05:32:55.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:54 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1996578376' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T05:32:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:54 smithi082 ceph-mon[36195]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T05:32:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:54 smithi082 ceph-mon[36195]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T05:32:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:54 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3115763799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T05:32:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:54 smithi082 ceph-mon[36195]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T05:32:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:54 smithi082 ceph-mon[36195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T05:32:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:54 smithi082 ceph-mon[36195]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-16T05:32:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:54 smithi082 ceph-mon[36195]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-16T05:32:55.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:54 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1996578376' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T05:32:55.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.684+0000 7f6f377fe640 1 -- 172.21.15.32:0/2121376787 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f6f380bf190 con 0x7f6f40107d80 2024-04-16T05:32:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6f2c0a3bb0 msgr2=0x7f6f2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6f2c0a3bb0 0x7f6f2c0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c005e10 tx=0x7f6f3c005da0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 msgr2=0x7f6f4019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 0x7f6f4019f870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f38037040 tx=0x7f6f38032f20 comp rx=0 tx=0).stop 2024-04-16T05:32:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 shutdown_connections 2024-04-16T05:32:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6f2c0a3bb0 0x7f6f2c0a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f40107d80 0x7f6f4019f870 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f40105450 0x7f6f40198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 --2- 172.21.15.32:0/2121376787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f40102b20 0x7f6f401982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 >> 172.21.15.32:0/2121376787 conn(0x7f6f400fc4b0 msgr2=0x7f6f40103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 shutdown_connections 2024-04-16T05:32:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.687+0000 7f6f48552640 1 -- 172.21.15.32:0/2121376787 wait complete. 2024-04-16T05:32:55.700 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-16T05:32:55.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 -- 172.21.15.32:0/3349325745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 msgr2=0x7fabbc101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 --2- 172.21.15.32:0/3349325745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 0x7fabbc101d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fabac009900 tx=0x7fabac02bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 -- 172.21.15.32:0/3349325745 shutdown_connections 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 --2- 172.21.15.32:0/3349325745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc102280 0x7fabbc108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 --2- 172.21.15.32:0/3349325745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 0x7fabbc101d40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 --2- 172.21.15.32:0/3349325745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc1006d0 0x7fabbc100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 -- 172.21.15.32:0/3349325745 >> 172.21.15.32:0/3349325745 conn(0x7fabbc0fbc40 msgr2=0x7fabbc0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 -- 172.21.15.32:0/3349325745 shutdown_connections 2024-04-16T05:32:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.817+0000 7fabc4775640 1 -- 172.21.15.32:0/3349325745 wait complete. 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc4775640 1 Processor -- start 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc4775640 1 -- start start 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc4775640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc1006d0 0x7fabbc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc4775640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 0x7fabbc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc4775640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc102280 0x7fabbc19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc4775640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10dd20 con 0x7fabbc1018c0 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc4775640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10dba0 con 0x7fabbc102280 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc4775640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10dea0 con 0x7fabbc1006d0 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc1d10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 0x7fabbc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc1d10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 0x7fabbc198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52620/0 (socket says 172.21.15.32:52620) 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc1d10640 1 -- 172.21.15.32:0/1378689059 learned_addr learned my addr 172.21.15.32:0/1378689059 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc2511640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc1006d0 0x7fabbc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc2d12640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc102280 0x7fabbc19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc1d10640 1 -- 172.21.15.32:0/1378689059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc1006d0 msgr2=0x7fabbc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc1d10640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc1006d0 0x7fabbc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc1d10640 1 -- 172.21.15.32:0/1378689059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc102280 msgr2=0x7fabbc19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc1d10640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc102280 0x7fabbc19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.818+0000 7fabc1d10640 1 -- 172.21.15.32:0/1378689059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc19ff70 con 0x7fabbc1018c0 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.819+0000 7fabc1d10640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 0x7fabbc198890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fabac009230 tx=0x7fabac005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.819+0000 7fabb37fe640 1 -- 172.21.15.32:0/1378689059 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac041070 con 0x7fabbc1018c0 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.819+0000 7fabb37fe640 1 -- 172.21.15.32:0/1378689059 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac0044d0 con 0x7fabbc1018c0 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.819+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc1a0200 con 0x7fabbc1018c0 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.819+0000 7fabb37fe640 1 -- 172.21.15.32:0/1378689059 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac034440 con 0x7fabbc1018c0 2024-04-16T05:32:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.819+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1a06f0 con 0x7fabbc1018c0 2024-04-16T05:32:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.820+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc107a70 con 0x7fabbc1018c0 2024-04-16T05:32:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.821+0000 7fabb37fe640 1 -- 172.21.15.32:0/1378689059 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fabac044070 con 0x7fabbc1018c0 2024-04-16T05:32:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.821+0000 7fabb37fe640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.822+0000 7fabc2511640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.822+0000 7fabb37fe640 1 -- 172.21.15.32:0/1378689059 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fabac0f2260 con 0x7fabbc1018c0 2024-04-16T05:32:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.822+0000 7fabc2511640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7faba80a3bb0 0x7faba80a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fabb8005fd0 tx=0x7fabb800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:55.826+0000 7fabb37fe640 1 -- 172.21.15.32:0/1378689059 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabac0be350 con 0x7fabbc1018c0 2024-04-16T05:32:55.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1996578376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T05:32:55.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] up:active 2024-04-16T05:32:55.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T05:32:55.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T05:32:55.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: Cluster is now healthy 2024-04-16T05:32:55.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-16T05:32:55.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-16T05:32:55.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T05:32:55.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:55 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2121376787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T05:32:56.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.039+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fabbc062440 con 0x7faba80a3bb0 2024-04-16T05:32:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.065+0000 7fabb37fe640 1 -- 172.21.15.32:0/1378689059 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fabbc062440 con 0x7faba80a3bb0 2024-04-16T05:32:56.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7faba80a3bb0 msgr2=0x7faba80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7faba80a3bb0 0x7faba80a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fabb8005fd0 tx=0x7fabb800b390 comp rx=0 tx=0).stop 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 msgr2=0x7fabbc198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 0x7fabbc198890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fabac009230 tx=0x7fabac005d50 comp rx=0 tx=0).stop 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 shutdown_connections 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc102280 0x7fabbc19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc1018c0 0x7fabbc198890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 --2- 172.21.15.32:0/1378689059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc1006d0 0x7fabbc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 >> 172.21.15.32:0/1378689059 conn(0x7fabbc0fbc40 msgr2=0x7fabbc104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 shutdown_connections 2024-04-16T05:32:56.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.067+0000 7fabc4775640 1 -- 172.21.15.32:0/1378689059 wait complete. 2024-04-16T05:32:56.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-16T05:32:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1996578376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T05:32:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] up:active 2024-04-16T05:32:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T05:32:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T05:32:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: Cluster is now healthy 2024-04-16T05:32:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-16T05:32:56.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-16T05:32:56.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T05:32:56.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:55 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2121376787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T05:32:56.149 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b[66419]: 2024-04-16T05:32:55.674+0000 7f2baedcc700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1996578376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] up:active 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: Cluster is now healthy 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T05:32:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:55 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2121376787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 -- 172.21.15.32:0/2744314768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b0102aa0 msgr2=0x7f86b01093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2744314768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b0102aa0 0x7f86b01093c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009ca0 tx=0x7f86a002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 -- 172.21.15.32:0/2744314768 shutdown_connections 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2744314768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b0102aa0 0x7f86b01093c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2744314768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b01020e0 0x7f86b0102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2744314768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86b0100f10 0x7f86b0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 -- 172.21.15.32:0/2744314768 >> 172.21.15.32:0/2744314768 conn(0x7f86b00fc4d0 msgr2=0x7f86b00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 -- 172.21.15.32:0/2744314768 shutdown_connections 2024-04-16T05:32:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.188+0000 7f86b52d9640 1 -- 172.21.15.32:0/2744314768 wait complete. 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86b52d9640 1 Processor -- start 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86b52d9640 1 -- start start 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86b52d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86b0100f10 0x7f86b0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86b52d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b01020e0 0x7f86b01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86b52d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b0102aa0 0x7f86b019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86b52d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010dd00 con 0x7f86b01020e0 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86b52d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010db80 con 0x7f86b0100f10 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86b52d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010de80 con 0x7f86b0102aa0 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86ae7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b01020e0 0x7f86b01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86ae7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b01020e0 0x7f86b01987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52652/0 (socket says 172.21.15.32:52652) 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86ae7fc640 1 -- 172.21.15.32:0/2848931891 learned_addr learned my addr 172.21.15.32:0/2848931891 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.189+0000 7f86aeffd640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86b0100f10 0x7f86b0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86ae7fc640 1 -- 172.21.15.32:0/2848931891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b0102aa0 msgr2=0x7f86b019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86af7fe640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b0102aa0 0x7f86b019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86ae7fc640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b0102aa0 0x7f86b019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86ae7fc640 1 -- 172.21.15.32:0/2848931891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86b0100f10 msgr2=0x7f86b0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86ae7fc640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86b0100f10 0x7f86b0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86ae7fc640 1 -- 172.21.15.32:0/2848931891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b019fea0 con 0x7f86b01020e0 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86aeffd640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86b0100f10 0x7f86b0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86ae7fc640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b01020e0 0x7f86b01987c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f869800c920 tx=0x7f869800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86a7fff640 1 -- 172.21.15.32:0/2848931891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698007c70 con 0x7f86b01020e0 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b01a0190 con 0x7f86b01020e0 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86a7fff640 1 -- 172.21.15.32:0/2848931891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8698007e10 con 0x7f86b01020e0 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86a7fff640 1 -- 172.21.15.32:0/2848931891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698018890 con 0x7f86b01020e0 2024-04-16T05:32:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.190+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01a06e0 con 0x7f86b01020e0 2024-04-16T05:32:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.191+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b01082a0 con 0x7f86b01020e0 2024-04-16T05:32:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.192+0000 7f86a7fff640 1 -- 172.21.15.32:0/2848931891 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8698018a70 con 0x7f86b01020e0 2024-04-16T05:32:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.193+0000 7f86a7fff640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f869c0a3bb0 0x7f869c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.193+0000 7f86aeffd640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f869c0a3bb0 0x7f869c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.193+0000 7f86a7fff640 1 -- 172.21.15.32:0/2848931891 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f86980d07e0 con 0x7f86b01020e0 2024-04-16T05:32:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.193+0000 7f86aeffd640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f869c0a3bb0 0x7f869c0a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86b0100c30 tx=0x7f86a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.197+0000 7f86a7fff640 1 -- 172.21.15.32:0/2848931891 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8698014030 con 0x7f86b01020e0 2024-04-16T05:32:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.415+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f86b0062440 con 0x7f869c0a3bb0 2024-04-16T05:32:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.439+0000 7f86a7fff640 1 -- 172.21.15.32:0/2848931891 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86b0062440 con 0x7f869c0a3bb0 2024-04-16T05:32:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f869c0a3bb0 msgr2=0x7f869c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f869c0a3bb0 0x7f869c0a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86b0100c30 tx=0x7f86a8005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b01020e0 msgr2=0x7f86b01987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b01020e0 0x7f86b01987c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f869800c920 tx=0x7f869800cde0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 shutdown_connections 2024-04-16T05:32:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f869c0a3bb0 0x7f869c0a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b0102aa0 0x7f86b019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86b01020e0 0x7f86b01987c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 --2- 172.21.15.32:0/2848931891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86b0100f10 0x7f86b0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 >> 172.21.15.32:0/2848931891 conn(0x7f86b00fc4d0 msgr2=0x7f86b0105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:56.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 shutdown_connections 2024-04-16T05:32:56.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.441+0000 7f86b52d9640 1 -- 172.21.15.32:0/2848931891 wait complete. 2024-04-16T05:32:56.450 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:32:56.044+0000 7f7ed337f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:32:56.450 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:32:56.044+0000 7f7ed337f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:32:56.450 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:32:56.044+0000 7f7ed337f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:32:56.450 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:32:56.044+0000 7f7ed337f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:32:56.450 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:32:56.044+0000 7f7ed337f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:32:56.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 -- 172.21.15.32:0/3934429586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c102aa0 msgr2=0x7fd23c1093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 --2- 172.21.15.32:0/3934429586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c102aa0 0x7fd23c1093c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd234009ca0 tx=0x7fd23402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 -- 172.21.15.32:0/3934429586 shutdown_connections 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 --2- 172.21.15.32:0/3934429586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c102aa0 0x7fd23c1093c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 --2- 172.21.15.32:0/3934429586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd23c1020e0 0x7fd23c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 --2- 172.21.15.32:0/3934429586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd23c100f10 0x7fd23c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 -- 172.21.15.32:0/3934429586 >> 172.21.15.32:0/3934429586 conn(0x7fd23c0fc4d0 msgr2=0x7fd23c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 -- 172.21.15.32:0/3934429586 shutdown_connections 2024-04-16T05:32:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.563+0000 7fd243132640 1 -- 172.21.15.32:0/3934429586 wait complete. 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 Processor -- start 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 -- start start 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c100f10 0x7fd23c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd23c1020e0 0x7fd23c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd23c102aa0 0x7fd23c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10dcc0 con 0x7fd23c100f10 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10db40 con 0x7fd23c1020e0 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10de40 con 0x7fd23c102aa0 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c100f10 0x7fd23c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c100f10 0x7fd23c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52662/0 (socket says 172.21.15.32:52662) 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 -- 172.21.15.32:0/1442126867 learned_addr learned my addr 172.21.15.32:0/1442126867 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd23bfff640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd23c1020e0 0x7fd23c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd2416cf640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd23c102aa0 0x7fd23c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 -- 172.21.15.32:0/1442126867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd23c102aa0 msgr2=0x7fd23c19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd23c102aa0 0x7fd23c19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 -- 172.21.15.32:0/1442126867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd23c1020e0 msgr2=0x7fd23c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd23c1020e0 0x7fd23c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 -- 172.21.15.32:0/1442126867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23c19ff20 con 0x7fd23c100f10 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd2416cf640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd23c102aa0 0x7fd23c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd240ece640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c100f10 0x7fd23c198300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd23000d770 tx=0x7fd23000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd239ffb640 1 -- 172.21.15.32:0/1442126867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230017070 con 0x7fd23c100f10 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd239ffb640 1 -- 172.21.15.32:0/1442126867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd230009c30 con 0x7fd23c100f10 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23c1a0210 con 0x7fd23c100f10 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.564+0000 7fd239ffb640 1 -- 172.21.15.32:0/1442126867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230003dd0 con 0x7fd23c100f10 2024-04-16T05:32:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.565+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23c1a0760 con 0x7fd23c100f10 2024-04-16T05:32:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.566+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd23c108320 con 0x7fd23c100f10 2024-04-16T05:32:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.567+0000 7fd239ffb640 1 -- 172.21.15.32:0/1442126867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd230003f70 con 0x7fd23c100f10 2024-04-16T05:32:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.567+0000 7fd239ffb640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd22c0a3bb0 0x7fd22c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.567+0000 7fd23bfff640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd22c0a3bb0 0x7fd22c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.567+0000 7fd239ffb640 1 -- 172.21.15.32:0/1442126867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd2300cffb0 con 0x7fd23c100f10 2024-04-16T05:32:56.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.568+0000 7fd23bfff640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd22c0a3bb0 0x7fd22c0a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd23c199810 tx=0x7fd228005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:56.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.572+0000 7fd239ffb640 1 -- 172.21.15.32:0/1442126867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd23009c750 con 0x7fd23c100f10 2024-04-16T05:32:56.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-mon[32351]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T05:32:56.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2121376787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T05:32:56.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] up:active 2024-04-16T05:32:56.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-mon[32351]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-16T05:32:56.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:56 smithi032 ceph-mon[32351]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T05:32:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.791+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd23c0624d0 con 0x7fd22c0a3bb0 2024-04-16T05:32:56.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.796+0000 7fd239ffb640 1 -- 172.21.15.32:0/1442126867 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd23c0624d0 con 0x7fd22c0a3bb0 2024-04-16T05:32:56.795 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/qa/sv_0/4937f2e3-0d1b-411b-8eb1-77635e097d4c 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd22c0a3bb0 msgr2=0x7fd22c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd22c0a3bb0 0x7fd22c0a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd23c199810 tx=0x7fd228005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c100f10 msgr2=0x7fd23c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c100f10 0x7fd23c198300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd23000d770 tx=0x7fd23000dc30 comp rx=0 tx=0).stop 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 shutdown_connections 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd22c0a3bb0 0x7fd22c0a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd23c102aa0 0x7fd23c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd23c1020e0 0x7fd23c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 --2- 172.21.15.32:0/1442126867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23c100f10 0x7fd23c198300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 >> 172.21.15.32:0/1442126867 conn(0x7fd23c0fc4d0 msgr2=0x7fd23c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:56.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 shutdown_connections 2024-04-16T05:32:56.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.798+0000 7fd243132640 1 -- 172.21.15.32:0/1442126867 wait complete. 2024-04-16T05:32:56.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-16T05:32:56.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 -- 172.21.15.32:0/2177421336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103c00 msgr2=0x7f1e9410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/2177421336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103c00 0x7f1e9410a490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009ca0 tx=0x7f1e8c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 -- 172.21.15.32:0/2177421336 shutdown_connections 2024-04-16T05:32:56.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/2177421336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103c00 0x7f1e9410a490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/2177421336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e94103240 0x7f1e941036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/2177421336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94102050 0x7f1e94102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 -- 172.21.15.32:0/2177421336 >> 172.21.15.32:0/2177421336 conn(0x7f1e940fd5e0 msgr2=0x7f1e940ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:56.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 -- 172.21.15.32:0/2177421336 shutdown_connections 2024-04-16T05:32:56.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 -- 172.21.15.32:0/2177421336 wait complete. 2024-04-16T05:32:56.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 Processor -- start 2024-04-16T05:32:56.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.921+0000 7f1e98ff4640 1 -- start start 2024-04-16T05:32:56.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e98ff4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94102050 0x7f1e94198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e98ff4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103240 0x7f1e94198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e98ff4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e94103c00 0x7f1e9419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e98ff4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9410ddc0 con 0x7f1e94103240 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e98ff4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9410dc40 con 0x7f1e94102050 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e98ff4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9410df40 con 0x7f1e94103c00 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103240 0x7f1e94198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103240 0x7f1e94198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52668/0 (socket says 172.21.15.32:52668) 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 -- 172.21.15.32:0/1149681814 learned_addr learned my addr 172.21.15.32:0/1149681814 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e9259c640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94102050 0x7f1e94198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e92d9d640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e94103c00 0x7f1e9419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 -- 172.21.15.32:0/1149681814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e94103c00 msgr2=0x7f1e9419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e94103c00 0x7f1e9419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 -- 172.21.15.32:0/1149681814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94102050 msgr2=0x7f1e94198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94102050 0x7f1e94198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:56.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 -- 172.21.15.32:0/1149681814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e941a0030 con 0x7f1e94103240 2024-04-16T05:32:56.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8bfff640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103240 0x7f1e94198950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00b4a0 tx=0x7f1e7c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:56.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/1149681814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c015020 con 0x7f1e94103240 2024-04-16T05:32:56.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/1149681814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c011d10 con 0x7f1e94103240 2024-04-16T05:32:56.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e941a0320 con 0x7f1e94103240 2024-04-16T05:32:56.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.922+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/1149681814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c00f2e0 con 0x7f1e94103240 2024-04-16T05:32:56.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.923+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e941a08f0 con 0x7f1e94103240 2024-04-16T05:32:56.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.924+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e94109370 con 0x7f1e94103240 2024-04-16T05:32:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.925+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/1149681814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1e7c018020 con 0x7f1e94103240 2024-04-16T05:32:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.925+0000 7f1e8b7fe640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1e800a3c30 0x7f1e800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.925+0000 7f1e9259c640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1e800a3c30 0x7f1e800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.925+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/1149681814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1e7c0cfeb0 con 0x7f1e94103240 2024-04-16T05:32:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.926+0000 7f1e9259c640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1e800a3c30 0x7f1e800a60f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e941030a0 tx=0x7f1e84009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:56.930+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/1149681814 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e7c01f020 con 0x7f1e94103240 2024-04-16T05:32:57.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:56 smithi134 ceph-mon[36153]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T05:32:57.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:56 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2121376787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T05:32:57.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:56 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] up:active 2024-04-16T05:32:57.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:56 smithi134 ceph-mon[36153]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-16T05:32:57.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:56 smithi134 ceph-mon[36153]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T05:32:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.145+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1e94062440 con 0x7f1e800a3c30 2024-04-16T05:32:57.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:56 smithi082 ceph-mon[36195]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T05:32:57.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:56 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2121376787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T05:32:57.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:56 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] up:active 2024-04-16T05:32:57.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:56 smithi082 ceph-mon[36195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-16T05:32:57.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:56 smithi082 ceph-mon[36195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T05:32:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.173+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/1149681814 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1e94062440 con 0x7f1e800a3c30 2024-04-16T05:32:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1e800a3c30 msgr2=0x7f1e800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1e800a3c30 0x7f1e800a60f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e941030a0 tx=0x7f1e84009110 comp rx=0 tx=0).stop 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103240 msgr2=0x7f1e94198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103240 0x7f1e94198950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00b4a0 tx=0x7f1e7c00b8a0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 shutdown_connections 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f1e800a3c30 0x7f1e800a60f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e94103c00 0x7f1e9419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e94103240 0x7f1e94198950 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.175+0000 7f1e98ff4640 1 --2- 172.21.15.32:0/1149681814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94102050 0x7f1e94198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.176+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 >> 172.21.15.32:0/1149681814 conn(0x7f1e940fd5e0 msgr2=0x7f1e94106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.176+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 shutdown_connections 2024-04-16T05:32:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.176+0000 7f1e98ff4640 1 -- 172.21.15.32:0/1149681814 wait complete. 2024-04-16T05:32:57.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-16T05:32:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.296+0000 7f3c7cec5640 1 -- 172.21.15.32:0/972421398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78100930 msgr2=0x7f3c78100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.296+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/972421398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78100930 0x7f3c78100d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009900 tx=0x7f3c6802bcd0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 -- 172.21.15.32:0/972421398 shutdown_connections 2024-04-16T05:32:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/972421398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78101c30 0x7f3c781020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/972421398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c78101290 0x7f3c781016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/972421398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78100930 0x7f3c78100d50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 -- 172.21.15.32:0/972421398 >> 172.21.15.32:0/972421398 conn(0x7f3c780fc4b0 msgr2=0x7f3c780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 -- 172.21.15.32:0/972421398 shutdown_connections 2024-04-16T05:32:57.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 -- 172.21.15.32:0/972421398 wait complete. 2024-04-16T05:32:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 Processor -- start 2024-04-16T05:32:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 -- start start 2024-04-16T05:32:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c78100930 0x7f3c7819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78101290 0x7f3c7819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.297+0000 7f3c7cec5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78101c30 0x7f3c781a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c7cec5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810fed0 con 0x7f3c78101c30 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c7cec5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810fd50 con 0x7f3c78100930 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c7cec5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78110050 con 0x7f3c78101290 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78101c30 0x7f3c781a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78101c30 0x7f3c781a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52686/0 (socket says 172.21.15.32:52686) 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 -- 172.21.15.32:0/454198617 learned_addr learned my addr 172.21.15.32:0/454198617 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c7659c640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c78100930 0x7f3c7819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 -- 172.21.15.32:0/454198617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78101290 msgr2=0x7f3c7819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78101290 0x7f3c7819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 -- 172.21.15.32:0/454198617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c78100930 msgr2=0x7f3c7819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c78100930 0x7f3c7819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 -- 172.21.15.32:0/454198617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c781a21d0 con 0x7f3c78101c30 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c76d9d640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78101c30 0x7f3c781a1b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000cad0 tx=0x7f3c7000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c6f7fe640 1 -- 172.21.15.32:0/454198617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000ae20 con 0x7f3c78101c30 2024-04-16T05:32:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c6f7fe640 1 -- 172.21.15.32:0/454198617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c70005020 con 0x7f3c78101c30 2024-04-16T05:32:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c781a24c0 con 0x7f3c78101c30 2024-04-16T05:32:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c6f7fe640 1 -- 172.21.15.32:0/454198617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000f6f0 con 0x7f3c78101c30 2024-04-16T05:32:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.298+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c781a2a10 con 0x7f3c78101c30 2024-04-16T05:32:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.299+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7810b850 con 0x7f3c78101c30 2024-04-16T05:32:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.301+0000 7f3c6f7fe640 1 -- 172.21.15.32:0/454198617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c700051d0 con 0x7f3c78101c30 2024-04-16T05:32:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.301+0000 7f3c6f7fe640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3c640a3bb0 0x7f3c640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.301+0000 7f3c7659c640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3c640a3bb0 0x7f3c640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.301+0000 7f3c6f7fe640 1 -- 172.21.15.32:0/454198617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3c700d0a50 con 0x7f3c78101c30 2024-04-16T05:32:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.302+0000 7f3c7659c640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3c640a3bb0 0x7f3c640a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c68036040 tx=0x7f3c68000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:57.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.306+0000 7f3c6f7fe640 1 -- 172.21.15.32:0/454198617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c70014030 con 0x7f3c78101c30 2024-04-16T05:32:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.524+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f3c78062440 con 0x7f3c640a3bb0 2024-04-16T05:32:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.529+0000 7f3c6f7fe640 1 -- 172.21.15.32:0/454198617 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3c78062440 con 0x7f3c640a3bb0 2024-04-16T05:32:57.528 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/qa/sv_1/96f1f0e6-c969-48f2-b2af-33d422a19791 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3c640a3bb0 msgr2=0x7f3c640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3c640a3bb0 0x7f3c640a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c68036040 tx=0x7f3c68000bc0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78101c30 msgr2=0x7f3c781a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78101c30 0x7f3c781a1b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000cad0 tx=0x7f3c7000cf90 comp rx=0 tx=0).stop 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 shutdown_connections 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3c640a3bb0 0x7f3c640a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78101c30 0x7f3c781a1b60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78101290 0x7f3c7819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 --2- 172.21.15.32:0/454198617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c78100930 0x7f3c7819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 >> 172.21.15.32:0/454198617 conn(0x7f3c780fc4b0 msgr2=0x7f3c780fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 shutdown_connections 2024-04-16T05:32:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.531+0000 7f3c7cec5640 1 -- 172.21.15.32:0/454198617 wait complete. 2024-04-16T05:32:57.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:32:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.646+0000 7f2784352640 1 -- 172.21.15.32:0/1157407451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c1053f0 msgr2=0x7f277c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.646+0000 7f2784352640 1 --2- 172.21.15.32:0/1157407451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c1053f0 0x7f277c1077e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2770009900 tx=0x7f277002bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 -- 172.21.15.32:0/1157407451 shutdown_connections 2024-04-16T05:32:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 --2- 172.21.15.32:0/1157407451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277c107d20 0x7f277c10a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 --2- 172.21.15.32:0/1157407451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c1053f0 0x7f277c1077e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 --2- 172.21.15.32:0/1157407451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f277c102ac0 0x7f277c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 -- 172.21.15.32:0/1157407451 >> 172.21.15.32:0/1157407451 conn(0x7f277c0fc4b0 msgr2=0x7f277c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 -- 172.21.15.32:0/1157407451 shutdown_connections 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 -- 172.21.15.32:0/1157407451 wait complete. 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 Processor -- start 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 -- start start 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277c102ac0 0x7f277c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f277c1053f0 0x7f277c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c107d20 0x7f277c19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10dcd0 con 0x7f277c107d20 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10db50 con 0x7f277c1053f0 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f2784352640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10de50 con 0x7f277c102ac0 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.647+0000 7f27828ef640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c107d20 0x7f277c19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27828ef640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c107d20 0x7f277c19f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52696/0 (socket says 172.21.15.32:52696) 2024-04-16T05:32:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27828ef640 1 -- 172.21.15.32:0/2937186221 learned_addr learned my addr 172.21.15.32:0/2937186221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27818ed640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f277c1053f0 0x7f277c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27828ef640 1 -- 172.21.15.32:0/2937186221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277c102ac0 msgr2=0x7f277c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27828ef640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277c102ac0 0x7f277c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27828ef640 1 -- 172.21.15.32:0/2937186221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f277c1053f0 msgr2=0x7f277c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27828ef640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f277c1053f0 0x7f277c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27828ef640 1 -- 172.21.15.32:0/2937186221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277c19ffb0 con 0x7f277c107d20 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27818ed640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f277c1053f0 0x7f277c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f27828ef640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c107d20 0x7f277c19f940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f276c00cad0 tx=0x7f276c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f277b7fe640 1 -- 172.21.15.32:0/2937186221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c00ae20 con 0x7f277c107d20 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f277b7fe640 1 -- 172.21.15.32:0/2937186221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276c005020 con 0x7f277c107d20 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277c1a02a0 con 0x7f277c107d20 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f277b7fe640 1 -- 172.21.15.32:0/2937186221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c00f6f0 con 0x7f277c107d20 2024-04-16T05:32:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.648+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277c1a0840 con 0x7f277c107d20 2024-04-16T05:32:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.649+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f277c0fdfa0 con 0x7f277c107d20 2024-04-16T05:32:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.650+0000 7f277b7fe640 1 -- 172.21.15.32:0/2937186221 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f276c0051d0 con 0x7f277c107d20 2024-04-16T05:32:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.651+0000 7f277b7fe640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f27640a3c30 0x7f27640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.651+0000 7f27820ee640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f27640a3c30 0x7f27640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.651+0000 7f277b7fe640 1 -- 172.21.15.32:0/2937186221 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f276c0d0a20 con 0x7f277c107d20 2024-04-16T05:32:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.651+0000 7f27820ee640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f27640a3c30 0x7f27640a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2774005fd0 tx=0x7f2774005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:57.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.656+0000 7f277b7fe640 1 -- 172.21.15.32:0/2937186221 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f276c014030 con 0x7f277c107d20 2024-04-16T05:32:57.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.700+0000 7f277b7fe640 1 -- 172.21.15.32:0/2937186221 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f276c0990d0 con 0x7f277c107d20 2024-04-16T05:32:57.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.890+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f277c02d620 con 0x7f277c107d20 2024-04-16T05:32:57.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.891+0000 7f277b7fe640 1 -- 172.21.15.32:0/2937186221 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11876 (secure 0 0 0) 0x7f276c0c3830 con 0x7f277c107d20 2024-04-16T05:32:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:57.890 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:32:55.688634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24461,24494]},"id":1}]} 2024-04-16T05:32:57.890 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-16T05:32:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f27640a3c30 msgr2=0x7f27640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f27640a3c30 0x7f27640a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2774005fd0 tx=0x7f2774005850 comp rx=0 tx=0).stop 2024-04-16T05:32:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c107d20 msgr2=0x7f277c19f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c107d20 0x7f277c19f940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f276c00cad0 tx=0x7f276c00cf90 comp rx=0 tx=0).stop 2024-04-16T05:32:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 shutdown_connections 2024-04-16T05:32:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f27640a3c30 0x7f27640a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c107d20 0x7f277c19f940 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f277c1053f0 0x7f277c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 --2- 172.21.15.32:0/2937186221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277c102ac0 0x7f277c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.893+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 >> 172.21.15.32:0/2937186221 conn(0x7f277c0fc4b0 msgr2=0x7f277c103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.894+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 shutdown_connections 2024-04-16T05:32:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:57.894+0000 7f2784352640 1 -- 172.21.15.32:0/2937186221 wait complete. 2024-04-16T05:32:57.902 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:57 smithi032 ceph-mon[32351]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:57.902 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:57 smithi032 ceph-mon[32351]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:57.903 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:57 smithi032 ceph-mon[32351]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:57.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:32:57.974 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:57 smithi082 ceph-mon[36195]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:57.974 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:57 smithi082 ceph-mon[36195]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:57.974 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:57 smithi082 ceph-mon[36195]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/697841896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4102b40 msgr2=0x7f4bf4104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/697841896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4102b40 0x7f4bf4104f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009900 tx=0x7f4be402bc30 comp rx=0 tx=0).stop 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/697841896 shutdown_connections 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/697841896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf4107da0 0x7f4bf410a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/697841896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf4105470 0x7f4bf4107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/697841896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4102b40 0x7f4bf4104f30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/697841896 >> 172.21.15.32:0/697841896 conn(0x7f4bf40fc550 msgr2=0x7f4bf40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/697841896 shutdown_connections 2024-04-16T05:32:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.010+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/697841896 wait complete. 2024-04-16T05:32:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4bf8d0e640 1 Processor -- start 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4bf8d0e640 1 -- start start 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4bf8d0e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf4102b40 0x7f4bf41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4bf8d0e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4105470 0x7f4bf41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4bf8d0e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf4107da0 0x7f4bf419f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4bf8d0e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf410dce0 con 0x7f4bf4105470 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4bf8d0e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf410db60 con 0x7f4bf4102b40 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4bf8d0e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf410de60 con 0x7f4bf4107da0 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4be9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4105470 0x7f4bf41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4be9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4105470 0x7f4bf41988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52706/0 (socket says 172.21.15.32:52706) 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.011+0000 7f4be9d9b640 1 -- 172.21.15.32:0/3257914625 learned_addr learned my addr 172.21.15.32:0/3257914625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4bf259c640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf4102b40 0x7f4bf41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4bf2d9d640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf4107da0 0x7f4bf419f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4be9d9b640 1 -- 172.21.15.32:0/3257914625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf4107da0 msgr2=0x7f4bf419f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4be9d9b640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf4107da0 0x7f4bf419f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4be9d9b640 1 -- 172.21.15.32:0/3257914625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf4102b40 msgr2=0x7f4bf41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4be9d9b640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf4102b40 0x7f4bf41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4be9d9b640 1 -- 172.21.15.32:0/3257914625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf419ffd0 con 0x7f4bf4105470 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4bf259c640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf4102b40 0x7f4bf41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4be9d9b640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4105470 0x7f4bf41988f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00d740 tx=0x7f4bdc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4bebfff640 1 -- 172.21.15.32:0/3257914625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc017070 con 0x7f4bf4105470 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4bebfff640 1 -- 172.21.15.32:0/3257914625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bdc00de50 con 0x7f4bf4105470 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf41a02c0 con 0x7f4bf4105470 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4bebfff640 1 -- 172.21.15.32:0/3257914625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc004e00 con 0x7f4bf4105470 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.012+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41a07c0 con 0x7f4bf4105470 2024-04-16T05:32:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.013+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf40fe040 con 0x7f4bf4105470 2024-04-16T05:32:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.014+0000 7f4bebfff640 1 -- 172.21.15.32:0/3257914625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4bdc004fa0 con 0x7f4bf4105470 2024-04-16T05:32:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.015+0000 7f4bebfff640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4be00a3c40 0x7f4be00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.015+0000 7f4bf259c640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4be00a3c40 0x7f4be00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.015+0000 7f4bebfff640 1 -- 172.21.15.32:0/3257914625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4bdc0d0290 con 0x7f4bf4105470 2024-04-16T05:32:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.016+0000 7f4bf259c640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4be00a3c40 0x7f4be00a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4be40096c0 tx=0x7f4be4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.020+0000 7f4bebfff640 1 -- 172.21.15.32:0/3257914625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bdc09c870 con 0x7f4bf4105470 2024-04-16T05:32:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:57 smithi134 ceph-mon[36153]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:57 smithi134 ceph-mon[36153]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:57 smithi134 ceph-mon[36153]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.219+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bf402d7d0 con 0x7f4bf4105470 2024-04-16T05:32:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.220+0000 7f4bebfff640 1 -- 172.21.15.32:0/3257914625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7f4bdc09c170 con 0x7f4bf4105470 2024-04-16T05:32:58.219 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:58.219 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:32:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4be00a3c40 msgr2=0x7f4be00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4be00a3c40 0x7f4be00a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4be40096c0 tx=0x7f4be4000bc0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4105470 msgr2=0x7f4bf41988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4105470 0x7f4bf41988f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00d740 tx=0x7f4bdc00dc00 comp rx=0 tx=0).stop 2024-04-16T05:32:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 shutdown_connections 2024-04-16T05:32:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4be00a3c40 0x7f4be00a6100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf4107da0 0x7f4bf419f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf4105470 0x7f4bf41988f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 --2- 172.21.15.32:0/3257914625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf4102b40 0x7f4bf41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 >> 172.21.15.32:0/3257914625 conn(0x7f4bf40fc550 msgr2=0x7f4bf4103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 shutdown_connections 2024-04-16T05:32:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.222+0000 7f4bf8d0e640 1 -- 172.21.15.32:0/3257914625 wait complete. 2024-04-16T05:32:58.232 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-16T05:32:58.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.340+0000 7effcd4a5640 1 -- 172.21.15.32:0/3830271693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 msgr2=0x7effc810a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.340+0000 7effcd4a5640 1 --2- 172.21.15.32:0/3830271693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 0x7effc810a170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7effb8009ca0 tx=0x7effb802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.340+0000 7effcd4a5640 1 -- 172.21.15.32:0/3830271693 shutdown_connections 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.340+0000 7effcd4a5640 1 --2- 172.21.15.32:0/3830271693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 0x7effc810a170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.340+0000 7effcd4a5640 1 --2- 172.21.15.32:0/3830271693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc81053f0 0x7effc81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.340+0000 7effcd4a5640 1 --2- 172.21.15.32:0/3830271693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8102ac0 0x7effc8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.340+0000 7effcd4a5640 1 -- 172.21.15.32:0/3830271693 >> 172.21.15.32:0/3830271693 conn(0x7effc80fc4b0 msgr2=0x7effc80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 -- 172.21.15.32:0/3830271693 shutdown_connections 2024-04-16T05:32:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 -- 172.21.15.32:0/3830271693 wait complete. 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 Processor -- start 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 -- start start 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc8102ac0 0x7effc8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc81053f0 0x7effc81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 0x7effc819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7effc810dcd0 con 0x7effc8107d20 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7effc810db50 con 0x7effc81053f0 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.341+0000 7effcd4a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7effc810de50 con 0x7effc8102ac0 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 0x7effc819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 0x7effc819f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52732/0 (socket says 172.21.15.32:52732) 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 -- 172.21.15.32:0/799125604 learned_addr learned my addr 172.21.15.32:0/799125604 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc6ffd640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc8102ac0 0x7effc8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc67fc640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc81053f0 0x7effc81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 -- 172.21.15.32:0/799125604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc8102ac0 msgr2=0x7effc8198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc8102ac0 0x7effc8198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 -- 172.21.15.32:0/799125604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc81053f0 msgr2=0x7effc81987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc81053f0 0x7effc81987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 -- 172.21.15.32:0/799125604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc819feb0 con 0x7effc8107d20 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effc77fe640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 0x7effc819f840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7effb8037040 tx=0x7effb8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effbffff640 1 -- 172.21.15.32:0/799125604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8041020 con 0x7effc8107d20 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effbffff640 1 -- 172.21.15.32:0/799125604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb8034d90 con 0x7effc8107d20 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc81a0140 con 0x7effc8107d20 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effbffff640 1 -- 172.21.15.32:0/799125604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8039040 con 0x7effc8107d20 2024-04-16T05:32:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.342+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc81a0630 con 0x7effc8107d20 2024-04-16T05:32:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.343+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc80fdfa0 con 0x7effc8107d20 2024-04-16T05:32:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.344+0000 7effbffff640 1 -- 172.21.15.32:0/799125604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7effb8044020 con 0x7effc8107d20 2024-04-16T05:32:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.345+0000 7effbffff640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7effb40a3c40 0x7effb40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.345+0000 7effbffff640 1 -- 172.21.15.32:0/799125604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7effb80f2720 con 0x7effc8107d20 2024-04-16T05:32:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.345+0000 7effc6ffd640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7effb40a3c40 0x7effb40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.345+0000 7effc6ffd640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7effb40a3c40 0x7effb40a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7effc0009aa0 tx=0x7effc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.349+0000 7effbffff640 1 -- 172.21.15.32:0/799125604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effb804b020 con 0x7effc8107d20 2024-04-16T05:32:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.533+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effc8062440 con 0x7effb40a3c40 2024-04-16T05:32:58.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.543+0000 7effbffff640 1 -- 172.21.15.32:0/799125604 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340522 (secure 0 0 0) 0x7effc8062440 con 0x7effb40a3c40 2024-04-16T05:32:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-16T05:32:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.546+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7effb40a3c40 msgr2=0x7effb40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.546+0000 7effcd4a5640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7effb40a3c40 0x7effb40a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7effc0009aa0 tx=0x7effc0005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.546+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 msgr2=0x7effc819f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.546+0000 7effcd4a5640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 0x7effc819f840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7effb8037040 tx=0x7effb8032f20 comp rx=0 tx=0).stop 2024-04-16T05:32:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.546+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 shutdown_connections 2024-04-16T05:32:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.547+0000 7effcd4a5640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7effb40a3c40 0x7effb40a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.547+0000 7effcd4a5640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc8107d20 0x7effc819f840 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.547+0000 7effcd4a5640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc81053f0 0x7effc81987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.547+0000 7effcd4a5640 1 --2- 172.21.15.32:0/799125604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc8102ac0 0x7effc8198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.547+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 >> 172.21.15.32:0/799125604 conn(0x7effc80fc4b0 msgr2=0x7effc8103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.547+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 shutdown_connections 2024-04-16T05:32:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.547+0000 7effcd4a5640 1 -- 172.21.15.32:0/799125604 wait complete. 2024-04-16T05:32:58.556 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-16T05:32:56.910350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615527,"num_objects":38,"num_object_clones":0,"num_object_copies":115,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":152,"num_read_kb":153,"num_write":267,"num_write_kb":2990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356628,"kb_used_data":11596,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585196,"statfs":{"total":1151940427776,"available":1151575240704,"internally_reserved":0,"allocated":11874304,"data_stored":7516410,"data_compressed":284526,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22749,"num_objects":14,"num_object_clones":0,"num_object_copies":43,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":16,"num_read_kb":34,"num_write":41,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211835+0000","last_change":"2024-04-16T05:32:34.759518+0000","last_active":"2024-04-16T05:32:40.211835+0000","last_peered":"2024-04-16T05:32:40.211835+0000","last_clean":"2024-04-16T05:32:40.211835+0000","last_became_active":"2024-04-16T05:32:34.759155+0000","last_became_peered":"2024-04-16T05:32:34.759155+0000","last_unstale":"2024-04-16T05:32:40.211835+0000","last_undegraded":"2024-04-16T05:32:40.211835+0000","last_fullsized":"2024-04-16T05:32:40.211835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:39.935886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210252+0000","last_change":"2024-04-16T05:32:34.758553+0000","last_active":"2024-04-16T05:32:40.210252+0000","last_peered":"2024-04-16T05:32:40.210252+0000","last_clean":"2024-04-16T05:32:40.210252+0000","last_became_active":"2024-04-16T05:32:34.758242+0000","last_became_peered":"2024-04-16T05:32:34.758242+0000","last_unstale":"2024-04-16T05:32:40.210252+0000","last_undegraded":"2024-04-16T05:32:40.210252+0000","last_fullsized":"2024-04-16T05:32:40.210252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:09.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209914+0000","last_change":"2024-04-16T05:32:34.754815+0000","last_active":"2024-04-16T05:32:40.209914+0000","last_peered":"2024-04-16T05:32:40.209914+0000","last_clean":"2024-04-16T05:32:40.209914+0000","last_became_active":"2024-04-16T05:32:34.754488+0000","last_became_peered":"2024-04-16T05:32:34.754488+0000","last_unstale":"2024-04-16T05:32:40.209914+0000","last_undegraded":"2024-04-16T05:32:40.209914+0000","last_fullsized":"2024-04-16T05:32:40.209914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:10:32.379770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211377+0000","last_change":"2024-04-16T05:32:34.755251+0000","last_active":"2024-04-16T05:32:40.211377+0000","last_peered":"2024-04-16T05:32:40.211377+0000","last_clean":"2024-04-16T05:32:40.211377+0000","last_became_active":"2024-04-16T05:32:34.754945+0000","last_became_peered":"2024-04-16T05:32:34.754945+0000","last_unstale":"2024-04-16T05:32:40.211377+0000","last_undegraded":"2024-04-16T05:32:40.211377+0000","last_fullsized":"2024-04-16T05:32:40.211377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:50:11.218608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210799+0000","last_change":"2024-04-16T05:32:34.755623+0000","last_active":"2024-04-16T05:32:40.210799+0000","last_peered":"2024-04-16T05:32:40.210799+0000","last_clean":"2024-04-16T05:32:40.210799+0000","last_became_active":"2024-04-16T05:32:34.754205+0000","last_became_peered":"2024-04-16T05:32:34.754205+0000","last_unstale":"2024-04-16T05:32:40.210799+0000","last_undegraded":"2024-04-16T05:32:40.210799+0000","last_fullsized":"2024-04-16T05:32:40.210799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:34:07.998734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909442+0000","last_change":"2024-04-16T05:32:34.753767+0000","last_active":"2024-04-16T05:32:39.909442+0000","last_peered":"2024-04-16T05:32:39.909442+0000","last_clean":"2024-04-16T05:32:39.909442+0000","last_became_active":"2024-04-16T05:32:34.753448+0000","last_became_peered":"2024-04-16T05:32:34.753448+0000","last_unstale":"2024-04-16T05:32:39.909442+0000","last_undegraded":"2024-04-16T05:32:39.909442+0000","last_fullsized":"2024-04-16T05:32:39.909442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:27:42.798615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210573+0000","last_change":"2024-04-16T05:32:34.754049+0000","last_active":"2024-04-16T05:32:40.210573+0000","last_peered":"2024-04-16T05:32:40.210573+0000","last_clean":"2024-04-16T05:32:40.210573+0000","last_became_active":"2024-04-16T05:32:34.753757+0000","last_became_peered":"2024-04-16T05:32:34.753757+0000","last_unstale":"2024-04-16T05:32:40.210573+0000","last_undegraded":"2024-04-16T05:32:40.210573+0000","last_fullsized":"2024-04-16T05:32:40.210573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:42:26.747100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210211+0000","last_change":"2024-04-16T05:32:34.762950+0000","last_active":"2024-04-16T05:32:40.210211+0000","last_peered":"2024-04-16T05:32:40.210211+0000","last_clean":"2024-04-16T05:32:40.210211+0000","last_became_active":"2024-04-16T05:32:34.762073+0000","last_became_peered":"2024-04-16T05:32:34.762073+0000","last_unstale":"2024-04-16T05:32:40.210211+0000","last_undegraded":"2024-04-16T05:32:40.210211+0000","last_fullsized":"2024-04-16T05:32:40.210211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:57:26.623655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211186+0000","last_change":"2024-04-16T05:32:34.751027+0000","last_active":"2024-04-16T05:32:40.211186+0000","last_peered":"2024-04-16T05:32:40.211186+0000","last_clean":"2024-04-16T05:32:40.211186+0000","last_became_active":"2024-04-16T05:32:34.750697+0000","last_became_peered":"2024-04-16T05:32:34.750697+0000","last_unstale":"2024-04-16T05:32:40.211186+0000","last_undegraded":"2024-04-16T05:32:40.211186+0000","last_fullsized":"2024-04-16T05:32:40.211186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:39:33.834972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909872+0000","last_change":"2024-04-16T05:32:34.752293+0000","last_active":"2024-04-16T05:32:39.909872+0000","last_peered":"2024-04-16T05:32:39.909872+0000","last_clean":"2024-04-16T05:32:39.909872+0000","last_became_active":"2024-04-16T05:32:34.752008+0000","last_became_peered":"2024-04-16T05:32:34.752008+0000","last_unstale":"2024-04-16T05:32:39.909872+0000","last_undegraded":"2024-04-16T05:32:39.909872+0000","last_fullsized":"2024-04-16T05:32:39.909872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:02:17.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.212169+0000","last_change":"2024-04-16T05:32:34.760961+0000","last_active":"2024-04-16T05:32:40.212169+0000","last_peered":"2024-04-16T05:32:40.212169+0000","last_clean":"2024-04-16T05:32:40.212169+0000","last_became_active":"2024-04-16T05:32:34.760640+0000","last_became_peered":"2024-04-16T05:32:34.760640+0000","last_unstale":"2024-04-16T05:32:40.212169+0000","last_undegraded":"2024-04-16T05:32:40.212169+0000","last_fullsized":"2024-04-16T05:32:40.212169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:47:28.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211274+0000","last_change":"2024-04-16T05:32:34.748646+0000","last_active":"2024-04-16T05:32:40.211274+0000","last_peered":"2024-04-16T05:32:40.211274+0000","last_clean":"2024-04-16T05:32:40.211274+0000","last_became_active":"2024-04-16T05:32:34.748090+0000","last_became_peered":"2024-04-16T05:32:34.748090+0000","last_unstale":"2024-04-16T05:32:40.211274+0000","last_undegraded":"2024-04-16T05:32:40.211274+0000","last_fullsized":"2024-04-16T05:32:40.211274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:00:19.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207402+0000","last_change":"2024-04-16T05:32:34.747025+0000","last_active":"2024-04-16T05:32:40.207402+0000","last_peered":"2024-04-16T05:32:40.207402+0000","last_clean":"2024-04-16T05:32:40.207402+0000","last_became_active":"2024-04-16T05:32:34.746711+0000","last_became_peered":"2024-04-16T05:32:34.746711+0000","last_unstale":"2024-04-16T05:32:40.207402+0000","last_undegraded":"2024-04-16T05:32:40.207402+0000","last_fullsized":"2024-04-16T05:32:40.207402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:40:30.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209624+0000","last_change":"2024-04-16T05:32:34.756756+0000","last_active":"2024-04-16T05:32:40.209624+0000","last_peered":"2024-04-16T05:32:40.209624+0000","last_clean":"2024-04-16T05:32:40.209624+0000","last_became_active":"2024-04-16T05:32:34.756442+0000","last_became_peered":"2024-04-16T05:32:34.756442+0000","last_unstale":"2024-04-16T05:32:40.209624+0000","last_undegraded":"2024-04-16T05:32:40.209624+0000","last_fullsized":"2024-04-16T05:32:40.209624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:33.864944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209489+0000","last_change":"2024-04-16T05:32:34.765443+0000","last_active":"2024-04-16T05:32:40.209489+0000","last_peered":"2024-04-16T05:32:40.209489+0000","last_clean":"2024-04-16T05:32:40.209489+0000","last_became_active":"2024-04-16T05:32:34.765133+0000","last_became_peered":"2024-04-16T05:32:34.765133+0000","last_unstale":"2024-04-16T05:32:40.209489+0000","last_undegraded":"2024-04-16T05:32:40.209489+0000","last_fullsized":"2024-04-16T05:32:40.209489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:31.452184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211029+0000","last_change":"2024-04-16T05:32:34.757577+0000","last_active":"2024-04-16T05:32:40.211029+0000","last_peered":"2024-04-16T05:32:40.211029+0000","last_clean":"2024-04-16T05:32:40.211029+0000","last_became_active":"2024-04-16T05:32:34.753451+0000","last_became_peered":"2024-04-16T05:32:34.753451+0000","last_unstale":"2024-04-16T05:32:40.211029+0000","last_undegraded":"2024-04-16T05:32:40.211029+0000","last_fullsized":"2024-04-16T05:32:40.211029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:56.617824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211639+0000","last_change":"2024-04-16T05:32:34.759776+0000","last_active":"2024-04-16T05:32:40.211639+0000","last_peered":"2024-04-16T05:32:40.211639+0000","last_clean":"2024-04-16T05:32:40.211639+0000","last_became_active":"2024-04-16T05:32:34.759371+0000","last_became_peered":"2024-04-16T05:32:34.759371+0000","last_unstale":"2024-04-16T05:32:40.211639+0000","last_undegraded":"2024-04-16T05:32:40.211639+0000","last_fullsized":"2024-04-16T05:32:40.211639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:36.458515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909081+0000","last_change":"2024-04-16T05:32:34.747803+0000","last_active":"2024-04-16T05:32:39.909081+0000","last_peered":"2024-04-16T05:32:39.909081+0000","last_clean":"2024-04-16T05:32:39.909081+0000","last_became_active":"2024-04-16T05:32:34.747472+0000","last_became_peered":"2024-04-16T05:32:34.747472+0000","last_unstale":"2024-04-16T05:32:39.909081+0000","last_undegraded":"2024-04-16T05:32:39.909081+0000","last_fullsized":"2024-04-16T05:32:39.909081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:41.118574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909946+0000","last_change":"2024-04-16T05:32:34.754151+0000","last_active":"2024-04-16T05:32:39.909946+0000","last_peered":"2024-04-16T05:32:39.909946+0000","last_clean":"2024-04-16T05:32:39.909946+0000","last_became_active":"2024-04-16T05:32:34.753814+0000","last_became_peered":"2024-04-16T05:32:34.753814+0000","last_unstale":"2024-04-16T05:32:39.909946+0000","last_undegraded":"2024-04-16T05:32:39.909946+0000","last_fullsized":"2024-04-16T05:32:39.909946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:13:00.207978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909884+0000","last_change":"2024-04-16T05:32:38.938563+0000","last_active":"2024-04-16T05:32:39.909884+0000","last_peered":"2024-04-16T05:32:39.909884+0000","last_clean":"2024-04-16T05:32:39.909884+0000","last_became_active":"2024-04-16T05:32:38.938097+0000","last_became_peered":"2024-04-16T05:32:38.938097+0000","last_unstale":"2024-04-16T05:32:39.909884+0000","last_undegraded":"2024-04-16T05:32:39.909884+0000","last_fullsized":"2024-04-16T05:32:39.909884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:47:52.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909196+0000","last_change":"2024-04-16T05:32:33.735694+0000","last_active":"2024-04-16T05:32:39.909196+0000","last_peered":"2024-04-16T05:32:39.909196+0000","last_clean":"2024-04-16T05:32:39.909196+0000","last_became_active":"2024-04-16T05:32:33.735414+0000","last_became_peered":"2024-04-16T05:32:33.735414+0000","last_unstale":"2024-04-16T05:32:39.909196+0000","last_undegraded":"2024-04-16T05:32:39.909196+0000","last_fullsized":"2024-04-16T05:32:39.909196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:19:06.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210321+0000","last_change":"2024-04-16T05:32:34.762315+0000","last_active":"2024-04-16T05:32:40.210321+0000","last_peered":"2024-04-16T05:32:40.210321+0000","last_clean":"2024-04-16T05:32:40.210321+0000","last_became_active":"2024-04-16T05:32:34.761969+0000","last_became_peered":"2024-04-16T05:32:34.761969+0000","last_unstale":"2024-04-16T05:32:40.210321+0000","last_undegraded":"2024-04-16T05:32:40.210321+0000","last_fullsized":"2024-04-16T05:32:40.210321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:00:45.166966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210393+0000","last_change":"2024-04-16T05:32:38.940681+0000","last_active":"2024-04-16T05:32:40.210393+0000","last_peered":"2024-04-16T05:32:40.210393+0000","last_clean":"2024-04-16T05:32:40.210393+0000","last_became_active":"2024-04-16T05:32:38.940414+0000","last_became_peered":"2024-04-16T05:32:38.940414+0000","last_unstale":"2024-04-16T05:32:40.210393+0000","last_undegraded":"2024-04-16T05:32:40.210393+0000","last_fullsized":"2024-04-16T05:32:40.210393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:46:50.061607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210641+0000","last_change":"2024-04-16T05:32:33.748333+0000","last_active":"2024-04-16T05:32:40.210641+0000","last_peered":"2024-04-16T05:32:40.210641+0000","last_clean":"2024-04-16T05:32:40.210641+0000","last_became_active":"2024-04-16T05:32:33.748022+0000","last_became_peered":"2024-04-16T05:32:33.748022+0000","last_unstale":"2024-04-16T05:32:40.210641+0000","last_undegraded":"2024-04-16T05:32:40.210641+0000","last_fullsized":"2024-04-16T05:32:40.210641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:50:27.771245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210060+0000","last_change":"2024-04-16T05:32:34.764832+0000","last_active":"2024-04-16T05:32:40.210060+0000","last_peered":"2024-04-16T05:32:40.210060+0000","last_clean":"2024-04-16T05:32:40.210060+0000","last_became_active":"2024-04-16T05:32:34.763180+0000","last_became_peered":"2024-04-16T05:32:34.763180+0000","last_unstale":"2024-04-16T05:32:40.210060+0000","last_undegraded":"2024-04-16T05:32:40.210060+0000","last_fullsized":"2024-04-16T05:32:40.210060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:37:27.804376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210508+0000","last_change":"2024-04-16T05:32:38.937226+0000","last_active":"2024-04-16T05:32:40.210508+0000","last_peered":"2024-04-16T05:32:40.210508+0000","last_clean":"2024-04-16T05:32:40.210508+0000","last_became_active":"2024-04-16T05:32:38.936770+0000","last_became_peered":"2024-04-16T05:32:38.936770+0000","last_unstale":"2024-04-16T05:32:40.210508+0000","last_undegraded":"2024-04-16T05:32:40.210508+0000","last_fullsized":"2024-04-16T05:32:40.210508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:05:28.059175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211197+0000","last_change":"2024-04-16T05:32:33.735700+0000","last_active":"2024-04-16T05:32:40.211197+0000","last_peered":"2024-04-16T05:32:40.211197+0000","last_clean":"2024-04-16T05:32:40.211197+0000","last_became_active":"2024-04-16T05:32:33.735366+0000","last_became_peered":"2024-04-16T05:32:33.735366+0000","last_unstale":"2024-04-16T05:32:40.211197+0000","last_undegraded":"2024-04-16T05:32:40.211197+0000","last_fullsized":"2024-04-16T05:32:40.211197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:35:09.357197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210661+0000","last_change":"2024-04-16T05:32:34.758891+0000","last_active":"2024-04-16T05:32:40.210661+0000","last_peered":"2024-04-16T05:32:40.210661+0000","last_clean":"2024-04-16T05:32:40.210661+0000","last_became_active":"2024-04-16T05:32:34.758559+0000","last_became_peered":"2024-04-16T05:32:34.758559+0000","last_unstale":"2024-04-16T05:32:40.210661+0000","last_undegraded":"2024-04-16T05:32:40.210661+0000","last_fullsized":"2024-04-16T05:32:40.210661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:31.213992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210908+0000","last_change":"2024-04-16T05:32:38.937163+0000","last_active":"2024-04-16T05:32:40.210908+0000","last_peered":"2024-04-16T05:32:40.210908+0000","last_clean":"2024-04-16T05:32:40.210908+0000","last_became_active":"2024-04-16T05:32:38.936745+0000","last_became_peered":"2024-04-16T05:32:38.936745+0000","last_unstale":"2024-04-16T05:32:40.210908+0000","last_undegraded":"2024-04-16T05:32:40.210908+0000","last_fullsized":"2024-04-16T05:32:40.210908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:54:37.915041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211359+0000","last_change":"2024-04-16T05:32:33.731842+0000","last_active":"2024-04-16T05:32:40.211359+0000","last_peered":"2024-04-16T05:32:40.211359+0000","last_clean":"2024-04-16T05:32:40.211359+0000","last_became_active":"2024-04-16T05:32:33.731417+0000","last_became_peered":"2024-04-16T05:32:33.731417+0000","last_unstale":"2024-04-16T05:32:40.211359+0000","last_undegraded":"2024-04-16T05:32:40.211359+0000","last_fullsized":"2024-04-16T05:32:40.211359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:04:03.141533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211431+0000","last_change":"2024-04-16T05:32:34.755062+0000","last_active":"2024-04-16T05:32:40.211431+0000","last_peered":"2024-04-16T05:32:40.211431+0000","last_clean":"2024-04-16T05:32:40.211431+0000","last_became_active":"2024-04-16T05:32:34.754759+0000","last_became_peered":"2024-04-16T05:32:34.754759+0000","last_unstale":"2024-04-16T05:32:40.211431+0000","last_undegraded":"2024-04-16T05:32:40.211431+0000","last_fullsized":"2024-04-16T05:32:40.211431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:53:59.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909924+0000","last_change":"2024-04-16T05:32:38.935646+0000","last_active":"2024-04-16T05:32:39.909924+0000","last_peered":"2024-04-16T05:32:39.909924+0000","last_clean":"2024-04-16T05:32:39.909924+0000","last_became_active":"2024-04-16T05:32:38.935206+0000","last_became_peered":"2024-04-16T05:32:38.935206+0000","last_unstale":"2024-04-16T05:32:39.909924+0000","last_undegraded":"2024-04-16T05:32:39.909924+0000","last_fullsized":"2024-04-16T05:32:39.909924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:09.772136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211330+0000","last_change":"2024-04-16T05:32:33.739343+0000","last_active":"2024-04-16T05:32:40.211330+0000","last_peered":"2024-04-16T05:32:40.211330+0000","last_clean":"2024-04-16T05:32:40.211330+0000","last_became_active":"2024-04-16T05:32:33.738982+0000","last_became_peered":"2024-04-16T05:32:33.738982+0000","last_unstale":"2024-04-16T05:32:40.211330+0000","last_undegraded":"2024-04-16T05:32:40.211330+0000","last_fullsized":"2024-04-16T05:32:40.211330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:36.002804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211869+0000","last_change":"2024-04-16T05:32:34.749316+0000","last_active":"2024-04-16T05:32:40.211869+0000","last_peered":"2024-04-16T05:32:40.211869+0000","last_clean":"2024-04-16T05:32:40.211869+0000","last_became_active":"2024-04-16T05:32:34.748976+0000","last_became_peered":"2024-04-16T05:32:34.748976+0000","last_unstale":"2024-04-16T05:32:40.211869+0000","last_undegraded":"2024-04-16T05:32:40.211869+0000","last_fullsized":"2024-04-16T05:32:40.211869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:16:29.569402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909783+0000","last_change":"2024-04-16T05:32:38.938633+0000","last_active":"2024-04-16T05:32:39.909783+0000","last_peered":"2024-04-16T05:32:39.909783+0000","last_clean":"2024-04-16T05:32:39.909783+0000","last_became_active":"2024-04-16T05:32:38.938215+0000","last_became_peered":"2024-04-16T05:32:38.938215+0000","last_unstale":"2024-04-16T05:32:39.909783+0000","last_undegraded":"2024-04-16T05:32:39.909783+0000","last_fullsized":"2024-04-16T05:32:39.909783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:05.078109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:54.688634+0000","last_change":"2024-04-16T05:32:33.744262+0000","last_active":"2024-04-16T05:32:54.688634+0000","last_peered":"2024-04-16T05:32:54.688634+0000","last_clean":"2024-04-16T05:32:54.688634+0000","last_became_active":"2024-04-16T05:32:33.743866+0000","last_became_peered":"2024-04-16T05:32:33.743866+0000","last_unstale":"2024-04-16T05:32:54.688634+0000","last_undegraded":"2024-04-16T05:32:54.688634+0000","last_fullsized":"2024-04-16T05:32:54.688634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:22:03.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211548+0000","last_change":"2024-04-16T05:32:34.750201+0000","last_active":"2024-04-16T05:32:40.211548+0000","last_peered":"2024-04-16T05:32:40.211548+0000","last_clean":"2024-04-16T05:32:40.211548+0000","last_became_active":"2024-04-16T05:32:34.749900+0000","last_became_peered":"2024-04-16T05:32:34.749900+0000","last_unstale":"2024-04-16T05:32:40.211548+0000","last_undegraded":"2024-04-16T05:32:40.211548+0000","last_fullsized":"2024-04-16T05:32:40.211548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:42.708584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211575+0000","last_change":"2024-04-16T05:32:38.931277+0000","last_active":"2024-04-16T05:32:40.211575+0000","last_peered":"2024-04-16T05:32:40.211575+0000","last_clean":"2024-04-16T05:32:40.211575+0000","last_became_active":"2024-04-16T05:32:38.930888+0000","last_became_peered":"2024-04-16T05:32:38.930888+0000","last_unstale":"2024-04-16T05:32:40.211575+0000","last_undegraded":"2024-04-16T05:32:40.211575+0000","last_fullsized":"2024-04-16T05:32:40.211575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:27:31.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207137+0000","last_change":"2024-04-16T05:32:33.747740+0000","last_active":"2024-04-16T05:32:40.207137+0000","last_peered":"2024-04-16T05:32:40.207137+0000","last_clean":"2024-04-16T05:32:40.207137+0000","last_became_active":"2024-04-16T05:32:33.747313+0000","last_became_peered":"2024-04-16T05:32:33.747313+0000","last_unstale":"2024-04-16T05:32:40.207137+0000","last_undegraded":"2024-04-16T05:32:40.207137+0000","last_fullsized":"2024-04-16T05:32:40.207137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:05:03.937790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210108+0000","last_change":"2024-04-16T05:32:34.760075+0000","last_active":"2024-04-16T05:32:40.210108+0000","last_peered":"2024-04-16T05:32:40.210108+0000","last_clean":"2024-04-16T05:32:40.210108+0000","last_became_active":"2024-04-16T05:32:34.759569+0000","last_became_peered":"2024-04-16T05:32:34.759569+0000","last_unstale":"2024-04-16T05:32:40.210108+0000","last_undegraded":"2024-04-16T05:32:40.210108+0000","last_fullsized":"2024-04-16T05:32:40.210108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:54.222877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210117+0000","last_change":"2024-04-16T05:32:38.928568+0000","last_active":"2024-04-16T05:32:40.210117+0000","last_peered":"2024-04-16T05:32:40.210117+0000","last_clean":"2024-04-16T05:32:40.210117+0000","last_became_active":"2024-04-16T05:32:38.928109+0000","last_became_peered":"2024-04-16T05:32:38.928109+0000","last_unstale":"2024-04-16T05:32:40.210117+0000","last_undegraded":"2024-04-16T05:32:40.210117+0000","last_fullsized":"2024-04-16T05:32:40.210117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:24:14.293528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210535+0000","last_change":"2024-04-16T05:32:33.737803+0000","last_active":"2024-04-16T05:32:40.210535+0000","last_peered":"2024-04-16T05:32:40.210535+0000","last_clean":"2024-04-16T05:32:40.210535+0000","last_became_active":"2024-04-16T05:32:33.737448+0000","last_became_peered":"2024-04-16T05:32:33.737448+0000","last_unstale":"2024-04-16T05:32:40.210535+0000","last_undegraded":"2024-04-16T05:32:40.210535+0000","last_fullsized":"2024-04-16T05:32:40.210535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:44:54.302323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908551+0000","last_change":"2024-04-16T05:32:34.754070+0000","last_active":"2024-04-16T05:32:39.908551+0000","last_peered":"2024-04-16T05:32:39.908551+0000","last_clean":"2024-04-16T05:32:39.908551+0000","last_became_active":"2024-04-16T05:32:34.753717+0000","last_became_peered":"2024-04-16T05:32:34.753717+0000","last_unstale":"2024-04-16T05:32:39.908551+0000","last_undegraded":"2024-04-16T05:32:39.908551+0000","last_fullsized":"2024-04-16T05:32:39.908551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:58:56.738245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211277+0000","last_change":"2024-04-16T05:32:38.931594+0000","last_active":"2024-04-16T05:32:40.211277+0000","last_peered":"2024-04-16T05:32:40.211277+0000","last_clean":"2024-04-16T05:32:40.211277+0000","last_became_active":"2024-04-16T05:32:38.931188+0000","last_became_peered":"2024-04-16T05:32:38.931188+0000","last_unstale":"2024-04-16T05:32:40.211277+0000","last_undegraded":"2024-04-16T05:32:40.211277+0000","last_fullsized":"2024-04-16T05:32:40.211277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:55.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:55.697401+0000","last_change":"2024-04-16T05:32:33.739752+0000","last_active":"2024-04-16T05:32:55.697401+0000","last_peered":"2024-04-16T05:32:55.697401+0000","last_clean":"2024-04-16T05:32:55.697401+0000","last_became_active":"2024-04-16T05:32:33.739444+0000","last_became_peered":"2024-04-16T05:32:33.739444+0000","last_unstale":"2024-04-16T05:32:55.697401+0000","last_undegraded":"2024-04-16T05:32:55.697401+0000","last_fullsized":"2024-04-16T05:32:55.697401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:04:18.091618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211342+0000","last_change":"2024-04-16T05:32:34.759197+0000","last_active":"2024-04-16T05:32:40.211342+0000","last_peered":"2024-04-16T05:32:40.211342+0000","last_clean":"2024-04-16T05:32:40.211342+0000","last_became_active":"2024-04-16T05:32:34.758783+0000","last_became_peered":"2024-04-16T05:32:34.758783+0000","last_unstale":"2024-04-16T05:32:40.211342+0000","last_undegraded":"2024-04-16T05:32:40.211342+0000","last_fullsized":"2024-04-16T05:32:40.211342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:23:37.020254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908435+0000","last_change":"2024-04-16T05:32:38.936198+0000","last_active":"2024-04-16T05:32:39.908435+0000","last_peered":"2024-04-16T05:32:39.908435+0000","last_clean":"2024-04-16T05:32:39.908435+0000","last_became_active":"2024-04-16T05:32:38.935698+0000","last_became_peered":"2024-04-16T05:32:38.935698+0000","last_unstale":"2024-04-16T05:32:39.908435+0000","last_undegraded":"2024-04-16T05:32:39.908435+0000","last_fullsized":"2024-04-16T05:32:39.908435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:17:35.602408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:53.684390+0000","last_change":"2024-04-16T05:32:33.737653+0000","last_active":"2024-04-16T05:32:53.684390+0000","last_peered":"2024-04-16T05:32:53.684390+0000","last_clean":"2024-04-16T05:32:53.684390+0000","last_became_active":"2024-04-16T05:32:33.737342+0000","last_became_peered":"2024-04-16T05:32:33.737342+0000","last_unstale":"2024-04-16T05:32:53.684390+0000","last_undegraded":"2024-04-16T05:32:53.684390+0000","last_fullsized":"2024-04-16T05:32:53.684390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:17:35.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210162+0000","last_change":"2024-04-16T05:32:34.765722+0000","last_active":"2024-04-16T05:32:40.210162+0000","last_peered":"2024-04-16T05:32:40.210162+0000","last_clean":"2024-04-16T05:32:40.210162+0000","last_became_active":"2024-04-16T05:32:34.765413+0000","last_became_peered":"2024-04-16T05:32:34.765413+0000","last_unstale":"2024-04-16T05:32:40.210162+0000","last_undegraded":"2024-04-16T05:32:40.210162+0000","last_fullsized":"2024-04-16T05:32:40.210162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:03.884041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210677+0000","last_change":"2024-04-16T05:32:38.936186+0000","last_active":"2024-04-16T05:32:40.210677+0000","last_peered":"2024-04-16T05:32:40.210677+0000","last_clean":"2024-04-16T05:32:40.210677+0000","last_became_active":"2024-04-16T05:32:38.935738+0000","last_became_peered":"2024-04-16T05:32:38.935738+0000","last_unstale":"2024-04-16T05:32:40.210677+0000","last_undegraded":"2024-04-16T05:32:40.210677+0000","last_fullsized":"2024-04-16T05:32:40.210677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:10.519095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211300+0000","last_change":"2024-04-16T05:32:33.742991+0000","last_active":"2024-04-16T05:32:40.211300+0000","last_peered":"2024-04-16T05:32:40.211300+0000","last_clean":"2024-04-16T05:32:40.211300+0000","last_became_active":"2024-04-16T05:32:33.742659+0000","last_became_peered":"2024-04-16T05:32:33.742659+0000","last_unstale":"2024-04-16T05:32:40.211300+0000","last_undegraded":"2024-04-16T05:32:40.211300+0000","last_fullsized":"2024-04-16T05:32:40.211300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:23.756123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210782+0000","last_change":"2024-04-16T05:32:34.759926+0000","last_active":"2024-04-16T05:32:40.210782+0000","last_peered":"2024-04-16T05:32:40.210782+0000","last_clean":"2024-04-16T05:32:40.210782+0000","last_became_active":"2024-04-16T05:32:34.759626+0000","last_became_peered":"2024-04-16T05:32:34.759626+0000","last_unstale":"2024-04-16T05:32:40.210782+0000","last_undegraded":"2024-04-16T05:32:40.210782+0000","last_fullsized":"2024-04-16T05:32:40.210782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:50.314638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908590+0000","last_change":"2024-04-16T05:32:38.942135+0000","last_active":"2024-04-16T05:32:39.908590+0000","last_peered":"2024-04-16T05:32:39.908590+0000","last_clean":"2024-04-16T05:32:39.908590+0000","last_became_active":"2024-04-16T05:32:38.941534+0000","last_became_peered":"2024-04-16T05:32:38.941534+0000","last_unstale":"2024-04-16T05:32:39.908590+0000","last_undegraded":"2024-04-16T05:32:39.908590+0000","last_fullsized":"2024-04-16T05:32:39.908590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:20.494516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'19","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:56.701122+0000","last_change":"2024-04-16T05:32:33.734275+0000","last_active":"2024-04-16T05:32:56.701122+0000","last_peered":"2024-04-16T05:32:56.701122+0000","last_clean":"2024-04-16T05:32:56.701122+0000","last_became_active":"2024-04-16T05:32:33.733906+0000","last_became_peered":"2024-04-16T05:32:33.733906+0000","last_unstale":"2024-04-16T05:32:56.701122+0000","last_undegraded":"2024-04-16T05:32:56.701122+0000","last_fullsized":"2024-04-16T05:32:56.701122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:39:23.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210895+0000","last_change":"2024-04-16T05:32:34.750179+0000","last_active":"2024-04-16T05:32:40.210895+0000","last_peered":"2024-04-16T05:32:40.210895+0000","last_clean":"2024-04-16T05:32:40.210895+0000","last_became_active":"2024-04-16T05:32:34.749773+0000","last_became_peered":"2024-04-16T05:32:34.749773+0000","last_unstale":"2024-04-16T05:32:40.210895+0000","last_undegraded":"2024-04-16T05:32:40.210895+0000","last_fullsized":"2024-04-16T05:32:40.210895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:37.621191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211950+0000","last_change":"2024-04-16T05:32:38.935690+0000","last_active":"2024-04-16T05:32:40.211950+0000","last_peered":"2024-04-16T05:32:40.211950+0000","last_clean":"2024-04-16T05:32:40.211950+0000","last_became_active":"2024-04-16T05:32:38.935282+0000","last_became_peered":"2024-04-16T05:32:38.935282+0000","last_unstale":"2024-04-16T05:32:40.211950+0000","last_undegraded":"2024-04-16T05:32:40.211950+0000","last_fullsized":"2024-04-16T05:32:40.211950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:25:00.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909684+0000","last_change":"2024-04-16T05:32:33.736555+0000","last_active":"2024-04-16T05:32:39.909684+0000","last_peered":"2024-04-16T05:32:39.909684+0000","last_clean":"2024-04-16T05:32:39.909684+0000","last_became_active":"2024-04-16T05:32:33.735839+0000","last_became_peered":"2024-04-16T05:32:33.735839+0000","last_unstale":"2024-04-16T05:32:39.909684+0000","last_undegraded":"2024-04-16T05:32:39.909684+0000","last_fullsized":"2024-04-16T05:32:39.909684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:34.340099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207386+0000","last_change":"2024-04-16T05:32:34.748760+0000","last_active":"2024-04-16T05:32:40.207386+0000","last_peered":"2024-04-16T05:32:40.207386+0000","last_clean":"2024-04-16T05:32:40.207386+0000","last_became_active":"2024-04-16T05:32:34.748432+0000","last_became_peered":"2024-04-16T05:32:34.748432+0000","last_unstale":"2024-04-16T05:32:40.207386+0000","last_undegraded":"2024-04-16T05:32:40.207386+0000","last_fullsized":"2024-04-16T05:32:40.207386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:12:42.859342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211839+0000","last_change":"2024-04-16T05:32:38.929705+0000","last_active":"2024-04-16T05:32:40.211839+0000","last_peered":"2024-04-16T05:32:40.211839+0000","last_clean":"2024-04-16T05:32:40.211839+0000","last_became_active":"2024-04-16T05:32:38.929282+0000","last_became_peered":"2024-04-16T05:32:38.929282+0000","last_unstale":"2024-04-16T05:32:40.211839+0000","last_undegraded":"2024-04-16T05:32:40.211839+0000","last_fullsized":"2024-04-16T05:32:40.211839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:51:27.763600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211509+0000","last_change":"2024-04-16T05:32:33.746371+0000","last_active":"2024-04-16T05:32:40.211509+0000","last_peered":"2024-04-16T05:32:40.211509+0000","last_clean":"2024-04-16T05:32:40.211509+0000","last_became_active":"2024-04-16T05:32:33.746077+0000","last_became_peered":"2024-04-16T05:32:33.746077+0000","last_unstale":"2024-04-16T05:32:40.211509+0000","last_undegraded":"2024-04-16T05:32:40.211509+0000","last_fullsized":"2024-04-16T05:32:40.211509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:57:43.193942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211892+0000","last_change":"2024-04-16T05:32:34.753196+0000","last_active":"2024-04-16T05:32:40.211892+0000","last_peered":"2024-04-16T05:32:40.211892+0000","last_clean":"2024-04-16T05:32:40.211892+0000","last_became_active":"2024-04-16T05:32:34.752881+0000","last_became_peered":"2024-04-16T05:32:34.752881+0000","last_unstale":"2024-04-16T05:32:40.211892+0000","last_undegraded":"2024-04-16T05:32:40.211892+0000","last_fullsized":"2024-04-16T05:32:40.211892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:35:56.715394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908905+0000","last_change":"2024-04-16T05:32:38.929518+0000","last_active":"2024-04-16T05:32:39.908905+0000","last_peered":"2024-04-16T05:32:39.908905+0000","last_clean":"2024-04-16T05:32:39.908905+0000","last_became_active":"2024-04-16T05:32:38.928980+0000","last_became_peered":"2024-04-16T05:32:38.928980+0000","last_unstale":"2024-04-16T05:32:39.908905+0000","last_undegraded":"2024-04-16T05:32:39.908905+0000","last_fullsized":"2024-04-16T05:32:39.908905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:52:18.708378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:55.699580+0000","last_change":"2024-04-16T05:32:33.735921+0000","last_active":"2024-04-16T05:32:55.699580+0000","last_peered":"2024-04-16T05:32:55.699580+0000","last_clean":"2024-04-16T05:32:55.699580+0000","last_became_active":"2024-04-16T05:32:33.735572+0000","last_became_peered":"2024-04-16T05:32:33.735572+0000","last_unstale":"2024-04-16T05:32:55.699580+0000","last_undegraded":"2024-04-16T05:32:55.699580+0000","last_fullsized":"2024-04-16T05:32:55.699580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:46:56.512827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211156+0000","last_change":"2024-04-16T05:32:34.762172+0000","last_active":"2024-04-16T05:32:40.211156+0000","last_peered":"2024-04-16T05:32:40.211156+0000","last_clean":"2024-04-16T05:32:40.211156+0000","last_became_active":"2024-04-16T05:32:34.761914+0000","last_became_peered":"2024-04-16T05:32:34.761914+0000","last_unstale":"2024-04-16T05:32:40.211156+0000","last_undegraded":"2024-04-16T05:32:40.211156+0000","last_fullsized":"2024-04-16T05:32:40.211156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:47:52.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207653+0000","last_change":"2024-04-16T05:32:38.942967+0000","last_active":"2024-04-16T05:32:40.207653+0000","last_peered":"2024-04-16T05:32:40.207653+0000","last_clean":"2024-04-16T05:32:40.207653+0000","last_became_active":"2024-04-16T05:32:38.942523+0000","last_became_peered":"2024-04-16T05:32:38.942523+0000","last_unstale":"2024-04-16T05:32:40.207653+0000","last_undegraded":"2024-04-16T05:32:40.207653+0000","last_fullsized":"2024-04-16T05:32:40.207653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:39:43.956549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210517+0000","last_change":"2024-04-16T05:32:33.743944+0000","last_active":"2024-04-16T05:32:40.210517+0000","last_peered":"2024-04-16T05:32:40.210517+0000","last_clean":"2024-04-16T05:32:40.210517+0000","last_became_active":"2024-04-16T05:32:33.743559+0000","last_became_peered":"2024-04-16T05:32:33.743559+0000","last_unstale":"2024-04-16T05:32:40.210517+0000","last_undegraded":"2024-04-16T05:32:40.210517+0000","last_fullsized":"2024-04-16T05:32:40.210517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:08:48.889356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211257+0000","last_change":"2024-04-16T05:32:34.756741+0000","last_active":"2024-04-16T05:32:40.211257+0000","last_peered":"2024-04-16T05:32:40.211257+0000","last_clean":"2024-04-16T05:32:40.211257+0000","last_became_active":"2024-04-16T05:32:34.756426+0000","last_became_peered":"2024-04-16T05:32:34.756426+0000","last_unstale":"2024-04-16T05:32:40.211257+0000","last_undegraded":"2024-04-16T05:32:40.211257+0000","last_fullsized":"2024-04-16T05:32:40.211257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:09:30.493857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211192+0000","last_change":"2024-04-16T05:32:38.934818+0000","last_active":"2024-04-16T05:32:40.211192+0000","last_peered":"2024-04-16T05:32:40.211192+0000","last_clean":"2024-04-16T05:32:40.211192+0000","last_became_active":"2024-04-16T05:32:38.934335+0000","last_became_peered":"2024-04-16T05:32:38.934335+0000","last_unstale":"2024-04-16T05:32:40.211192+0000","last_undegraded":"2024-04-16T05:32:40.211192+0000","last_fullsized":"2024-04-16T05:32:40.211192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:02:41.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211225+0000","last_change":"2024-04-16T05:32:33.731951+0000","last_active":"2024-04-16T05:32:40.211225+0000","last_peered":"2024-04-16T05:32:40.211225+0000","last_clean":"2024-04-16T05:32:40.211225+0000","last_became_active":"2024-04-16T05:32:33.731649+0000","last_became_peered":"2024-04-16T05:32:33.731649+0000","last_unstale":"2024-04-16T05:32:40.211225+0000","last_undegraded":"2024-04-16T05:32:40.211225+0000","last_fullsized":"2024-04-16T05:32:40.211225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:41.258012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210772+0000","last_change":"2024-04-16T05:32:34.751884+0000","last_active":"2024-04-16T05:32:40.210772+0000","last_peered":"2024-04-16T05:32:40.210772+0000","last_clean":"2024-04-16T05:32:40.210772+0000","last_became_active":"2024-04-16T05:32:34.751559+0000","last_became_peered":"2024-04-16T05:32:34.751559+0000","last_unstale":"2024-04-16T05:32:40.210772+0000","last_undegraded":"2024-04-16T05:32:40.210772+0000","last_fullsized":"2024-04-16T05:32:40.210772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:32:59.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211741+0000","last_change":"2024-04-16T05:32:39.109330+0000","last_active":"2024-04-16T05:32:40.211741+0000","last_peered":"2024-04-16T05:32:40.211741+0000","last_clean":"2024-04-16T05:32:40.211741+0000","last_became_active":"2024-04-16T05:32:39.108718+0000","last_became_peered":"2024-04-16T05:32:39.108718+0000","last_unstale":"2024-04-16T05:32:40.211741+0000","last_undegraded":"2024-04-16T05:32:40.211741+0000","last_fullsized":"2024-04-16T05:32:40.211741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:36:55.418034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908777+0000","last_change":"2024-04-16T05:32:33.741603+0000","last_active":"2024-04-16T05:32:39.908777+0000","last_peered":"2024-04-16T05:32:39.908777+0000","last_clean":"2024-04-16T05:32:39.908777+0000","last_became_active":"2024-04-16T05:32:33.741184+0000","last_became_peered":"2024-04-16T05:32:33.741184+0000","last_unstale":"2024-04-16T05:32:39.908777+0000","last_undegraded":"2024-04-16T05:32:39.908777+0000","last_fullsized":"2024-04-16T05:32:39.908777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:08:38.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909748+0000","last_change":"2024-04-16T05:32:34.747768+0000","last_active":"2024-04-16T05:32:39.909748+0000","last_peered":"2024-04-16T05:32:39.909748+0000","last_clean":"2024-04-16T05:32:39.909748+0000","last_became_active":"2024-04-16T05:32:34.747440+0000","last_became_peered":"2024-04-16T05:32:34.747440+0000","last_unstale":"2024-04-16T05:32:39.909748+0000","last_undegraded":"2024-04-16T05:32:39.909748+0000","last_fullsized":"2024-04-16T05:32:39.909748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:14:31.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207501+0000","last_change":"2024-04-16T05:32:38.938673+0000","last_active":"2024-04-16T05:32:40.207501+0000","last_peered":"2024-04-16T05:32:40.207501+0000","last_clean":"2024-04-16T05:32:40.207501+0000","last_became_active":"2024-04-16T05:32:38.938252+0000","last_became_peered":"2024-04-16T05:32:38.938252+0000","last_unstale":"2024-04-16T05:32:40.207501+0000","last_undegraded":"2024-04-16T05:32:40.207501+0000","last_fullsized":"2024-04-16T05:32:40.207501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:06.620693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211618+0000","last_change":"2024-04-16T05:32:33.750812+0000","last_active":"2024-04-16T05:32:40.211618+0000","last_peered":"2024-04-16T05:32:40.211618+0000","last_clean":"2024-04-16T05:32:40.211618+0000","last_became_active":"2024-04-16T05:32:33.750514+0000","last_became_peered":"2024-04-16T05:32:33.750514+0000","last_unstale":"2024-04-16T05:32:40.211618+0000","last_undegraded":"2024-04-16T05:32:40.211618+0000","last_fullsized":"2024-04-16T05:32:40.211618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:53:43.075914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909523+0000","last_change":"2024-04-16T05:32:34.754987+0000","last_active":"2024-04-16T05:32:39.909523+0000","last_peered":"2024-04-16T05:32:39.909523+0000","last_clean":"2024-04-16T05:32:39.909523+0000","last_became_active":"2024-04-16T05:32:34.754698+0000","last_became_peered":"2024-04-16T05:32:34.754698+0000","last_unstale":"2024-04-16T05:32:39.909523+0000","last_undegraded":"2024-04-16T05:32:39.909523+0000","last_fullsized":"2024-04-16T05:32:39.909523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:16:00.066573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207543+0000","last_change":"2024-04-16T05:32:39.109394+0000","last_active":"2024-04-16T05:32:40.207543+0000","last_peered":"2024-04-16T05:32:40.207543+0000","last_clean":"2024-04-16T05:32:40.207543+0000","last_became_active":"2024-04-16T05:32:39.108924+0000","last_became_peered":"2024-04-16T05:32:39.108924+0000","last_unstale":"2024-04-16T05:32:40.207543+0000","last_undegraded":"2024-04-16T05:32:40.207543+0000","last_fullsized":"2024-04-16T05:32:40.207543+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:50:37.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210655+0000","last_change":"2024-04-16T05:32:34.758778+0000","last_active":"2024-04-16T05:32:40.210655+0000","last_peered":"2024-04-16T05:32:40.210655+0000","last_clean":"2024-04-16T05:32:40.210655+0000","last_became_active":"2024-04-16T05:32:34.757426+0000","last_became_peered":"2024-04-16T05:32:34.757426+0000","last_unstale":"2024-04-16T05:32:40.210655+0000","last_undegraded":"2024-04-16T05:32:40.210655+0000","last_fullsized":"2024-04-16T05:32:40.210655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:25.371669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207063+0000","last_change":"2024-04-16T05:32:33.748521+0000","last_active":"2024-04-16T05:32:40.207063+0000","last_peered":"2024-04-16T05:32:40.207063+0000","last_clean":"2024-04-16T05:32:40.207063+0000","last_became_active":"2024-04-16T05:32:33.748222+0000","last_became_peered":"2024-04-16T05:32:33.748222+0000","last_unstale":"2024-04-16T05:32:40.207063+0000","last_undegraded":"2024-04-16T05:32:40.207063+0000","last_fullsized":"2024-04-16T05:32:40.207063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:04.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:41.795311+0000","last_change":"2024-04-16T05:32:38.935995+0000","last_active":"2024-04-16T05:32:41.795311+0000","last_peered":"2024-04-16T05:32:41.795311+0000","last_clean":"2024-04-16T05:32:41.795311+0000","last_became_active":"2024-04-16T05:32:38.935511+0000","last_became_peered":"2024-04-16T05:32:38.935511+0000","last_unstale":"2024-04-16T05:32:41.795311+0000","last_undegraded":"2024-04-16T05:32:41.795311+0000","last_fullsized":"2024-04-16T05:32:41.795311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:17:08.144537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210195+0000","last_change":"2024-04-16T05:32:34.760249+0000","last_active":"2024-04-16T05:32:40.210195+0000","last_peered":"2024-04-16T05:32:40.210195+0000","last_clean":"2024-04-16T05:32:40.210195+0000","last_became_active":"2024-04-16T05:32:34.759735+0000","last_became_peered":"2024-04-16T05:32:34.759735+0000","last_unstale":"2024-04-16T05:32:40.210195+0000","last_undegraded":"2024-04-16T05:32:40.210195+0000","last_fullsized":"2024-04-16T05:32:40.210195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:13.349558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:55.695267+0000","last_change":"2024-04-16T05:32:33.731170+0000","last_active":"2024-04-16T05:32:55.695267+0000","last_peered":"2024-04-16T05:32:55.695267+0000","last_clean":"2024-04-16T05:32:55.695267+0000","last_became_active":"2024-04-16T05:32:33.730821+0000","last_became_peered":"2024-04-16T05:32:33.730821+0000","last_unstale":"2024-04-16T05:32:55.695267+0000","last_undegraded":"2024-04-16T05:32:55.695267+0000","last_fullsized":"2024-04-16T05:32:55.695267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:38:15.707590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210940+0000","last_change":"2024-04-16T05:31:24.926319+0000","last_active":"2024-04-16T05:32:40.210940+0000","last_peered":"2024-04-16T05:32:40.210940+0000","last_clean":"2024-04-16T05:32:40.210940+0000","last_became_active":"2024-04-16T05:31:24.925936+0000","last_became_peered":"2024-04-16T05:31:24.925936+0000","last_unstale":"2024-04-16T05:32:40.210940+0000","last_undegraded":"2024-04-16T05:32:40.210940+0000","last_fullsized":"2024-04-16T05:32:40.210940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_clean_scrub_stamp":"2024-04-16T05:27:57.876673+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:28:19.212127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211141+0000","last_change":"2024-04-16T05:32:39.110241+0000","last_active":"2024-04-16T05:32:40.211141+0000","last_peered":"2024-04-16T05:32:40.211141+0000","last_clean":"2024-04-16T05:32:40.211141+0000","last_became_active":"2024-04-16T05:32:39.109792+0000","last_became_peered":"2024-04-16T05:32:39.109792+0000","last_unstale":"2024-04-16T05:32:40.211141+0000","last_undegraded":"2024-04-16T05:32:40.211141+0000","last_fullsized":"2024-04-16T05:32:40.211141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:15:05.925370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207057+0000","last_change":"2024-04-16T05:32:34.751693+0000","last_active":"2024-04-16T05:32:40.207057+0000","last_peered":"2024-04-16T05:32:40.207057+0000","last_clean":"2024-04-16T05:32:40.207057+0000","last_became_active":"2024-04-16T05:32:34.751210+0000","last_became_peered":"2024-04-16T05:32:34.751210+0000","last_unstale":"2024-04-16T05:32:40.207057+0000","last_undegraded":"2024-04-16T05:32:40.207057+0000","last_fullsized":"2024-04-16T05:32:40.207057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:33:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:54.689833+0000","last_change":"2024-04-16T05:32:33.738310+0000","last_active":"2024-04-16T05:32:54.689833+0000","last_peered":"2024-04-16T05:32:54.689833+0000","last_clean":"2024-04-16T05:32:54.689833+0000","last_became_active":"2024-04-16T05:32:33.738012+0000","last_became_peered":"2024-04-16T05:32:33.738012+0000","last_unstale":"2024-04-16T05:32:54.689833+0000","last_undegraded":"2024-04-16T05:32:54.689833+0000","last_fullsized":"2024-04-16T05:32:54.689833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:22:18.311458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210947+0000","last_change":"2024-04-16T05:32:38.933465+0000","last_active":"2024-04-16T05:32:40.210947+0000","last_peered":"2024-04-16T05:32:40.210947+0000","last_clean":"2024-04-16T05:32:40.210947+0000","last_became_active":"2024-04-16T05:32:38.932391+0000","last_became_peered":"2024-04-16T05:32:38.932391+0000","last_unstale":"2024-04-16T05:32:40.210947+0000","last_undegraded":"2024-04-16T05:32:40.210947+0000","last_fullsized":"2024-04-16T05:32:40.210947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:39.494556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211010+0000","last_change":"2024-04-16T05:32:33.743241+0000","last_active":"2024-04-16T05:32:40.211010+0000","last_peered":"2024-04-16T05:32:40.211010+0000","last_clean":"2024-04-16T05:32:40.211010+0000","last_became_active":"2024-04-16T05:32:33.742887+0000","last_became_peered":"2024-04-16T05:32:33.742887+0000","last_unstale":"2024-04-16T05:32:40.211010+0000","last_undegraded":"2024-04-16T05:32:40.211010+0000","last_fullsized":"2024-04-16T05:32:40.211010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:24:38.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207636+0000","last_change":"2024-04-16T05:32:34.746228+0000","last_active":"2024-04-16T05:32:40.207636+0000","last_peered":"2024-04-16T05:32:40.207636+0000","last_clean":"2024-04-16T05:32:40.207636+0000","last_became_active":"2024-04-16T05:32:34.745883+0000","last_became_peered":"2024-04-16T05:32:34.745883+0000","last_unstale":"2024-04-16T05:32:40.207636+0000","last_undegraded":"2024-04-16T05:32:40.207636+0000","last_fullsized":"2024-04-16T05:32:40.207636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:33.656358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210193+0000","last_change":"2024-04-16T05:32:39.110156+0000","last_active":"2024-04-16T05:32:40.210193+0000","last_peered":"2024-04-16T05:32:40.210193+0000","last_clean":"2024-04-16T05:32:40.210193+0000","last_became_active":"2024-04-16T05:32:39.108958+0000","last_became_peered":"2024-04-16T05:32:39.108958+0000","last_unstale":"2024-04-16T05:32:40.210193+0000","last_undegraded":"2024-04-16T05:32:40.210193+0000","last_fullsized":"2024-04-16T05:32:40.210193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:27:58.658187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909018+0000","last_change":"2024-04-16T05:32:33.738179+0000","last_active":"2024-04-16T05:32:39.909018+0000","last_peered":"2024-04-16T05:32:39.909018+0000","last_clean":"2024-04-16T05:32:39.909018+0000","last_became_active":"2024-04-16T05:32:33.737841+0000","last_became_peered":"2024-04-16T05:32:33.737841+0000","last_unstale":"2024-04-16T05:32:39.909018+0000","last_undegraded":"2024-04-16T05:32:39.909018+0000","last_fullsized":"2024-04-16T05:32:39.909018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:13:11.210559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210651+0000","last_change":"2024-04-16T05:32:34.758142+0000","last_active":"2024-04-16T05:32:40.210651+0000","last_peered":"2024-04-16T05:32:40.210651+0000","last_clean":"2024-04-16T05:32:40.210651+0000","last_became_active":"2024-04-16T05:32:34.757825+0000","last_became_peered":"2024-04-16T05:32:34.757825+0000","last_unstale":"2024-04-16T05:32:40.210651+0000","last_undegraded":"2024-04-16T05:32:40.210651+0000","last_fullsized":"2024-04-16T05:32:40.210651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:37.575764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909464+0000","last_change":"2024-04-16T05:32:38.933327+0000","last_active":"2024-04-16T05:32:39.909464+0000","last_peered":"2024-04-16T05:32:39.909464+0000","last_clean":"2024-04-16T05:32:39.909464+0000","last_became_active":"2024-04-16T05:32:38.932808+0000","last_became_peered":"2024-04-16T05:32:38.932808+0000","last_unstale":"2024-04-16T05:32:39.909464+0000","last_undegraded":"2024-04-16T05:32:39.909464+0000","last_fullsized":"2024-04-16T05:32:39.909464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:03.217103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:53.682183+0000","last_change":"2024-04-16T05:32:33.742405+0000","last_active":"2024-04-16T05:32:53.682183+0000","last_peered":"2024-04-16T05:32:53.682183+0000","last_clean":"2024-04-16T05:32:53.682183+0000","last_became_active":"2024-04-16T05:32:33.742051+0000","last_became_peered":"2024-04-16T05:32:33.742051+0000","last_unstale":"2024-04-16T05:32:53.682183+0000","last_undegraded":"2024-04-16T05:32:53.682183+0000","last_fullsized":"2024-04-16T05:32:53.682183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:35:13.914483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210353+0000","last_change":"2024-04-16T05:32:34.763141+0000","last_active":"2024-04-16T05:32:40.210353+0000","last_peered":"2024-04-16T05:32:40.210353+0000","last_clean":"2024-04-16T05:32:40.210353+0000","last_became_active":"2024-04-16T05:32:34.762785+0000","last_became_peered":"2024-04-16T05:32:34.762785+0000","last_unstale":"2024-04-16T05:32:40.210353+0000","last_undegraded":"2024-04-16T05:32:40.210353+0000","last_fullsized":"2024-04-16T05:32:40.210353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:07:19.315100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211198+0000","last_change":"2024-04-16T05:32:39.110198+0000","last_active":"2024-04-16T05:32:40.211198+0000","last_peered":"2024-04-16T05:32:40.211198+0000","last_clean":"2024-04-16T05:32:40.211198+0000","last_became_active":"2024-04-16T05:32:39.109470+0000","last_became_peered":"2024-04-16T05:32:39.109470+0000","last_unstale":"2024-04-16T05:32:40.211198+0000","last_undegraded":"2024-04-16T05:32:40.211198+0000","last_fullsized":"2024-04-16T05:32:40.211198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:39:34.674270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:53.683311+0000","last_change":"2024-04-16T05:32:33.743031+0000","last_active":"2024-04-16T05:32:53.683311+0000","last_peered":"2024-04-16T05:32:53.683311+0000","last_clean":"2024-04-16T05:32:53.683311+0000","last_became_active":"2024-04-16T05:32:33.742771+0000","last_became_peered":"2024-04-16T05:32:33.742771+0000","last_unstale":"2024-04-16T05:32:53.683311+0000","last_undegraded":"2024-04-16T05:32:53.683311+0000","last_fullsized":"2024-04-16T05:32:53.683311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:06.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210672+0000","last_change":"2024-04-16T05:32:34.749428+0000","last_active":"2024-04-16T05:32:40.210672+0000","last_peered":"2024-04-16T05:32:40.210672+0000","last_clean":"2024-04-16T05:32:40.210672+0000","last_became_active":"2024-04-16T05:32:34.749087+0000","last_became_peered":"2024-04-16T05:32:34.749087+0000","last_unstale":"2024-04-16T05:32:40.210672+0000","last_undegraded":"2024-04-16T05:32:40.210672+0000","last_fullsized":"2024-04-16T05:32:40.210672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:59:25.124558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909084+0000","last_change":"2024-04-16T05:32:38.942407+0000","last_active":"2024-04-16T05:32:39.909084+0000","last_peered":"2024-04-16T05:32:39.909084+0000","last_clean":"2024-04-16T05:32:39.909084+0000","last_became_active":"2024-04-16T05:32:38.941731+0000","last_became_peered":"2024-04-16T05:32:38.941731+0000","last_unstale":"2024-04-16T05:32:39.909084+0000","last_undegraded":"2024-04-16T05:32:39.909084+0000","last_fullsized":"2024-04-16T05:32:39.909084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:42:27.700203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210971+0000","last_change":"2024-04-16T05:32:33.741487+0000","last_active":"2024-04-16T05:32:40.210971+0000","last_peered":"2024-04-16T05:32:40.210971+0000","last_clean":"2024-04-16T05:32:40.210971+0000","last_became_active":"2024-04-16T05:32:33.741140+0000","last_became_peered":"2024-04-16T05:32:33.741140+0000","last_unstale":"2024-04-16T05:32:40.210971+0000","last_undegraded":"2024-04-16T05:32:40.210971+0000","last_fullsized":"2024-04-16T05:32:40.210971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:21:36.086164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210303+0000","last_change":"2024-04-16T05:32:34.756608+0000","last_active":"2024-04-16T05:32:40.210303+0000","last_peered":"2024-04-16T05:32:40.210303+0000","last_clean":"2024-04-16T05:32:40.210303+0000","last_became_active":"2024-04-16T05:32:34.756296+0000","last_became_peered":"2024-04-16T05:32:34.756296+0000","last_unstale":"2024-04-16T05:32:40.210303+0000","last_undegraded":"2024-04-16T05:32:40.210303+0000","last_fullsized":"2024-04-16T05:32:40.210303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:28.397333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211158+0000","last_change":"2024-04-16T05:32:38.929876+0000","last_active":"2024-04-16T05:32:40.211158+0000","last_peered":"2024-04-16T05:32:40.211158+0000","last_clean":"2024-04-16T05:32:40.211158+0000","last_became_active":"2024-04-16T05:32:38.928626+0000","last_became_peered":"2024-04-16T05:32:38.928626+0000","last_unstale":"2024-04-16T05:32:40.211158+0000","last_undegraded":"2024-04-16T05:32:40.211158+0000","last_fullsized":"2024-04-16T05:32:40.211158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:08:09.697413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909341+0000","last_change":"2024-04-16T05:32:33.736047+0000","last_active":"2024-04-16T05:32:39.909341+0000","last_peered":"2024-04-16T05:32:39.909341+0000","last_clean":"2024-04-16T05:32:39.909341+0000","last_became_active":"2024-04-16T05:32:33.735683+0000","last_became_peered":"2024-04-16T05:32:33.735683+0000","last_unstale":"2024-04-16T05:32:39.909341+0000","last_undegraded":"2024-04-16T05:32:39.909341+0000","last_fullsized":"2024-04-16T05:32:39.909341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:58:32.969924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210666+0000","last_change":"2024-04-16T05:32:34.763558+0000","last_active":"2024-04-16T05:32:40.210666+0000","last_peered":"2024-04-16T05:32:40.210666+0000","last_clean":"2024-04-16T05:32:40.210666+0000","last_became_active":"2024-04-16T05:32:34.763190+0000","last_became_peered":"2024-04-16T05:32:34.763190+0000","last_unstale":"2024-04-16T05:32:40.210666+0000","last_undegraded":"2024-04-16T05:32:40.210666+0000","last_fullsized":"2024-04-16T05:32:40.210666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:38:31.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210956+0000","last_change":"2024-04-16T05:32:39.109426+0000","last_active":"2024-04-16T05:32:40.210956+0000","last_peered":"2024-04-16T05:32:40.210956+0000","last_clean":"2024-04-16T05:32:40.210956+0000","last_became_active":"2024-04-16T05:32:39.108970+0000","last_became_peered":"2024-04-16T05:32:39.108970+0000","last_unstale":"2024-04-16T05:32:40.210956+0000","last_undegraded":"2024-04-16T05:32:40.210956+0000","last_fullsized":"2024-04-16T05:32:40.210956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:54.824654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210912+0000","last_change":"2024-04-16T05:32:33.734983+0000","last_active":"2024-04-16T05:32:40.210912+0000","last_peered":"2024-04-16T05:32:40.210912+0000","last_clean":"2024-04-16T05:32:40.210912+0000","last_became_active":"2024-04-16T05:32:33.734609+0000","last_became_peered":"2024-04-16T05:32:33.734609+0000","last_unstale":"2024-04-16T05:32:40.210912+0000","last_undegraded":"2024-04-16T05:32:40.210912+0000","last_fullsized":"2024-04-16T05:32:40.210912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:35:59.436499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211678+0000","last_change":"2024-04-16T05:32:34.759811+0000","last_active":"2024-04-16T05:32:40.211678+0000","last_peered":"2024-04-16T05:32:40.211678+0000","last_clean":"2024-04-16T05:32:40.211678+0000","last_became_active":"2024-04-16T05:32:34.759499+0000","last_became_peered":"2024-04-16T05:32:34.759499+0000","last_unstale":"2024-04-16T05:32:40.211678+0000","last_undegraded":"2024-04-16T05:32:40.211678+0000","last_fullsized":"2024-04-16T05:32:40.211678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:46:28.554191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210570+0000","last_change":"2024-04-16T05:32:38.929021+0000","last_active":"2024-04-16T05:32:40.210570+0000","last_peered":"2024-04-16T05:32:40.210570+0000","last_clean":"2024-04-16T05:32:40.210570+0000","last_became_active":"2024-04-16T05:32:38.928574+0000","last_became_peered":"2024-04-16T05:32:38.928574+0000","last_unstale":"2024-04-16T05:32:40.210570+0000","last_undegraded":"2024-04-16T05:32:40.210570+0000","last_fullsized":"2024-04-16T05:32:40.210570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:11:35.693364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909966+0000","last_change":"2024-04-16T05:32:33.747352+0000","last_active":"2024-04-16T05:32:39.909966+0000","last_peered":"2024-04-16T05:32:39.909966+0000","last_clean":"2024-04-16T05:32:39.909966+0000","last_became_active":"2024-04-16T05:32:33.746989+0000","last_became_peered":"2024-04-16T05:32:33.746989+0000","last_unstale":"2024-04-16T05:32:39.909966+0000","last_undegraded":"2024-04-16T05:32:39.909966+0000","last_fullsized":"2024-04-16T05:32:39.909966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:38.422274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.212219+0000","last_change":"2024-04-16T05:32:34.760636+0000","last_active":"2024-04-16T05:32:40.212219+0000","last_peered":"2024-04-16T05:32:40.212219+0000","last_clean":"2024-04-16T05:32:40.212219+0000","last_became_active":"2024-04-16T05:32:34.760335+0000","last_became_peered":"2024-04-16T05:32:34.760335+0000","last_unstale":"2024-04-16T05:32:40.212219+0000","last_undegraded":"2024-04-16T05:32:40.212219+0000","last_fullsized":"2024-04-16T05:32:40.212219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:04:34.322799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209682+0000","last_change":"2024-04-16T05:32:38.928924+0000","last_active":"2024-04-16T05:32:40.209682+0000","last_peered":"2024-04-16T05:32:40.209682+0000","last_clean":"2024-04-16T05:32:40.209682+0000","last_became_active":"2024-04-16T05:32:38.928172+0000","last_became_peered":"2024-04-16T05:32:38.928172+0000","last_unstale":"2024-04-16T05:32:40.209682+0000","last_undegraded":"2024-04-16T05:32:40.209682+0000","last_fullsized":"2024-04-16T05:32:40.209682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:28:03.810439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211549+0000","last_change":"2024-04-16T05:32:33.746346+0000","last_active":"2024-04-16T05:32:40.211549+0000","last_peered":"2024-04-16T05:32:40.211549+0000","last_clean":"2024-04-16T05:32:40.211549+0000","last_became_active":"2024-04-16T05:32:33.746030+0000","last_became_peered":"2024-04-16T05:32:33.746030+0000","last_unstale":"2024-04-16T05:32:40.211549+0000","last_undegraded":"2024-04-16T05:32:40.211549+0000","last_fullsized":"2024-04-16T05:32:40.211549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:54:35.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211933+0000","last_change":"2024-04-16T05:32:34.752283+0000","last_active":"2024-04-16T05:32:40.211933+0000","last_peered":"2024-04-16T05:32:40.211933+0000","last_clean":"2024-04-16T05:32:40.211933+0000","last_became_active":"2024-04-16T05:32:34.751904+0000","last_became_peered":"2024-04-16T05:32:34.751904+0000","last_unstale":"2024-04-16T05:32:40.211933+0000","last_undegraded":"2024-04-16T05:32:40.211933+0000","last_fullsized":"2024-04-16T05:32:40.211933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:15:52.688683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210459+0000","last_change":"2024-04-16T05:32:38.934194+0000","last_active":"2024-04-16T05:32:40.210459+0000","last_peered":"2024-04-16T05:32:40.210459+0000","last_clean":"2024-04-16T05:32:40.210459+0000","last_became_active":"2024-04-16T05:32:38.933636+0000","last_became_peered":"2024-04-16T05:32:38.933636+0000","last_unstale":"2024-04-16T05:32:40.210459+0000","last_undegraded":"2024-04-16T05:32:40.210459+0000","last_fullsized":"2024-04-16T05:32:40.210459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:14:40.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207566+0000","last_change":"2024-04-16T05:32:33.744267+0000","last_active":"2024-04-16T05:32:40.207566+0000","last_peered":"2024-04-16T05:32:40.207566+0000","last_clean":"2024-04-16T05:32:40.207566+0000","last_became_active":"2024-04-16T05:32:33.743931+0000","last_became_peered":"2024-04-16T05:32:33.743931+0000","last_unstale":"2024-04-16T05:32:40.207566+0000","last_undegraded":"2024-04-16T05:32:40.207566+0000","last_fullsized":"2024-04-16T05:32:40.207566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:30.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.212242+0000","last_change":"2024-04-16T05:32:34.756833+0000","last_active":"2024-04-16T05:32:40.212242+0000","last_peered":"2024-04-16T05:32:40.212242+0000","last_clean":"2024-04-16T05:32:40.212242+0000","last_became_active":"2024-04-16T05:32:34.756436+0000","last_became_peered":"2024-04-16T05:32:34.756436+0000","last_unstale":"2024-04-16T05:32:40.212242+0000","last_undegraded":"2024-04-16T05:32:40.212242+0000","last_fullsized":"2024-04-16T05:32:40.212242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:20.732946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210535+0000","last_change":"2024-04-16T05:32:34.752744+0000","last_active":"2024-04-16T05:32:40.210535+0000","last_peered":"2024-04-16T05:32:40.210535+0000","last_clean":"2024-04-16T05:32:40.210535+0000","last_became_active":"2024-04-16T05:32:34.752442+0000","last_became_peered":"2024-04-16T05:32:34.752442+0000","last_unstale":"2024-04-16T05:32:40.210535+0000","last_undegraded":"2024-04-16T05:32:40.210535+0000","last_fullsized":"2024-04-16T05:32:40.210535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:01:23.644692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909450+0000","last_change":"2024-04-16T05:32:34.755118+0000","last_active":"2024-04-16T05:32:39.909450+0000","last_peered":"2024-04-16T05:32:39.909450+0000","last_clean":"2024-04-16T05:32:39.909450+0000","last_became_active":"2024-04-16T05:32:34.754824+0000","last_became_peered":"2024-04-16T05:32:34.754824+0000","last_unstale":"2024-04-16T05:32:39.909450+0000","last_undegraded":"2024-04-16T05:32:39.909450+0000","last_fullsized":"2024-04-16T05:32:39.909450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:58:37.860305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909398+0000","last_change":"2024-04-16T05:32:34.756144+0000","last_active":"2024-04-16T05:32:39.909398+0000","last_peered":"2024-04-16T05:32:39.909398+0000","last_clean":"2024-04-16T05:32:39.909398+0000","last_became_active":"2024-04-16T05:32:34.755841+0000","last_became_peered":"2024-04-16T05:32:34.755841+0000","last_unstale":"2024-04-16T05:32:39.909398+0000","last_undegraded":"2024-04-16T05:32:39.909398+0000","last_fullsized":"2024-04-16T05:32:39.909398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:23:00.482444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209802+0000","last_change":"2024-04-16T05:32:34.766204+0000","last_active":"2024-04-16T05:32:40.209802+0000","last_peered":"2024-04-16T05:32:40.209802+0000","last_clean":"2024-04-16T05:32:40.209802+0000","last_became_active":"2024-04-16T05:32:34.765709+0000","last_became_peered":"2024-04-16T05:32:34.765709+0000","last_unstale":"2024-04-16T05:32:40.209802+0000","last_undegraded":"2024-04-16T05:32:40.209802+0000","last_fullsized":"2024-04-16T05:32:40.209802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:23:53.926477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207522+0000","last_change":"2024-04-16T05:32:34.758513+0000","last_active":"2024-04-16T05:32:40.207522+0000","last_peered":"2024-04-16T05:32:40.207522+0000","last_clean":"2024-04-16T05:32:40.207522+0000","last_became_active":"2024-04-16T05:32:34.758145+0000","last_became_peered":"2024-04-16T05:32:34.758145+0000","last_unstale":"2024-04-16T05:32:40.207522+0000","last_undegraded":"2024-04-16T05:32:40.207522+0000","last_fullsized":"2024-04-16T05:32:40.207522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:09:17.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211322+0000","last_change":"2024-04-16T05:32:34.750651+0000","last_active":"2024-04-16T05:32:40.211322+0000","last_peered":"2024-04-16T05:32:40.211322+0000","last_clean":"2024-04-16T05:32:40.211322+0000","last_became_active":"2024-04-16T05:32:34.750348+0000","last_became_peered":"2024-04-16T05:32:34.750348+0000","last_unstale":"2024-04-16T05:32:40.211322+0000","last_undegraded":"2024-04-16T05:32:40.211322+0000","last_fullsized":"2024-04-16T05:32:40.211322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:29.282449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.910063+0000","last_change":"2024-04-16T05:32:34.750079+0000","last_active":"2024-04-16T05:32:39.910063+0000","last_peered":"2024-04-16T05:32:39.910063+0000","last_clean":"2024-04-16T05:32:39.910063+0000","last_became_active":"2024-04-16T05:32:34.749775+0000","last_became_peered":"2024-04-16T05:32:34.749775+0000","last_unstale":"2024-04-16T05:32:39.910063+0000","last_undegraded":"2024-04-16T05:32:39.910063+0000","last_fullsized":"2024-04-16T05:32:39.910063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:02.297258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210746+0000","last_change":"2024-04-16T05:32:34.752528+0000","last_active":"2024-04-16T05:32:40.210746+0000","last_peered":"2024-04-16T05:32:40.210746+0000","last_clean":"2024-04-16T05:32:40.210746+0000","last_became_active":"2024-04-16T05:32:34.752150+0000","last_became_peered":"2024-04-16T05:32:34.752150+0000","last_unstale":"2024-04-16T05:32:40.210746+0000","last_undegraded":"2024-04-16T05:32:40.210746+0000","last_fullsized":"2024-04-16T05:32:40.210746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:41:16.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210541+0000","last_change":"2024-04-16T05:32:34.758486+0000","last_active":"2024-04-16T05:32:40.210541+0000","last_peered":"2024-04-16T05:32:40.210541+0000","last_clean":"2024-04-16T05:32:40.210541+0000","last_became_active":"2024-04-16T05:32:34.754840+0000","last_became_peered":"2024-04-16T05:32:34.754840+0000","last_unstale":"2024-04-16T05:32:40.210541+0000","last_undegraded":"2024-04-16T05:32:40.210541+0000","last_fullsized":"2024-04-16T05:32:40.210541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:35.644337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211046+0000","last_change":"2024-04-16T05:32:34.758933+0000","last_active":"2024-04-16T05:32:40.211046+0000","last_peered":"2024-04-16T05:32:40.211046+0000","last_clean":"2024-04-16T05:32:40.211046+0000","last_became_active":"2024-04-16T05:32:34.758587+0000","last_became_peered":"2024-04-16T05:32:34.758587+0000","last_unstale":"2024-04-16T05:32:40.211046+0000","last_undegraded":"2024-04-16T05:32:40.211046+0000","last_fullsized":"2024-04-16T05:32:40.211046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:19:20.674248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210922+0000","last_change":"2024-04-16T05:32:34.750846+0000","last_active":"2024-04-16T05:32:40.210922+0000","last_peered":"2024-04-16T05:32:40.210922+0000","last_clean":"2024-04-16T05:32:40.210922+0000","last_became_active":"2024-04-16T05:32:34.750529+0000","last_became_peered":"2024-04-16T05:32:34.750529+0000","last_unstale":"2024-04-16T05:32:40.210922+0000","last_undegraded":"2024-04-16T05:32:40.210922+0000","last_fullsized":"2024-04-16T05:32:40.210922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:32.288459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207279+0000","last_change":"2024-04-16T05:32:34.752566+0000","last_active":"2024-04-16T05:32:40.207279+0000","last_peered":"2024-04-16T05:32:40.207279+0000","last_clean":"2024-04-16T05:32:40.207279+0000","last_became_active":"2024-04-16T05:32:34.752236+0000","last_became_peered":"2024-04-16T05:32:34.752236+0000","last_unstale":"2024-04-16T05:32:40.207279+0000","last_undegraded":"2024-04-16T05:32:40.207279+0000","last_fullsized":"2024-04-16T05:32:40.207279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:43:36.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211693+0000","last_change":"2024-04-16T05:32:34.750972+0000","last_active":"2024-04-16T05:32:40.211693+0000","last_peered":"2024-04-16T05:32:40.211693+0000","last_clean":"2024-04-16T05:32:40.211693+0000","last_became_active":"2024-04-16T05:32:34.750671+0000","last_became_peered":"2024-04-16T05:32:34.750671+0000","last_unstale":"2024-04-16T05:32:40.211693+0000","last_undegraded":"2024-04-16T05:32:40.211693+0000","last_fullsized":"2024-04-16T05:32:40.211693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:19:45.331816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283986,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25042,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":16,"num_read_kb":34,"num_write":85,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":39589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1073003,"data_compressed":50796,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":477999,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":477819,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1069375,"data_compressed":50796,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":481057,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":478693,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":481091,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":477319,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":477859,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1067765,"data_compressed":50796,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:32:58.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:32:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 -- 172.21.15.32:0/2995035852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101bb0 msgr2=0x7f1808102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 --2- 172.21.15.32:0/2995035852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101bb0 0x7f1808102030 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1800009ca0 tx=0x7f180002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 -- 172.21.15.32:0/2995035852 shutdown_connections 2024-04-16T05:32:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 --2- 172.21.15.32:0/2995035852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101bb0 0x7f1808102030 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 --2- 172.21.15.32:0/2995035852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1808101210 0x7f1808101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 --2- 172.21.15.32:0/2995035852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18081008b0 0x7f1808100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 -- 172.21.15.32:0/2995035852 >> 172.21.15.32:0/2995035852 conn(0x7f18080fc4b0 msgr2=0x7f18080fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 -- 172.21.15.32:0/2995035852 shutdown_connections 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.705+0000 7f180d91c640 1 -- 172.21.15.32:0/2995035852 wait complete. 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f180d91c640 1 Processor -- start 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f180d91c640 1 -- start start 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f180d91c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18081008b0 0x7f180819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f180d91c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101210 0x7f180819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f180d91c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1808101bb0 0x7f18081a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f180d91c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f180810fee0 con 0x7f1808101210 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f180d91c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f180810fd60 con 0x7f18081008b0 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f180d91c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1808110060 con 0x7f1808101bb0 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f17fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101210 0x7f180819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f17fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101210 0x7f180819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52440/0 (socket says 172.21.15.32:52440) 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f17fffff640 1 -- 172.21.15.32:0/3221863773 learned_addr learned my addr 172.21.15.32:0/3221863773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f18077fe640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1808101bb0 0x7f18081a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f1806ffd640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18081008b0 0x7f180819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f17fffff640 1 -- 172.21.15.32:0/3221863773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1808101bb0 msgr2=0x7f18081a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f17fffff640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1808101bb0 0x7f18081a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f17fffff640 1 -- 172.21.15.32:0/3221863773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18081008b0 msgr2=0x7f180819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f17fffff640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18081008b0 0x7f180819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f17fffff640 1 -- 172.21.15.32:0/3221863773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18081a20b0 con 0x7f1808101210 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.706+0000 7f1806ffd640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18081008b0 0x7f180819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.707+0000 7f17fffff640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101210 0x7f180819a9d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9f0 tx=0x7f17f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.707+0000 7f180c91a640 1 -- 172.21.15.32:0/3221863773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000cd80 con 0x7f1808101210 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.707+0000 7f180c91a640 1 -- 172.21.15.32:0/3221863773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f0010910 con 0x7f1808101210 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.707+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18081a23a0 con 0x7f1808101210 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.707+0000 7f180c91a640 1 -- 172.21.15.32:0/3221863773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0018990 con 0x7f1808101210 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.707+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18081a28f0 con 0x7f1808101210 2024-04-16T05:32:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.708+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180810b740 con 0x7f1808101210 2024-04-16T05:32:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.709+0000 7f180c91a640 1 -- 172.21.15.32:0/3221863773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f17f0010ac0 con 0x7f1808101210 2024-04-16T05:32:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.710+0000 7f180c91a640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f17f40a3c40 0x7f17f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:32:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.710+0000 7f1806ffd640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f17f40a3c40 0x7f17f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:32:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.710+0000 7f180c91a640 1 -- 172.21.15.32:0/3221863773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f17f00d0cc0 con 0x7f1808101210 2024-04-16T05:32:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.710+0000 7f1806ffd640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f17f40a3c40 0x7f17f40a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009b50 tx=0x7f17f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:32:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.714+0000 7f180c91a640 1 -- 172.21.15.32:0/3221863773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17f00d7080 con 0x7f1808101210 2024-04-16T05:32:58.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.901+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1808062440 con 0x7f17f40a3c40 2024-04-16T05:32:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.910+0000 7f180c91a640 1 -- 172.21.15.32:0/3221863773 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340522 (secure 0 0 0) 0x7f1808062440 con 0x7f17f40a3c40 2024-04-16T05:32:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:32:58.913 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-16T05:32:58.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f17f40a3c40 msgr2=0x7f17f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f17f40a3c40 0x7f17f40a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009b50 tx=0x7f17f8005f90 comp rx=0 tx=0).stop 2024-04-16T05:32:58.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101210 msgr2=0x7f180819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:32:58.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101210 0x7f180819a9d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9f0 tx=0x7f17f000eeb0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 shutdown_connections 2024-04-16T05:32:58.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f17f40a3c40 0x7f17f40a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1808101bb0 0x7f18081a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1808101210 0x7f180819a9d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 --2- 172.21.15.32:0/3221863773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18081008b0 0x7f180819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:32:58.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.913+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 >> 172.21.15.32:0/3221863773 conn(0x7f18080fc4b0 msgr2=0x7f18080fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:32:58.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.914+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 shutdown_connections 2024-04-16T05:32:58.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:32:58.914+0000 7f180d91c640 1 -- 172.21.15.32:0/3221863773 wait complete. 2024-04-16T05:32:58.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:58 smithi032 ceph-mon[32351]: pgmap v301: 129 pgs: 129 active+clean; 601 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-16T05:32:58.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:58 smithi032 ceph-mon[32351]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:58.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:58 smithi032 ceph-mon[32351]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:58.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:58 smithi032 ceph-mon[32351]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-16T05:32:58.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2937186221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:58.923 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3257914625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:58.923 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-16T05:32:56.910350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615527,"num_objects":38,"num_object_clones":0,"num_object_copies":115,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":152,"num_read_kb":153,"num_write":267,"num_write_kb":2990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356628,"kb_used_data":11596,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585196,"statfs":{"total":1151940427776,"available":1151575240704,"internally_reserved":0,"allocated":11874304,"data_stored":7516410,"data_compressed":284526,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22749,"num_objects":14,"num_object_clones":0,"num_object_copies":43,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":16,"num_read_kb":34,"num_write":41,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211835+0000","last_change":"2024-04-16T05:32:34.759518+0000","last_active":"2024-04-16T05:32:40.211835+0000","last_peered":"2024-04-16T05:32:40.211835+0000","last_clean":"2024-04-16T05:32:40.211835+0000","last_became_active":"2024-04-16T05:32:34.759155+0000","last_became_peered":"2024-04-16T05:32:34.759155+0000","last_unstale":"2024-04-16T05:32:40.211835+0000","last_undegraded":"2024-04-16T05:32:40.211835+0000","last_fullsized":"2024-04-16T05:32:40.211835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:39.935886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210252+0000","last_change":"2024-04-16T05:32:34.758553+0000","last_active":"2024-04-16T05:32:40.210252+0000","last_peered":"2024-04-16T05:32:40.210252+0000","last_clean":"2024-04-16T05:32:40.210252+0000","last_became_active":"2024-04-16T05:32:34.758242+0000","last_became_peered":"2024-04-16T05:32:34.758242+0000","last_unstale":"2024-04-16T05:32:40.210252+0000","last_undegraded":"2024-04-16T05:32:40.210252+0000","last_fullsized":"2024-04-16T05:32:40.210252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:09.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209914+0000","last_change":"2024-04-16T05:32:34.754815+0000","last_active":"2024-04-16T05:32:40.209914+0000","last_peered":"2024-04-16T05:32:40.209914+0000","last_clean":"2024-04-16T05:32:40.209914+0000","last_became_active":"2024-04-16T05:32:34.754488+0000","last_became_peered":"2024-04-16T05:32:34.754488+0000","last_unstale":"2024-04-16T05:32:40.209914+0000","last_undegraded":"2024-04-16T05:32:40.209914+0000","last_fullsized":"2024-04-16T05:32:40.209914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:10:32.379770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211377+0000","last_change":"2024-04-16T05:32:34.755251+0000","last_active":"2024-04-16T05:32:40.211377+0000","last_peered":"2024-04-16T05:32:40.211377+0000","last_clean":"2024-04-16T05:32:40.211377+0000","last_became_active":"2024-04-16T05:32:34.754945+0000","last_became_peered":"2024-04-16T05:32:34.754945+0000","last_unstale":"2024-04-16T05:32:40.211377+0000","last_undegraded":"2024-04-16T05:32:40.211377+0000","last_fullsized":"2024-04-16T05:32:40.211377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:50:11.218608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210799+0000","last_change":"2024-04-16T05:32:34.755623+0000","last_active":"2024-04-16T05:32:40.210799+0000","last_peered":"2024-04-16T05:32:40.210799+0000","last_clean":"2024-04-16T05:32:40.210799+0000","last_became_active":"2024-04-16T05:32:34.754205+0000","last_became_peered":"2024-04-16T05:32:34.754205+0000","last_unstale":"2024-04-16T05:32:40.210799+0000","last_undegraded":"2024-04-16T05:32:40.210799+0000","last_fullsized":"2024-04-16T05:32:40.210799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:34:07.998734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909442+0000","last_change":"2024-04-16T05:32:34.753767+0000","last_active":"2024-04-16T05:32:39.909442+0000","last_peered":"2024-04-16T05:32:39.909442+0000","last_clean":"2024-04-16T05:32:39.909442+0000","last_became_active":"2024-04-16T05:32:34.753448+0000","last_became_peered":"2024-04-16T05:32:34.753448+0000","last_unstale":"2024-04-16T05:32:39.909442+0000","last_undegraded":"2024-04-16T05:32:39.909442+0000","last_fullsized":"2024-04-16T05:32:39.909442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:27:42.798615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210573+0000","last_change":"2024-04-16T05:32:34.754049+0000","last_active":"2024-04-16T05:32:40.210573+0000","last_peered":"2024-04-16T05:32:40.210573+0000","last_clean":"2024-04-16T05:32:40.210573+0000","last_became_active":"2024-04-16T05:32:34.753757+0000","last_became_peered":"2024-04-16T05:32:34.753757+0000","last_unstale":"2024-04-16T05:32:40.210573+0000","last_undegraded":"2024-04-16T05:32:40.210573+0000","last_fullsized":"2024-04-16T05:32:40.210573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:42:26.747100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210211+0000","last_change":"2024-04-16T05:32:34.762950+0000","last_active":"2024-04-16T05:32:40.210211+0000","last_peered":"2024-04-16T05:32:40.210211+0000","last_clean":"2024-04-16T05:32:40.210211+0000","last_became_active":"2024-04-16T05:32:34.762073+0000","last_became_peered":"2024-04-16T05:32:34.762073+0000","last_unstale":"2024-04-16T05:32:40.210211+0000","last_undegraded":"2024-04-16T05:32:40.210211+0000","last_fullsized":"2024-04-16T05:32:40.210211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:57:26.623655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211186+0000","last_change":"2024-04-16T05:32:34.751027+0000","last_active":"2024-04-16T05:32:40.211186+0000","last_peered":"2024-04-16T05:32:40.211186+0000","last_clean":"2024-04-16T05:32:40.211186+0000","last_became_active":"2024-04-16T05:32:34.750697+0000","last_became_peered":"2024-04-16T05:32:34.750697+0000","last_unstale":"2024-04-16T05:32:40.211186+0000","last_undegraded":"2024-04-16T05:32:40.211186+0000","last_fullsized":"2024-04-16T05:32:40.211186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:39:33.834972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909872+0000","last_change":"2024-04-16T05:32:34.752293+0000","last_active":"2024-04-16T05:32:39.909872+0000","last_peered":"2024-04-16T05:32:39.909872+0000","last_clean":"2024-04-16T05:32:39.909872+0000","last_became_active":"2024-04-16T05:32:34.752008+0000","last_became_peered":"2024-04-16T05:32:34.752008+0000","last_unstale":"2024-04-16T05:32:39.909872+0000","last_undegraded":"2024-04-16T05:32:39.909872+0000","last_fullsized":"2024-04-16T05:32:39.909872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:02:17.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.212169+0000","last_change":"2024-04-16T05:32:34.760961+0000","last_active":"2024-04-16T05:32:40.212169+0000","last_peered":"2024-04-16T05:32:40.212169+0000","last_clean":"2024-04-16T05:32:40.212169+0000","last_became_active":"2024-04-16T05:32:34.760640+0000","last_became_peered":"2024-04-16T05:32:34.760640+0000","last_unstale":"2024-04-16T05:32:40.212169+0000","last_undegraded":"2024-04-16T05:32:40.212169+0000","last_fullsized":"2024-04-16T05:32:40.212169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:47:28.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211274+0000","last_change":"2024-04-16T05:32:34.748646+0000","last_active":"2024-04-16T05:32:40.211274+0000","last_peered":"2024-04-16T05:32:40.211274+0000","last_clean":"2024-04-16T05:32:40.211274+0000","last_became_active":"2024-04-16T05:32:34.748090+0000","last_became_peered":"2024-04-16T05:32:34.748090+0000","last_unstale":"2024-04-16T05:32:40.211274+0000","last_undegraded":"2024-04-16T05:32:40.211274+0000","last_fullsized":"2024-04-16T05:32:40.211274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:00:19.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207402+0000","last_change":"2024-04-16T05:32:34.747025+0000","last_active":"2024-04-16T05:32:40.207402+0000","last_peered":"2024-04-16T05:32:40.207402+0000","last_clean":"2024-04-16T05:32:40.207402+0000","last_became_active":"2024-04-16T05:32:34.746711+0000","last_became_peered":"2024-04-16T05:32:34.746711+0000","last_unstale":"2024-04-16T05:32:40.207402+0000","last_undegraded":"2024-04-16T05:32:40.207402+0000","last_fullsized":"2024-04-16T05:32:40.207402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:40:30.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209624+0000","last_change":"2024-04-16T05:32:34.756756+0000","last_active":"2024-04-16T05:32:40.209624+0000","last_peered":"2024-04-16T05:32:40.209624+0000","last_clean":"2024-04-16T05:32:40.209624+0000","last_became_active":"2024-04-16T05:32:34.756442+0000","last_became_peered":"2024-04-16T05:32:34.756442+0000","last_unstale":"2024-04-16T05:32:40.209624+0000","last_undegraded":"2024-04-16T05:32:40.209624+0000","last_fullsized":"2024-04-16T05:32:40.209624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:33.864944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209489+0000","last_change":"2024-04-16T05:32:34.765443+0000","last_active":"2024-04-16T05:32:40.209489+0000","last_peered":"2024-04-16T05:32:40.209489+0000","last_clean":"2024-04-16T05:32:40.209489+0000","last_became_active":"2024-04-16T05:32:34.765133+0000","last_became_peered":"2024-04-16T05:32:34.765133+0000","last_unstale":"2024-04-16T05:32:40.209489+0000","last_undegraded":"2024-04-16T05:32:40.209489+0000","last_fullsized":"2024-04-16T05:32:40.209489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:31.452184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211029+0000","last_change":"2024-04-16T05:32:34.757577+0000","last_active":"2024-04-16T05:32:40.211029+0000","last_peered":"2024-04-16T05:32:40.211029+0000","last_clean":"2024-04-16T05:32:40.211029+0000","last_became_active":"2024-04-16T05:32:34.753451+0000","last_became_peered":"2024-04-16T05:32:34.753451+0000","last_unstale":"2024-04-16T05:32:40.211029+0000","last_undegraded":"2024-04-16T05:32:40.211029+0000","last_fullsized":"2024-04-16T05:32:40.211029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:56.617824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211639+0000","last_change":"2024-04-16T05:32:34.759776+0000","last_active":"2024-04-16T05:32:40.211639+0000","last_peered":"2024-04-16T05:32:40.211639+0000","last_clean":"2024-04-16T05:32:40.211639+0000","last_became_active":"2024-04-16T05:32:34.759371+0000","last_became_peered":"2024-04-16T05:32:34.759371+0000","last_unstale":"2024-04-16T05:32:40.211639+0000","last_undegraded":"2024-04-16T05:32:40.211639+0000","last_fullsized":"2024-04-16T05:32:40.211639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:36.458515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909081+0000","last_change":"2024-04-16T05:32:34.747803+0000","last_active":"2024-04-16T05:32:39.909081+0000","last_peered":"2024-04-16T05:32:39.909081+0000","last_clean":"2024-04-16T05:32:39.909081+0000","last_became_active":"2024-04-16T05:32:34.747472+0000","last_became_peered":"2024-04-16T05:32:34.747472+0000","last_unstale":"2024-04-16T05:32:39.909081+0000","last_undegraded":"2024-04-16T05:32:39.909081+0000","last_fullsized":"2024-04-16T05:32:39.909081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:41.118574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909946+0000","last_change":"2024-04-16T05:32:34.754151+0000","last_active":"2024-04-16T05:32:39.909946+0000","last_peered":"2024-04-16T05:32:39.909946+0000","last_clean":"2024-04-16T05:32:39.909946+0000","last_became_active":"2024-04-16T05:32:34.753814+0000","last_became_peered":"2024-04-16T05:32:34.753814+0000","last_unstale":"2024-04-16T05:32:39.909946+0000","last_undegraded":"2024-04-16T05:32:39.909946+0000","last_fullsized":"2024-04-16T05:32:39.909946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:13:00.207978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909884+0000","last_change":"2024-04-16T05:32:38.938563+0000","last_active":"2024-04-16T05:32:39.909884+0000","last_peered":"2024-04-16T05:32:39.909884+0000","last_clean":"2024-04-16T05:32:39.909884+0000","last_became_active":"2024-04-16T05:32:38.938097+0000","last_became_peered":"2024-04-16T05:32:38.938097+0000","last_unstale":"2024-04-16T05:32:39.909884+0000","last_undegraded":"2024-04-16T05:32:39.909884+0000","last_fullsized":"2024-04-16T05:32:39.909884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:47:52.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909196+0000","last_change":"2024-04-16T05:32:33.735694+0000","last_active":"2024-04-16T05:32:39.909196+0000","last_peered":"2024-04-16T05:32:39.909196+0000","last_clean":"2024-04-16T05:32:39.909196+0000","last_became_active":"2024-04-16T05:32:33.735414+0000","last_became_peered":"2024-04-16T05:32:33.735414+0000","last_unstale":"2024-04-16T05:32:39.909196+0000","last_undegraded":"2024-04-16T05:32:39.909196+0000","last_fullsized":"2024-04-16T05:32:39.909196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:19:06.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210321+0000","last_change":"2024-04-16T05:32:34.762315+0000","last_active":"2024-04-16T05:32:40.210321+0000","last_peered":"2024-04-16T05:32:40.210321+0000","last_clean":"2024-04-16T05:32:40.210321+0000","last_became_active":"2024-04-16T05:32:34.761969+0000","last_became_peered":"2024-04-16T05:32:34.761969+0000","last_unstale":"2024-04-16T05:32:40.210321+0000","last_undegraded":"2024-04-16T05:32:40.210321+0000","last_fullsized":"2024-04-16T05:32:40.210321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:00:45.166966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210393+0000","last_change":"2024-04-16T05:32:38.940681+0000","last_active":"2024-04-16T05:32:40.210393+0000","last_peered":"2024-04-16T05:32:40.210393+0000","last_clean":"2024-04-16T05:32:40.210393+0000","last_became_active":"2024-04-16T05:32:38.940414+0000","last_became_peered":"2024-04-16T05:32:38.940414+0000","last_unstale":"2024-04-16T05:32:40.210393+0000","last_undegraded":"2024-04-16T05:32:40.210393+0000","last_fullsized":"2024-04-16T05:32:40.210393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:46:50.061607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210641+0000","last_change":"2024-04-16T05:32:33.748333+0000","last_active":"2024-04-16T05:32:40.210641+0000","last_peered":"2024-04-16T05:32:40.210641+0000","last_clean":"2024-04-16T05:32:40.210641+0000","last_became_active":"2024-04-16T05:32:33.748022+0000","last_became_peered":"2024-04-16T05:32:33.748022+0000","last_unstale":"2024-04-16T05:32:40.210641+0000","last_undegraded":"2024-04-16T05:32:40.210641+0000","last_fullsized":"2024-04-16T05:32:40.210641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:50:27.771245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210060+0000","last_change":"2024-04-16T05:32:34.764832+0000","last_active":"2024-04-16T05:32:40.210060+0000","last_peered":"2024-04-16T05:32:40.210060+0000","last_clean":"2024-04-16T05:32:40.210060+0000","last_became_active":"2024-04-16T05:32:34.763180+0000","last_became_peered":"2024-04-16T05:32:34.763180+0000","last_unstale":"2024-04-16T05:32:40.210060+0000","last_undegraded":"2024-04-16T05:32:40.210060+0000","last_fullsized":"2024-04-16T05:32:40.210060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:37:27.804376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210508+0000","last_change":"2024-04-16T05:32:38.937226+0000","last_active":"2024-04-16T05:32:40.210508+0000","last_peered":"2024-04-16T05:32:40.210508+0000","last_clean":"2024-04-16T05:32:40.210508+0000","last_became_active":"2024-04-16T05:32:38.936770+0000","last_became_peered":"2024-04-16T05:32:38.936770+0000","last_unstale":"2024-04-16T05:32:40.210508+0000","last_undegraded":"2024-04-16T05:32:40.210508+0000","last_fullsized":"2024-04-16T05:32:40.210508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:05:28.059175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211197+0000","last_change":"2024-04-16T05:32:33.735700+0000","last_active":"2024-04-16T05:32:40.211197+0000","last_peered":"2024-04-16T05:32:40.211197+0000","last_clean":"2024-04-16T05:32:40.211197+0000","last_became_active":"2024-04-16T05:32:33.735366+0000","last_became_peered":"2024-04-16T05:32:33.735366+0000","last_unstale":"2024-04-16T05:32:40.211197+0000","last_undegraded":"2024-04-16T05:32:40.211197+0000","last_fullsized":"2024-04-16T05:32:40.211197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:35:09.357197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210661+0000","last_change":"2024-04-16T05:32:34.758891+0000","last_active":"2024-04-16T05:32:40.210661+0000","last_peered":"2024-04-16T05:32:40.210661+0000","last_clean":"2024-04-16T05:32:40.210661+0000","last_became_active":"2024-04-16T05:32:34.758559+0000","last_became_peered":"2024-04-16T05:32:34.758559+0000","last_unstale":"2024-04-16T05:32:40.210661+0000","last_undegraded":"2024-04-16T05:32:40.210661+0000","last_fullsized":"2024-04-16T05:32:40.210661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:31.213992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210908+0000","last_change":"2024-04-16T05:32:38.937163+0000","last_active":"2024-04-16T05:32:40.210908+0000","last_peered":"2024-04-16T05:32:40.210908+0000","last_clean":"2024-04-16T05:32:40.210908+0000","last_became_active":"2024-04-16T05:32:38.936745+0000","last_became_peered":"2024-04-16T05:32:38.936745+0000","last_unstale":"2024-04-16T05:32:40.210908+0000","last_undegraded":"2024-04-16T05:32:40.210908+0000","last_fullsized":"2024-04-16T05:32:40.210908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:54:37.915041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211359+0000","last_change":"2024-04-16T05:32:33.731842+0000","last_active":"2024-04-16T05:32:40.211359+0000","last_peered":"2024-04-16T05:32:40.211359+0000","last_clean":"2024-04-16T05:32:40.211359+0000","last_became_active":"2024-04-16T05:32:33.731417+0000","last_became_peered":"2024-04-16T05:32:33.731417+0000","last_unstale":"2024-04-16T05:32:40.211359+0000","last_undegraded":"2024-04-16T05:32:40.211359+0000","last_fullsized":"2024-04-16T05:32:40.211359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:04:03.141533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211431+0000","last_change":"2024-04-16T05:32:34.755062+0000","last_active":"2024-04-16T05:32:40.211431+0000","last_peered":"2024-04-16T05:32:40.211431+0000","last_clean":"2024-04-16T05:32:40.211431+0000","last_became_active":"2024-04-16T05:32:34.754759+0000","last_became_peered":"2024-04-16T05:32:34.754759+0000","last_unstale":"2024-04-16T05:32:40.211431+0000","last_undegraded":"2024-04-16T05:32:40.211431+0000","last_fullsized":"2024-04-16T05:32:40.211431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:53:59.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909924+0000","last_change":"2024-04-16T05:32:38.935646+0000","last_active":"2024-04-16T05:32:39.909924+0000","last_peered":"2024-04-16T05:32:39.909924+0000","last_clean":"2024-04-16T05:32:39.909924+0000","last_became_active":"2024-04-16T05:32:38.935206+0000","last_became_peered":"2024-04-16T05:32:38.935206+0000","last_unstale":"2024-04-16T05:32:39.909924+0000","last_undegraded":"2024-04-16T05:32:39.909924+0000","last_fullsized":"2024-04-16T05:32:39.909924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:09.772136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211330+0000","last_change":"2024-04-16T05:32:33.739343+0000","last_active":"2024-04-16T05:32:40.211330+0000","last_peered":"2024-04-16T05:32:40.211330+0000","last_clean":"2024-04-16T05:32:40.211330+0000","last_became_active":"2024-04-16T05:32:33.738982+0000","last_became_peered":"2024-04-16T05:32:33.738982+0000","last_unstale":"2024-04-16T05:32:40.211330+0000","last_undegraded":"2024-04-16T05:32:40.211330+0000","last_fullsized":"2024-04-16T05:32:40.211330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:36.002804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211869+0000","last_change":"2024-04-16T05:32:34.749316+0000","last_active":"2024-04-16T05:32:40.211869+0000","last_peered":"2024-04-16T05:32:40.211869+0000","last_clean":"2024-04-16T05:32:40.211869+0000","last_became_active":"2024-04-16T05:32:34.748976+0000","last_became_peered":"2024-04-16T05:32:34.748976+0000","last_unstale":"2024-04-16T05:32:40.211869+0000","last_undegraded":"2024-04-16T05:32:40.211869+0000","last_fullsized":"2024-04-16T05:32:40.211869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:16:29.569402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909783+0000","last_change":"2024-04-16T05:32:38.938633+0000","last_active":"2024-04-16T05:32:39.909783+0000","last_peered":"2024-04-16T05:32:39.909783+0000","last_clean":"2024-04-16T05:32:39.909783+0000","last_became_active":"2024-04-16T05:32:38.938215+0000","last_became_peered":"2024-04-16T05:32:38.938215+0000","last_unstale":"2024-04-16T05:32:39.909783+0000","last_undegraded":"2024-04-16T05:32:39.909783+0000","last_fullsized":"2024-04-16T05:32:39.909783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:05.078109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:54.688634+0000","last_change":"2024-04-16T05:32:33.744262+0000","last_active":"2024-04-16T05:32:54.688634+0000","last_peered":"2024-04-16T05:32:54.688634+0000","last_clean":"2024-04-16T05:32:54.688634+0000","last_became_active":"2024-04-16T05:32:33.743866+0000","last_became_peered":"2024-04-16T05:32:33.743866+0000","last_unstale":"2024-04-16T05:32:54.688634+0000","last_undegraded":"2024-04-16T05:32:54.688634+0000","last_fullsized":"2024-04-16T05:32:54.688634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:22:03.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211548+0000","last_change":"2024-04-16T05:32:34.750201+0000","last_active":"2024-04-16T05:32:40.211548+0000","last_peered":"2024-04-16T05:32:40.211548+0000","last_clean":"2024-04-16T05:32:40.211548+0000","last_became_active":"2024-04-16T05:32:34.749900+0000","last_became_peered":"2024-04-16T05:32:34.749900+0000","last_unstale":"2024-04-16T05:32:40.211548+0000","last_undegraded":"2024-04-16T05:32:40.211548+0000","last_fullsized":"2024-04-16T05:32:40.211548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:42.708584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211575+0000","last_change":"2024-04-16T05:32:38.931277+0000","last_active":"2024-04-16T05:32:40.211575+0000","last_peered":"2024-04-16T05:32:40.211575+0000","last_clean":"2024-04-16T05:32:40.211575+0000","last_became_active":"2024-04-16T05:32:38.930888+0000","last_became_peered":"2024-04-16T05:32:38.930888+0000","last_unstale":"2024-04-16T05:32:40.211575+0000","last_undegraded":"2024-04-16T05:32:40.211575+0000","last_fullsized":"2024-04-16T05:32:40.211575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:27:31.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207137+0000","last_change":"2024-04-16T05:32:33.747740+0000","last_active":"2024-04-16T05:32:40.207137+0000","last_peered":"2024-04-16T05:32:40.207137+0000","last_clean":"2024-04-16T05:32:40.207137+0000","last_became_active":"2024-04-16T05:32:33.747313+0000","last_became_peered":"2024-04-16T05:32:33.747313+0000","last_unstale":"2024-04-16T05:32:40.207137+0000","last_undegraded":"2024-04-16T05:32:40.207137+0000","last_fullsized":"2024-04-16T05:32:40.207137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:05:03.937790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210108+0000","last_change":"2024-04-16T05:32:34.760075+0000","last_active":"2024-04-16T05:32:40.210108+0000","last_peered":"2024-04-16T05:32:40.210108+0000","last_clean":"2024-04-16T05:32:40.210108+0000","last_became_active":"2024-04-16T05:32:34.759569+0000","last_became_peered":"2024-04-16T05:32:34.759569+0000","last_unstale":"2024-04-16T05:32:40.210108+0000","last_undegraded":"2024-04-16T05:32:40.210108+0000","last_fullsized":"2024-04-16T05:32:40.210108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:54.222877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210117+0000","last_change":"2024-04-16T05:32:38.928568+0000","last_active":"2024-04-16T05:32:40.210117+0000","last_peered":"2024-04-16T05:32:40.210117+0000","last_clean":"2024-04-16T05:32:40.210117+0000","last_became_active":"2024-04-16T05:32:38.928109+0000","last_became_peered":"2024-04-16T05:32:38.928109+0000","last_unstale":"2024-04-16T05:32:40.210117+0000","last_undegraded":"2024-04-16T05:32:40.210117+0000","last_fullsized":"2024-04-16T05:32:40.210117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:24:14.293528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210535+0000","last_change":"2024-04-16T05:32:33.737803+0000","last_active":"2024-04-16T05:32:40.210535+0000","last_peered":"2024-04-16T05:32:40.210535+0000","last_clean":"2024-04-16T05:32:40.210535+0000","last_became_active":"2024-04-16T05:32:33.737448+0000","last_became_peered":"2024-04-16T05:32:33.737448+0000","last_unstale":"2024-04-16T05:32:40.210535+0000","last_undegraded":"2024-04-16T05:32:40.210535+0000","last_fullsized":"2024-04-16T05:32:40.210535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:44:54.302323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908551+0000","last_change":"2024-04-16T05:32:34.754070+0000","last_active":"2024-04-16T05:32:39.908551+0000","last_peered":"2024-04-16T05:32:39.908551+0000","last_clean":"2024-04-16T05:32:39.908551+0000","last_became_active":"2024-04-16T05:32:34.753717+0000","last_became_peered":"2024-04-16T05:32:34.753717+0000","last_unstale":"2024-04-16T05:32:39.908551+0000","last_undegraded":"2024-04-16T05:32:39.908551+0000","last_fullsized":"2024-04-16T05:32:39.908551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:58:56.738245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211277+0000","last_change":"2024-04-16T05:32:38.931594+0000","last_active":"2024-04-16T05:32:40.211277+0000","last_peered":"2024-04-16T05:32:40.211277+0000","last_clean":"2024-04-16T05:32:40.211277+0000","last_became_active":"2024-04-16T05:32:38.931188+0000","last_became_peered":"2024-04-16T05:32:38.931188+0000","last_unstale":"2024-04-16T05:32:40.211277+0000","last_undegraded":"2024-04-16T05:32:40.211277+0000","last_fullsized":"2024-04-16T05:32:40.211277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:55.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:55.697401+0000","last_change":"2024-04-16T05:32:33.739752+0000","last_active":"2024-04-16T05:32:55.697401+0000","last_peered":"2024-04-16T05:32:55.697401+0000","last_clean":"2024-04-16T05:32:55.697401+0000","last_became_active":"2024-04-16T05:32:33.739444+0000","last_became_peered":"2024-04-16T05:32:33.739444+0000","last_unstale":"2024-04-16T05:32:55.697401+0000","last_undegraded":"2024-04-16T05:32:55.697401+0000","last_fullsized":"2024-04-16T05:32:55.697401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:04:18.091618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211342+0000","last_change":"2024-04-16T05:32:34.759197+0000","last_active":"2024-04-16T05:32:40.211342+0000","last_peered":"2024-04-16T05:32:40.211342+0000","last_clean":"2024-04-16T05:32:40.211342+0000","last_became_active":"2024-04-16T05:32:34.758783+0000","last_became_peered":"2024-04-16T05:32:34.758783+0000","last_unstale":"2024-04-16T05:32:40.211342+0000","last_undegraded":"2024-04-16T05:32:40.211342+0000","last_fullsized":"2024-04-16T05:32:40.211342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:23:37.020254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908435+0000","last_change":"2024-04-16T05:32:38.936198+0000","last_active":"2024-04-16T05:32:39.908435+0000","last_peered":"2024-04-16T05:32:39.908435+0000","last_clean":"2024-04-16T05:32:39.908435+0000","last_became_active":"2024-04-16T05:32:38.935698+0000","last_became_peered":"2024-04-16T05:32:38.935698+0000","last_unstale":"2024-04-16T05:32:39.908435+0000","last_undegraded":"2024-04-16T05:32:39.908435+0000","last_fullsized":"2024-04-16T05:32:39.908435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:17:35.602408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:53.684390+0000","last_change":"2024-04-16T05:32:33.737653+0000","last_active":"2024-04-16T05:32:53.684390+0000","last_peered":"2024-04-16T05:32:53.684390+0000","last_clean":"2024-04-16T05:32:53.684390+0000","last_became_active":"2024-04-16T05:32:33.737342+0000","last_became_peered":"2024-04-16T05:32:33.737342+0000","last_unstale":"2024-04-16T05:32:53.684390+0000","last_undegraded":"2024-04-16T05:32:53.684390+0000","last_fullsized":"2024-04-16T05:32:53.684390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:17:35.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210162+0000","last_change":"2024-04-16T05:32:34.765722+0000","last_active":"2024-04-16T05:32:40.210162+0000","last_peered":"2024-04-16T05:32:40.210162+0000","last_clean":"2024-04-16T05:32:40.210162+0000","last_became_active":"2024-04-16T05:32:34.765413+0000","last_became_peered":"2024-04-16T05:32:34.765413+0000","last_unstale":"2024-04-16T05:32:40.210162+0000","last_undegraded":"2024-04-16T05:32:40.210162+0000","last_fullsized":"2024-04-16T05:32:40.210162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:03.884041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210677+0000","last_change":"2024-04-16T05:32:38.936186+0000","last_active":"2024-04-16T05:32:40.210677+0000","last_peered":"2024-04-16T05:32:40.210677+0000","last_clean":"2024-04-16T05:32:40.210677+0000","last_became_active":"2024-04-16T05:32:38.935738+0000","last_became_peered":"2024-04-16T05:32:38.935738+0000","last_unstale":"2024-04-16T05:32:40.210677+0000","last_undegraded":"2024-04-16T05:32:40.210677+0000","last_fullsized":"2024-04-16T05:32:40.210677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:10.519095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211300+0000","last_change":"2024-04-16T05:32:33.742991+0000","last_active":"2024-04-16T05:32:40.211300+0000","last_peered":"2024-04-16T05:32:40.211300+0000","last_clean":"2024-04-16T05:32:40.211300+0000","last_became_active":"2024-04-16T05:32:33.742659+0000","last_became_peered":"2024-04-16T05:32:33.742659+0000","last_unstale":"2024-04-16T05:32:40.211300+0000","last_undegraded":"2024-04-16T05:32:40.211300+0000","last_fullsized":"2024-04-16T05:32:40.211300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:23.756123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210782+0000","last_change":"2024-04-16T05:32:34.759926+0000","last_active":"2024-04-16T05:32:40.210782+0000","last_peered":"2024-04-16T05:32:40.210782+0000","last_clean":"2024-04-16T05:32:40.210782+0000","last_became_active":"2024-04-16T05:32:34.759626+0000","last_became_peered":"2024-04-16T05:32:34.759626+0000","last_unstale":"2024-04-16T05:32:40.210782+0000","last_undegraded":"2024-04-16T05:32:40.210782+0000","last_fullsized":"2024-04-16T05:32:40.210782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:50.314638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908590+0000","last_change":"2024-04-16T05:32:38.942135+0000","last_active":"2024-04-16T05:32:39.908590+0000","last_peered":"2024-04-16T05:32:39.908590+0000","last_clean":"2024-04-16T05:32:39.908590+0000","last_became_active":"2024-04-16T05:32:38.941534+0000","last_became_peered":"2024-04-16T05:32:38.941534+0000","last_unstale":"2024-04-16T05:32:39.908590+0000","last_undegraded":"2024-04-16T05:32:39.908590+0000","last_fullsized":"2024-04-16T05:32:39.908590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:20.494516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'19","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:56.701122+0000","last_change":"2024-04-16T05:32:33.734275+0000","last_active":"2024-04-16T05:32:56.701122+0000","last_peered":"2024-04-16T05:32:56.701122+0000","last_clean":"2024-04-16T05:32:56.701122+0000","last_became_active":"2024-04-16T05:32:33.733906+0000","last_became_peered":"2024-04-16T05:32:33.733906+0000","last_unstale":"2024-04-16T05:32:56.701122+0000","last_undegraded":"2024-04-16T05:32:56.701122+0000","last_fullsized":"2024-04-16T05:32:56.701122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:39:23.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210895+0000","last_change":"2024-04-16T05:32:34.750179+0000","last_active":"2024-04-16T05:32:40.210895+0000","last_peered":"2024-04-16T05:32:40.210895+0000","last_clean":"2024-04-16T05:32:40.210895+0000","last_became_active":"2024-04-16T05:32:34.749773+0000","last_became_peered":"2024-04-16T05:32:34.749773+0000","last_unstale":"2024-04-16T05:32:40.210895+0000","last_undegraded":"2024-04-16T05:32:40.210895+0000","last_fullsized":"2024-04-16T05:32:40.210895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:37.621191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211950+0000","last_change":"2024-04-16T05:32:38.935690+0000","last_active":"2024-04-16T05:32:40.211950+0000","last_peered":"2024-04-16T05:32:40.211950+0000","last_clean":"2024-04-16T05:32:40.211950+0000","last_became_active":"2024-04-16T05:32:38.935282+0000","last_became_peered":"2024-04-16T05:32:38.935282+0000","last_unstale":"2024-04-16T05:32:40.211950+0000","last_undegraded":"2024-04-16T05:32:40.211950+0000","last_fullsized":"2024-04-16T05:32:40.211950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:25:00.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909684+0000","last_change":"2024-04-16T05:32:33.736555+0000","last_active":"2024-04-16T05:32:39.909684+0000","last_peered":"2024-04-16T05:32:39.909684+0000","last_clean":"2024-04-16T05:32:39.909684+0000","last_became_active":"2024-04-16T05:32:33.735839+0000","last_became_peered":"2024-04-16T05:32:33.735839+0000","last_unstale":"2024-04-16T05:32:39.909684+0000","last_undegraded":"2024-04-16T05:32:39.909684+0000","last_fullsized":"2024-04-16T05:32:39.909684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:16:34.340099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207386+0000","last_change":"2024-04-16T05:32:34.748760+0000","last_active":"2024-04-16T05:32:40.207386+0000","last_peered":"2024-04-16T05:32:40.207386+0000","last_clean":"2024-04-16T05:32:40.207386+0000","last_became_active":"2024-04-16T05:32:34.748432+0000","last_became_peered":"2024-04-16T05:32:34.748432+0000","last_unstale":"2024-04-16T05:32:40.207386+0000","last_undegraded":"2024-04-16T05:32:40.207386+0000","last_fullsized":"2024-04-16T05:32:40.207386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:12:42.859342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211839+0000","last_change":"2024-04-16T05:32:38.929705+0000","last_active":"2024-04-16T05:32:40.211839+0000","last_peered":"2024-04-16T05:32:40.211839+0000","last_clean":"2024-04-16T05:32:40.211839+0000","last_became_active":"2024-04-16T05:32:38.929282+0000","last_became_peered":"2024-04-16T05:32:38.929282+0000","last_unstale":"2024-04-16T05:32:40.211839+0000","last_undegraded":"2024-04-16T05:32:40.211839+0000","last_fullsized":"2024-04-16T05:32:40.211839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:51:27.763600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211509+0000","last_change":"2024-04-16T05:32:33.746371+0000","last_active":"2024-04-16T05:32:40.211509+0000","last_peered":"2024-04-16T05:32:40.211509+0000","last_clean":"2024-04-16T05:32:40.211509+0000","last_became_active":"2024-04-16T05:32:33.746077+0000","last_became_peered":"2024-04-16T05:32:33.746077+0000","last_unstale":"2024-04-16T05:32:40.211509+0000","last_undegraded":"2024-04-16T05:32:40.211509+0000","last_fullsized":"2024-04-16T05:32:40.211509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:57:43.193942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211892+0000","last_change":"2024-04-16T05:32:34.753196+0000","last_active":"2024-04-16T05:32:40.211892+0000","last_peered":"2024-04-16T05:32:40.211892+0000","last_clean":"2024-04-16T05:32:40.211892+0000","last_became_active":"2024-04-16T05:32:34.752881+0000","last_became_peered":"2024-04-16T05:32:34.752881+0000","last_unstale":"2024-04-16T05:32:40.211892+0000","last_undegraded":"2024-04-16T05:32:40.211892+0000","last_fullsized":"2024-04-16T05:32:40.211892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:35:56.715394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908905+0000","last_change":"2024-04-16T05:32:38.929518+0000","last_active":"2024-04-16T05:32:39.908905+0000","last_peered":"2024-04-16T05:32:39.908905+0000","last_clean":"2024-04-16T05:32:39.908905+0000","last_became_active":"2024-04-16T05:32:38.928980+0000","last_became_peered":"2024-04-16T05:32:38.928980+0000","last_unstale":"2024-04-16T05:32:39.908905+0000","last_undegraded":"2024-04-16T05:32:39.908905+0000","last_fullsized":"2024-04-16T05:32:39.908905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:52:18.708378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:55.699580+0000","last_change":"2024-04-16T05:32:33.735921+0000","last_active":"2024-04-16T05:32:55.699580+0000","last_peered":"2024-04-16T05:32:55.699580+0000","last_clean":"2024-04-16T05:32:55.699580+0000","last_became_active":"2024-04-16T05:32:33.735572+0000","last_became_peered":"2024-04-16T05:32:33.735572+0000","last_unstale":"2024-04-16T05:32:55.699580+0000","last_undegraded":"2024-04-16T05:32:55.699580+0000","last_fullsized":"2024-04-16T05:32:55.699580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:46:56.512827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211156+0000","last_change":"2024-04-16T05:32:34.762172+0000","last_active":"2024-04-16T05:32:40.211156+0000","last_peered":"2024-04-16T05:32:40.211156+0000","last_clean":"2024-04-16T05:32:40.211156+0000","last_became_active":"2024-04-16T05:32:34.761914+0000","last_became_peered":"2024-04-16T05:32:34.761914+0000","last_unstale":"2024-04-16T05:32:40.211156+0000","last_undegraded":"2024-04-16T05:32:40.211156+0000","last_fullsized":"2024-04-16T05:32:40.211156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:47:52.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207653+0000","last_change":"2024-04-16T05:32:38.942967+0000","last_active":"2024-04-16T05:32:40.207653+0000","last_peered":"2024-04-16T05:32:40.207653+0000","last_clean":"2024-04-16T05:32:40.207653+0000","last_became_active":"2024-04-16T05:32:38.942523+0000","last_became_peered":"2024-04-16T05:32:38.942523+0000","last_unstale":"2024-04-16T05:32:40.207653+0000","last_undegraded":"2024-04-16T05:32:40.207653+0000","last_fullsized":"2024-04-16T05:32:40.207653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:39:43.956549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210517+0000","last_change":"2024-04-16T05:32:33.743944+0000","last_active":"2024-04-16T05:32:40.210517+0000","last_peered":"2024-04-16T05:32:40.210517+0000","last_clean":"2024-04-16T05:32:40.210517+0000","last_became_active":"2024-04-16T05:32:33.743559+0000","last_became_peered":"2024-04-16T05:32:33.743559+0000","last_unstale":"2024-04-16T05:32:40.210517+0000","last_undegraded":"2024-04-16T05:32:40.210517+0000","last_fullsized":"2024-04-16T05:32:40.210517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:08:48.889356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211257+0000","last_change":"2024-04-16T05:32:34.756741+0000","last_active":"2024-04-16T05:32:40.211257+0000","last_peered":"2024-04-16T05:32:40.211257+0000","last_clean":"2024-04-16T05:32:40.211257+0000","last_became_active":"2024-04-16T05:32:34.756426+0000","last_became_peered":"2024-04-16T05:32:34.756426+0000","last_unstale":"2024-04-16T05:32:40.211257+0000","last_undegraded":"2024-04-16T05:32:40.211257+0000","last_fullsized":"2024-04-16T05:32:40.211257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:09:30.493857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211192+0000","last_change":"2024-04-16T05:32:38.934818+0000","last_active":"2024-04-16T05:32:40.211192+0000","last_peered":"2024-04-16T05:32:40.211192+0000","last_clean":"2024-04-16T05:32:40.211192+0000","last_became_active":"2024-04-16T05:32:38.934335+0000","last_became_peered":"2024-04-16T05:32:38.934335+0000","last_unstale":"2024-04-16T05:32:40.211192+0000","last_undegraded":"2024-04-16T05:32:40.211192+0000","last_fullsized":"2024-04-16T05:32:40.211192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:02:41.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211225+0000","last_change":"2024-04-16T05:32:33.731951+0000","last_active":"2024-04-16T05:32:40.211225+0000","last_peered":"2024-04-16T05:32:40.211225+0000","last_clean":"2024-04-16T05:32:40.211225+0000","last_became_active":"2024-04-16T05:32:33.731649+0000","last_became_peered":"2024-04-16T05:32:33.731649+0000","last_unstale":"2024-04-16T05:32:40.211225+0000","last_undegraded":"2024-04-16T05:32:40.211225+0000","last_fullsized":"2024-04-16T05:32:40.211225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:41.258012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210772+0000","last_change":"2024-04-16T05:32:34.751884+0000","last_active":"2024-04-16T05:32:40.210772+0000","last_peered":"2024-04-16T05:32:40.210772+0000","last_clean":"2024-04-16T05:32:40.210772+0000","last_became_active":"2024-04-16T05:32:34.751559+0000","last_became_peered":"2024-04-16T05:32:34.751559+0000","last_unstale":"2024-04-16T05:32:40.210772+0000","last_undegraded":"2024-04-16T05:32:40.210772+0000","last_fullsized":"2024-04-16T05:32:40.210772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:32:59.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211741+0000","last_change":"2024-04-16T05:32:39.109330+0000","last_active":"2024-04-16T05:32:40.211741+0000","last_peered":"2024-04-16T05:32:40.211741+0000","last_clean":"2024-04-16T05:32:40.211741+0000","last_became_active":"2024-04-16T05:32:39.108718+0000","last_became_peered":"2024-04-16T05:32:39.108718+0000","last_unstale":"2024-04-16T05:32:40.211741+0000","last_undegraded":"2024-04-16T05:32:40.211741+0000","last_fullsized":"2024-04-16T05:32:40.211741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:36:55.418034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.908777+0000","last_change":"2024-04-16T05:32:33.741603+0000","last_active":"2024-04-16T05:32:39.908777+0000","last_peered":"2024-04-16T05:32:39.908777+0000","last_clean":"2024-04-16T05:32:39.908777+0000","last_became_active":"2024-04-16T05:32:33.741184+0000","last_became_peered":"2024-04-16T05:32:33.741184+0000","last_unstale":"2024-04-16T05:32:39.908777+0000","last_undegraded":"2024-04-16T05:32:39.908777+0000","last_fullsized":"2024-04-16T05:32:39.908777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:08:38.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909748+0000","last_change":"2024-04-16T05:32:34.747768+0000","last_active":"2024-04-16T05:32:39.909748+0000","last_peered":"2024-04-16T05:32:39.909748+0000","last_clean":"2024-04-16T05:32:39.909748+0000","last_became_active":"2024-04-16T05:32:34.747440+0000","last_became_peered":"2024-04-16T05:32:34.747440+0000","last_unstale":"2024-04-16T05:32:39.909748+0000","last_undegraded":"2024-04-16T05:32:39.909748+0000","last_fullsized":"2024-04-16T05:32:39.909748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:14:31.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207501+0000","last_change":"2024-04-16T05:32:38.938673+0000","last_active":"2024-04-16T05:32:40.207501+0000","last_peered":"2024-04-16T05:32:40.207501+0000","last_clean":"2024-04-16T05:32:40.207501+0000","last_became_active":"2024-04-16T05:32:38.938252+0000","last_became_peered":"2024-04-16T05:32:38.938252+0000","last_unstale":"2024-04-16T05:32:40.207501+0000","last_undegraded":"2024-04-16T05:32:40.207501+0000","last_fullsized":"2024-04-16T05:32:40.207501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:06.620693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211618+0000","last_change":"2024-04-16T05:32:33.750812+0000","last_active":"2024-04-16T05:32:40.211618+0000","last_peered":"2024-04-16T05:32:40.211618+0000","last_clean":"2024-04-16T05:32:40.211618+0000","last_became_active":"2024-04-16T05:32:33.750514+0000","last_became_peered":"2024-04-16T05:32:33.750514+0000","last_unstale":"2024-04-16T05:32:40.211618+0000","last_undegraded":"2024-04-16T05:32:40.211618+0000","last_fullsized":"2024-04-16T05:32:40.211618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:53:43.075914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909523+0000","last_change":"2024-04-16T05:32:34.754987+0000","last_active":"2024-04-16T05:32:39.909523+0000","last_peered":"2024-04-16T05:32:39.909523+0000","last_clean":"2024-04-16T05:32:39.909523+0000","last_became_active":"2024-04-16T05:32:34.754698+0000","last_became_peered":"2024-04-16T05:32:34.754698+0000","last_unstale":"2024-04-16T05:32:39.909523+0000","last_undegraded":"2024-04-16T05:32:39.909523+0000","last_fullsized":"2024-04-16T05:32:39.909523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:16:00.066573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207543+0000","last_change":"2024-04-16T05:32:39.109394+0000","last_active":"2024-04-16T05:32:40.207543+0000","last_peered":"2024-04-16T05:32:40.207543+0000","last_clean":"2024-04-16T05:32:40.207543+0000","last_became_active":"2024-04-16T05:32:39.108924+0000","last_became_peered":"2024-04-16T05:32:39.108924+0000","last_unstale":"2024-04-16T05:32:40.207543+0000","last_undegraded":"2024-04-16T05:32:40.207543+0000","last_fullsized":"2024-04-16T05:32:40.207543+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:50:37.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210655+0000","last_change":"2024-04-16T05:32:34.758778+0000","last_active":"2024-04-16T05:32:40.210655+0000","last_peered":"2024-04-16T05:32:40.210655+0000","last_clean":"2024-04-16T05:32:40.210655+0000","last_became_active":"2024-04-16T05:32:34.757426+0000","last_became_peered":"2024-04-16T05:32:34.757426+0000","last_unstale":"2024-04-16T05:32:40.210655+0000","last_undegraded":"2024-04-16T05:32:40.210655+0000","last_fullsized":"2024-04-16T05:32:40.210655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:25.371669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207063+0000","last_change":"2024-04-16T05:32:33.748521+0000","last_active":"2024-04-16T05:32:40.207063+0000","last_peered":"2024-04-16T05:32:40.207063+0000","last_clean":"2024-04-16T05:32:40.207063+0000","last_became_active":"2024-04-16T05:32:33.748222+0000","last_became_peered":"2024-04-16T05:32:33.748222+0000","last_unstale":"2024-04-16T05:32:40.207063+0000","last_undegraded":"2024-04-16T05:32:40.207063+0000","last_fullsized":"2024-04-16T05:32:40.207063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:04.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:41.795311+0000","last_change":"2024-04-16T05:32:38.935995+0000","last_active":"2024-04-16T05:32:41.795311+0000","last_peered":"2024-04-16T05:32:41.795311+0000","last_clean":"2024-04-16T05:32:41.795311+0000","last_became_active":"2024-04-16T05:32:38.935511+0000","last_became_peered":"2024-04-16T05:32:38.935511+0000","last_unstale":"2024-04-16T05:32:41.795311+0000","last_undegraded":"2024-04-16T05:32:41.795311+0000","last_fullsized":"2024-04-16T05:32:41.795311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:17:08.144537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210195+0000","last_change":"2024-04-16T05:32:34.760249+0000","last_active":"2024-04-16T05:32:40.210195+0000","last_peered":"2024-04-16T05:32:40.210195+0000","last_clean":"2024-04-16T05:32:40.210195+0000","last_became_active":"2024-04-16T05:32:34.759735+0000","last_became_peered":"2024-04-16T05:32:34.759735+0000","last_unstale":"2024-04-16T05:32:40.210195+0000","last_undegraded":"2024-04-16T05:32:40.210195+0000","last_fullsized":"2024-04-16T05:32:40.210195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:13.349558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:55.695267+0000","last_change":"2024-04-16T05:32:33.731170+0000","last_active":"2024-04-16T05:32:55.695267+0000","last_peered":"2024-04-16T05:32:55.695267+0000","last_clean":"2024-04-16T05:32:55.695267+0000","last_became_active":"2024-04-16T05:32:33.730821+0000","last_became_peered":"2024-04-16T05:32:33.730821+0000","last_unstale":"2024-04-16T05:32:55.695267+0000","last_undegraded":"2024-04-16T05:32:55.695267+0000","last_fullsized":"2024-04-16T05:32:55.695267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:38:15.707590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210940+0000","last_change":"2024-04-16T05:31:24.926319+0000","last_active":"2024-04-16T05:32:40.210940+0000","last_peered":"2024-04-16T05:32:40.210940+0000","last_clean":"2024-04-16T05:32:40.210940+0000","last_became_active":"2024-04-16T05:31:24.925936+0000","last_became_peered":"2024-04-16T05:31:24.925936+0000","last_unstale":"2024-04-16T05:32:40.210940+0000","last_undegraded":"2024-04-16T05:32:40.210940+0000","last_fullsized":"2024-04-16T05:32:40.210940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_clean_scrub_stamp":"2024-04-16T05:27:57.876673+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:28:19.212127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211141+0000","last_change":"2024-04-16T05:32:39.110241+0000","last_active":"2024-04-16T05:32:40.211141+0000","last_peered":"2024-04-16T05:32:40.211141+0000","last_clean":"2024-04-16T05:32:40.211141+0000","last_became_active":"2024-04-16T05:32:39.109792+0000","last_became_peered":"2024-04-16T05:32:39.109792+0000","last_unstale":"2024-04-16T05:32:40.211141+0000","last_undegraded":"2024-04-16T05:32:40.211141+0000","last_fullsized":"2024-04-16T05:32:40.211141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:15:05.925370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207057+0000","last_change":"2024-04-16T05:32:34.751693+0000","last_active":"2024-04-16T05:32:40.207057+0000","last_peered":"2024-04-16T05:32:40.207057+0000","last_clean":"2024-04-16T05:32:40.207057+0000","last_became_active":"2024-04-16T05:32:34.751210+0000","last_became_peered":"2024-04-16T05:32:34.751210+0000","last_unstale":"2024-04-16T05:32:40.207057+0000","last_undegraded":"2024-04-16T05:32:40.207057+0000","last_fullsized":"2024-04-16T05:32:40.207057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:33:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:54.689833+0000","last_change":"2024-04-16T05:32:33.738310+0000","last_active":"2024-04-16T05:32:54.689833+0000","last_peered":"2024-04-16T05:32:54.689833+0000","last_clean":"2024-04-16T05:32:54.689833+0000","last_became_active":"2024-04-16T05:32:33.738012+0000","last_became_peered":"2024-04-16T05:32:33.738012+0000","last_unstale":"2024-04-16T05:32:54.689833+0000","last_undegraded":"2024-04-16T05:32:54.689833+0000","last_fullsized":"2024-04-16T05:32:54.689833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:22:18.311458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210947+0000","last_change":"2024-04-16T05:32:38.933465+0000","last_active":"2024-04-16T05:32:40.210947+0000","last_peered":"2024-04-16T05:32:40.210947+0000","last_clean":"2024-04-16T05:32:40.210947+0000","last_became_active":"2024-04-16T05:32:38.932391+0000","last_became_peered":"2024-04-16T05:32:38.932391+0000","last_unstale":"2024-04-16T05:32:40.210947+0000","last_undegraded":"2024-04-16T05:32:40.210947+0000","last_fullsized":"2024-04-16T05:32:40.210947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:39.494556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211010+0000","last_change":"2024-04-16T05:32:33.743241+0000","last_active":"2024-04-16T05:32:40.211010+0000","last_peered":"2024-04-16T05:32:40.211010+0000","last_clean":"2024-04-16T05:32:40.211010+0000","last_became_active":"2024-04-16T05:32:33.742887+0000","last_became_peered":"2024-04-16T05:32:33.742887+0000","last_unstale":"2024-04-16T05:32:40.211010+0000","last_undegraded":"2024-04-16T05:32:40.211010+0000","last_fullsized":"2024-04-16T05:32:40.211010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:24:38.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207636+0000","last_change":"2024-04-16T05:32:34.746228+0000","last_active":"2024-04-16T05:32:40.207636+0000","last_peered":"2024-04-16T05:32:40.207636+0000","last_clean":"2024-04-16T05:32:40.207636+0000","last_became_active":"2024-04-16T05:32:34.745883+0000","last_became_peered":"2024-04-16T05:32:34.745883+0000","last_unstale":"2024-04-16T05:32:40.207636+0000","last_undegraded":"2024-04-16T05:32:40.207636+0000","last_fullsized":"2024-04-16T05:32:40.207636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:33.656358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210193+0000","last_change":"2024-04-16T05:32:39.110156+0000","last_active":"2024-04-16T05:32:40.210193+0000","last_peered":"2024-04-16T05:32:40.210193+0000","last_clean":"2024-04-16T05:32:40.210193+0000","last_became_active":"2024-04-16T05:32:39.108958+0000","last_became_peered":"2024-04-16T05:32:39.108958+0000","last_unstale":"2024-04-16T05:32:40.210193+0000","last_undegraded":"2024-04-16T05:32:40.210193+0000","last_fullsized":"2024-04-16T05:32:40.210193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:27:58.658187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909018+0000","last_change":"2024-04-16T05:32:33.738179+0000","last_active":"2024-04-16T05:32:39.909018+0000","last_peered":"2024-04-16T05:32:39.909018+0000","last_clean":"2024-04-16T05:32:39.909018+0000","last_became_active":"2024-04-16T05:32:33.737841+0000","last_became_peered":"2024-04-16T05:32:33.737841+0000","last_unstale":"2024-04-16T05:32:39.909018+0000","last_undegraded":"2024-04-16T05:32:39.909018+0000","last_fullsized":"2024-04-16T05:32:39.909018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:13:11.210559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210651+0000","last_change":"2024-04-16T05:32:34.758142+0000","last_active":"2024-04-16T05:32:40.210651+0000","last_peered":"2024-04-16T05:32:40.210651+0000","last_clean":"2024-04-16T05:32:40.210651+0000","last_became_active":"2024-04-16T05:32:34.757825+0000","last_became_peered":"2024-04-16T05:32:34.757825+0000","last_unstale":"2024-04-16T05:32:40.210651+0000","last_undegraded":"2024-04-16T05:32:40.210651+0000","last_fullsized":"2024-04-16T05:32:40.210651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:37.575764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909464+0000","last_change":"2024-04-16T05:32:38.933327+0000","last_active":"2024-04-16T05:32:39.909464+0000","last_peered":"2024-04-16T05:32:39.909464+0000","last_clean":"2024-04-16T05:32:39.909464+0000","last_became_active":"2024-04-16T05:32:38.932808+0000","last_became_peered":"2024-04-16T05:32:38.932808+0000","last_unstale":"2024-04-16T05:32:39.909464+0000","last_undegraded":"2024-04-16T05:32:39.909464+0000","last_fullsized":"2024-04-16T05:32:39.909464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:03.217103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:53.682183+0000","last_change":"2024-04-16T05:32:33.742405+0000","last_active":"2024-04-16T05:32:53.682183+0000","last_peered":"2024-04-16T05:32:53.682183+0000","last_clean":"2024-04-16T05:32:53.682183+0000","last_became_active":"2024-04-16T05:32:33.742051+0000","last_became_peered":"2024-04-16T05:32:33.742051+0000","last_unstale":"2024-04-16T05:32:53.682183+0000","last_undegraded":"2024-04-16T05:32:53.682183+0000","last_fullsized":"2024-04-16T05:32:53.682183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:35:13.914483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210353+0000","last_change":"2024-04-16T05:32:34.763141+0000","last_active":"2024-04-16T05:32:40.210353+0000","last_peered":"2024-04-16T05:32:40.210353+0000","last_clean":"2024-04-16T05:32:40.210353+0000","last_became_active":"2024-04-16T05:32:34.762785+0000","last_became_peered":"2024-04-16T05:32:34.762785+0000","last_unstale":"2024-04-16T05:32:40.210353+0000","last_undegraded":"2024-04-16T05:32:40.210353+0000","last_fullsized":"2024-04-16T05:32:40.210353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:07:19.315100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211198+0000","last_change":"2024-04-16T05:32:39.110198+0000","last_active":"2024-04-16T05:32:40.211198+0000","last_peered":"2024-04-16T05:32:40.211198+0000","last_clean":"2024-04-16T05:32:40.211198+0000","last_became_active":"2024-04-16T05:32:39.109470+0000","last_became_peered":"2024-04-16T05:32:39.109470+0000","last_unstale":"2024-04-16T05:32:40.211198+0000","last_undegraded":"2024-04-16T05:32:40.211198+0000","last_fullsized":"2024-04-16T05:32:40.211198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:39:34.674270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:53.683311+0000","last_change":"2024-04-16T05:32:33.743031+0000","last_active":"2024-04-16T05:32:53.683311+0000","last_peered":"2024-04-16T05:32:53.683311+0000","last_clean":"2024-04-16T05:32:53.683311+0000","last_became_active":"2024-04-16T05:32:33.742771+0000","last_became_peered":"2024-04-16T05:32:33.742771+0000","last_unstale":"2024-04-16T05:32:53.683311+0000","last_undegraded":"2024-04-16T05:32:53.683311+0000","last_fullsized":"2024-04-16T05:32:53.683311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:06.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210672+0000","last_change":"2024-04-16T05:32:34.749428+0000","last_active":"2024-04-16T05:32:40.210672+0000","last_peered":"2024-04-16T05:32:40.210672+0000","last_clean":"2024-04-16T05:32:40.210672+0000","last_became_active":"2024-04-16T05:32:34.749087+0000","last_became_peered":"2024-04-16T05:32:34.749087+0000","last_unstale":"2024-04-16T05:32:40.210672+0000","last_undegraded":"2024-04-16T05:32:40.210672+0000","last_fullsized":"2024-04-16T05:32:40.210672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:59:25.124558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909084+0000","last_change":"2024-04-16T05:32:38.942407+0000","last_active":"2024-04-16T05:32:39.909084+0000","last_peered":"2024-04-16T05:32:39.909084+0000","last_clean":"2024-04-16T05:32:39.909084+0000","last_became_active":"2024-04-16T05:32:38.941731+0000","last_became_peered":"2024-04-16T05:32:38.941731+0000","last_unstale":"2024-04-16T05:32:39.909084+0000","last_undegraded":"2024-04-16T05:32:39.909084+0000","last_fullsized":"2024-04-16T05:32:39.909084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:42:27.700203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210971+0000","last_change":"2024-04-16T05:32:33.741487+0000","last_active":"2024-04-16T05:32:40.210971+0000","last_peered":"2024-04-16T05:32:40.210971+0000","last_clean":"2024-04-16T05:32:40.210971+0000","last_became_active":"2024-04-16T05:32:33.741140+0000","last_became_peered":"2024-04-16T05:32:33.741140+0000","last_unstale":"2024-04-16T05:32:40.210971+0000","last_undegraded":"2024-04-16T05:32:40.210971+0000","last_fullsized":"2024-04-16T05:32:40.210971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:21:36.086164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210303+0000","last_change":"2024-04-16T05:32:34.756608+0000","last_active":"2024-04-16T05:32:40.210303+0000","last_peered":"2024-04-16T05:32:40.210303+0000","last_clean":"2024-04-16T05:32:40.210303+0000","last_became_active":"2024-04-16T05:32:34.756296+0000","last_became_peered":"2024-04-16T05:32:34.756296+0000","last_unstale":"2024-04-16T05:32:40.210303+0000","last_undegraded":"2024-04-16T05:32:40.210303+0000","last_fullsized":"2024-04-16T05:32:40.210303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:28.397333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211158+0000","last_change":"2024-04-16T05:32:38.929876+0000","last_active":"2024-04-16T05:32:40.211158+0000","last_peered":"2024-04-16T05:32:40.211158+0000","last_clean":"2024-04-16T05:32:40.211158+0000","last_became_active":"2024-04-16T05:32:38.928626+0000","last_became_peered":"2024-04-16T05:32:38.928626+0000","last_unstale":"2024-04-16T05:32:40.211158+0000","last_undegraded":"2024-04-16T05:32:40.211158+0000","last_fullsized":"2024-04-16T05:32:40.211158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:08:09.697413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909341+0000","last_change":"2024-04-16T05:32:33.736047+0000","last_active":"2024-04-16T05:32:39.909341+0000","last_peered":"2024-04-16T05:32:39.909341+0000","last_clean":"2024-04-16T05:32:39.909341+0000","last_became_active":"2024-04-16T05:32:33.735683+0000","last_became_peered":"2024-04-16T05:32:33.735683+0000","last_unstale":"2024-04-16T05:32:39.909341+0000","last_undegraded":"2024-04-16T05:32:39.909341+0000","last_fullsized":"2024-04-16T05:32:39.909341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:58:32.969924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210666+0000","last_change":"2024-04-16T05:32:34.763558+0000","last_active":"2024-04-16T05:32:40.210666+0000","last_peered":"2024-04-16T05:32:40.210666+0000","last_clean":"2024-04-16T05:32:40.210666+0000","last_became_active":"2024-04-16T05:32:34.763190+0000","last_became_peered":"2024-04-16T05:32:34.763190+0000","last_unstale":"2024-04-16T05:32:40.210666+0000","last_undegraded":"2024-04-16T05:32:40.210666+0000","last_fullsized":"2024-04-16T05:32:40.210666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:38:31.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210956+0000","last_change":"2024-04-16T05:32:39.109426+0000","last_active":"2024-04-16T05:32:40.210956+0000","last_peered":"2024-04-16T05:32:40.210956+0000","last_clean":"2024-04-16T05:32:40.210956+0000","last_became_active":"2024-04-16T05:32:39.108970+0000","last_became_peered":"2024-04-16T05:32:39.108970+0000","last_unstale":"2024-04-16T05:32:40.210956+0000","last_undegraded":"2024-04-16T05:32:40.210956+0000","last_fullsized":"2024-04-16T05:32:40.210956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:54.824654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210912+0000","last_change":"2024-04-16T05:32:33.734983+0000","last_active":"2024-04-16T05:32:40.210912+0000","last_peered":"2024-04-16T05:32:40.210912+0000","last_clean":"2024-04-16T05:32:40.210912+0000","last_became_active":"2024-04-16T05:32:33.734609+0000","last_became_peered":"2024-04-16T05:32:33.734609+0000","last_unstale":"2024-04-16T05:32:40.210912+0000","last_undegraded":"2024-04-16T05:32:40.210912+0000","last_fullsized":"2024-04-16T05:32:40.210912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:35:59.436499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211678+0000","last_change":"2024-04-16T05:32:34.759811+0000","last_active":"2024-04-16T05:32:40.211678+0000","last_peered":"2024-04-16T05:32:40.211678+0000","last_clean":"2024-04-16T05:32:40.211678+0000","last_became_active":"2024-04-16T05:32:34.759499+0000","last_became_peered":"2024-04-16T05:32:34.759499+0000","last_unstale":"2024-04-16T05:32:40.211678+0000","last_undegraded":"2024-04-16T05:32:40.211678+0000","last_fullsized":"2024-04-16T05:32:40.211678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:46:28.554191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210570+0000","last_change":"2024-04-16T05:32:38.929021+0000","last_active":"2024-04-16T05:32:40.210570+0000","last_peered":"2024-04-16T05:32:40.210570+0000","last_clean":"2024-04-16T05:32:40.210570+0000","last_became_active":"2024-04-16T05:32:38.928574+0000","last_became_peered":"2024-04-16T05:32:38.928574+0000","last_unstale":"2024-04-16T05:32:40.210570+0000","last_undegraded":"2024-04-16T05:32:40.210570+0000","last_fullsized":"2024-04-16T05:32:40.210570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:11:35.693364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909966+0000","last_change":"2024-04-16T05:32:33.747352+0000","last_active":"2024-04-16T05:32:39.909966+0000","last_peered":"2024-04-16T05:32:39.909966+0000","last_clean":"2024-04-16T05:32:39.909966+0000","last_became_active":"2024-04-16T05:32:33.746989+0000","last_became_peered":"2024-04-16T05:32:33.746989+0000","last_unstale":"2024-04-16T05:32:39.909966+0000","last_undegraded":"2024-04-16T05:32:39.909966+0000","last_fullsized":"2024-04-16T05:32:39.909966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:38.422274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.212219+0000","last_change":"2024-04-16T05:32:34.760636+0000","last_active":"2024-04-16T05:32:40.212219+0000","last_peered":"2024-04-16T05:32:40.212219+0000","last_clean":"2024-04-16T05:32:40.212219+0000","last_became_active":"2024-04-16T05:32:34.760335+0000","last_became_peered":"2024-04-16T05:32:34.760335+0000","last_unstale":"2024-04-16T05:32:40.212219+0000","last_undegraded":"2024-04-16T05:32:40.212219+0000","last_fullsized":"2024-04-16T05:32:40.212219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:04:34.322799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209682+0000","last_change":"2024-04-16T05:32:38.928924+0000","last_active":"2024-04-16T05:32:40.209682+0000","last_peered":"2024-04-16T05:32:40.209682+0000","last_clean":"2024-04-16T05:32:40.209682+0000","last_became_active":"2024-04-16T05:32:38.928172+0000","last_became_peered":"2024-04-16T05:32:38.928172+0000","last_unstale":"2024-04-16T05:32:40.209682+0000","last_undegraded":"2024-04-16T05:32:40.209682+0000","last_fullsized":"2024-04-16T05:32:40.209682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:28:03.810439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211549+0000","last_change":"2024-04-16T05:32:33.746346+0000","last_active":"2024-04-16T05:32:40.211549+0000","last_peered":"2024-04-16T05:32:40.211549+0000","last_clean":"2024-04-16T05:32:40.211549+0000","last_became_active":"2024-04-16T05:32:33.746030+0000","last_became_peered":"2024-04-16T05:32:33.746030+0000","last_unstale":"2024-04-16T05:32:40.211549+0000","last_undegraded":"2024-04-16T05:32:40.211549+0000","last_fullsized":"2024-04-16T05:32:40.211549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:54:35.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211933+0000","last_change":"2024-04-16T05:32:34.752283+0000","last_active":"2024-04-16T05:32:40.211933+0000","last_peered":"2024-04-16T05:32:40.211933+0000","last_clean":"2024-04-16T05:32:40.211933+0000","last_became_active":"2024-04-16T05:32:34.751904+0000","last_became_peered":"2024-04-16T05:32:34.751904+0000","last_unstale":"2024-04-16T05:32:40.211933+0000","last_undegraded":"2024-04-16T05:32:40.211933+0000","last_fullsized":"2024-04-16T05:32:40.211933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:15:52.688683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210459+0000","last_change":"2024-04-16T05:32:38.934194+0000","last_active":"2024-04-16T05:32:40.210459+0000","last_peered":"2024-04-16T05:32:40.210459+0000","last_clean":"2024-04-16T05:32:40.210459+0000","last_became_active":"2024-04-16T05:32:38.933636+0000","last_became_peered":"2024-04-16T05:32:38.933636+0000","last_unstale":"2024-04-16T05:32:40.210459+0000","last_undegraded":"2024-04-16T05:32:40.210459+0000","last_fullsized":"2024-04-16T05:32:40.210459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:14:40.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207566+0000","last_change":"2024-04-16T05:32:33.744267+0000","last_active":"2024-04-16T05:32:40.207566+0000","last_peered":"2024-04-16T05:32:40.207566+0000","last_clean":"2024-04-16T05:32:40.207566+0000","last_became_active":"2024-04-16T05:32:33.743931+0000","last_became_peered":"2024-04-16T05:32:33.743931+0000","last_unstale":"2024-04-16T05:32:40.207566+0000","last_undegraded":"2024-04-16T05:32:40.207566+0000","last_fullsized":"2024-04-16T05:32:40.207566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:30.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.212242+0000","last_change":"2024-04-16T05:32:34.756833+0000","last_active":"2024-04-16T05:32:40.212242+0000","last_peered":"2024-04-16T05:32:40.212242+0000","last_clean":"2024-04-16T05:32:40.212242+0000","last_became_active":"2024-04-16T05:32:34.756436+0000","last_became_peered":"2024-04-16T05:32:34.756436+0000","last_unstale":"2024-04-16T05:32:40.212242+0000","last_undegraded":"2024-04-16T05:32:40.212242+0000","last_fullsized":"2024-04-16T05:32:40.212242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:20.732946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210535+0000","last_change":"2024-04-16T05:32:34.752744+0000","last_active":"2024-04-16T05:32:40.210535+0000","last_peered":"2024-04-16T05:32:40.210535+0000","last_clean":"2024-04-16T05:32:40.210535+0000","last_became_active":"2024-04-16T05:32:34.752442+0000","last_became_peered":"2024-04-16T05:32:34.752442+0000","last_unstale":"2024-04-16T05:32:40.210535+0000","last_undegraded":"2024-04-16T05:32:40.210535+0000","last_fullsized":"2024-04-16T05:32:40.210535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:01:23.644692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909450+0000","last_change":"2024-04-16T05:32:34.755118+0000","last_active":"2024-04-16T05:32:39.909450+0000","last_peered":"2024-04-16T05:32:39.909450+0000","last_clean":"2024-04-16T05:32:39.909450+0000","last_became_active":"2024-04-16T05:32:34.754824+0000","last_became_peered":"2024-04-16T05:32:34.754824+0000","last_unstale":"2024-04-16T05:32:39.909450+0000","last_undegraded":"2024-04-16T05:32:39.909450+0000","last_fullsized":"2024-04-16T05:32:39.909450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:58:37.860305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.909398+0000","last_change":"2024-04-16T05:32:34.756144+0000","last_active":"2024-04-16T05:32:39.909398+0000","last_peered":"2024-04-16T05:32:39.909398+0000","last_clean":"2024-04-16T05:32:39.909398+0000","last_became_active":"2024-04-16T05:32:34.755841+0000","last_became_peered":"2024-04-16T05:32:34.755841+0000","last_unstale":"2024-04-16T05:32:39.909398+0000","last_undegraded":"2024-04-16T05:32:39.909398+0000","last_fullsized":"2024-04-16T05:32:39.909398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:23:00.482444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.209802+0000","last_change":"2024-04-16T05:32:34.766204+0000","last_active":"2024-04-16T05:32:40.209802+0000","last_peered":"2024-04-16T05:32:40.209802+0000","last_clean":"2024-04-16T05:32:40.209802+0000","last_became_active":"2024-04-16T05:32:34.765709+0000","last_became_peered":"2024-04-16T05:32:34.765709+0000","last_unstale":"2024-04-16T05:32:40.209802+0000","last_undegraded":"2024-04-16T05:32:40.209802+0000","last_fullsized":"2024-04-16T05:32:40.209802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:23:53.926477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207522+0000","last_change":"2024-04-16T05:32:34.758513+0000","last_active":"2024-04-16T05:32:40.207522+0000","last_peered":"2024-04-16T05:32:40.207522+0000","last_clean":"2024-04-16T05:32:40.207522+0000","last_became_active":"2024-04-16T05:32:34.758145+0000","last_became_peered":"2024-04-16T05:32:34.758145+0000","last_unstale":"2024-04-16T05:32:40.207522+0000","last_undegraded":"2024-04-16T05:32:40.207522+0000","last_fullsized":"2024-04-16T05:32:40.207522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:09:17.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211322+0000","last_change":"2024-04-16T05:32:34.750651+0000","last_active":"2024-04-16T05:32:40.211322+0000","last_peered":"2024-04-16T05:32:40.211322+0000","last_clean":"2024-04-16T05:32:40.211322+0000","last_became_active":"2024-04-16T05:32:34.750348+0000","last_became_peered":"2024-04-16T05:32:34.750348+0000","last_unstale":"2024-04-16T05:32:40.211322+0000","last_undegraded":"2024-04-16T05:32:40.211322+0000","last_fullsized":"2024-04-16T05:32:40.211322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:29.282449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:39.910063+0000","last_change":"2024-04-16T05:32:34.750079+0000","last_active":"2024-04-16T05:32:39.910063+0000","last_peered":"2024-04-16T05:32:39.910063+0000","last_clean":"2024-04-16T05:32:39.910063+0000","last_became_active":"2024-04-16T05:32:34.749775+0000","last_became_peered":"2024-04-16T05:32:34.749775+0000","last_unstale":"2024-04-16T05:32:39.910063+0000","last_undegraded":"2024-04-16T05:32:39.910063+0000","last_fullsized":"2024-04-16T05:32:39.910063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:02.297258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210746+0000","last_change":"2024-04-16T05:32:34.752528+0000","last_active":"2024-04-16T05:32:40.210746+0000","last_peered":"2024-04-16T05:32:40.210746+0000","last_clean":"2024-04-16T05:32:40.210746+0000","last_became_active":"2024-04-16T05:32:34.752150+0000","last_became_peered":"2024-04-16T05:32:34.752150+0000","last_unstale":"2024-04-16T05:32:40.210746+0000","last_undegraded":"2024-04-16T05:32:40.210746+0000","last_fullsized":"2024-04-16T05:32:40.210746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:41:16.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210541+0000","last_change":"2024-04-16T05:32:34.758486+0000","last_active":"2024-04-16T05:32:40.210541+0000","last_peered":"2024-04-16T05:32:40.210541+0000","last_clean":"2024-04-16T05:32:40.210541+0000","last_became_active":"2024-04-16T05:32:34.754840+0000","last_became_peered":"2024-04-16T05:32:34.754840+0000","last_unstale":"2024-04-16T05:32:40.210541+0000","last_undegraded":"2024-04-16T05:32:40.210541+0000","last_fullsized":"2024-04-16T05:32:40.210541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:35.644337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211046+0000","last_change":"2024-04-16T05:32:34.758933+0000","last_active":"2024-04-16T05:32:40.211046+0000","last_peered":"2024-04-16T05:32:40.211046+0000","last_clean":"2024-04-16T05:32:40.211046+0000","last_became_active":"2024-04-16T05:32:34.758587+0000","last_became_peered":"2024-04-16T05:32:34.758587+0000","last_unstale":"2024-04-16T05:32:40.211046+0000","last_undegraded":"2024-04-16T05:32:40.211046+0000","last_fullsized":"2024-04-16T05:32:40.211046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:19:20.674248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.210922+0000","last_change":"2024-04-16T05:32:34.750846+0000","last_active":"2024-04-16T05:32:40.210922+0000","last_peered":"2024-04-16T05:32:40.210922+0000","last_clean":"2024-04-16T05:32:40.210922+0000","last_became_active":"2024-04-16T05:32:34.750529+0000","last_became_peered":"2024-04-16T05:32:34.750529+0000","last_unstale":"2024-04-16T05:32:40.210922+0000","last_undegraded":"2024-04-16T05:32:40.210922+0000","last_fullsized":"2024-04-16T05:32:40.210922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:32.288459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.207279+0000","last_change":"2024-04-16T05:32:34.752566+0000","last_active":"2024-04-16T05:32:40.207279+0000","last_peered":"2024-04-16T05:32:40.207279+0000","last_clean":"2024-04-16T05:32:40.207279+0000","last_became_active":"2024-04-16T05:32:34.752236+0000","last_became_peered":"2024-04-16T05:32:34.752236+0000","last_unstale":"2024-04-16T05:32:40.207279+0000","last_undegraded":"2024-04-16T05:32:40.207279+0000","last_fullsized":"2024-04-16T05:32:40.207279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:43:36.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T05:32:40.211693+0000","last_change":"2024-04-16T05:32:34.750972+0000","last_active":"2024-04-16T05:32:40.211693+0000","last_peered":"2024-04-16T05:32:40.211693+0000","last_clean":"2024-04-16T05:32:40.211693+0000","last_became_active":"2024-04-16T05:32:34.750671+0000","last_became_peered":"2024-04-16T05:32:34.750671+0000","last_unstale":"2024-04-16T05:32:40.211693+0000","last_undegraded":"2024-04-16T05:32:40.211693+0000","last_fullsized":"2024-04-16T05:32:40.211693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:19:45.331816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283986,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25042,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":16,"num_read_kb":34,"num_write":85,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":39589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1073003,"data_compressed":50796,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":477999,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":477819,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1069375,"data_compressed":50796,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":481057,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":478693,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":481091,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":477319,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":477859,"data_compressed":14682,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1067765,"data_compressed":50796,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:32:58.928 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-16T05:32:58.929 INFO:tasks.cephadm:Setting up client nodes... 2024-04-16T05:32:58.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-16T05:32:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:58 smithi134 ceph-mon[36153]: pgmap v301: 129 pgs: 129 active+clean; 601 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-16T05:32:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:58 smithi134 ceph-mon[36153]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:58 smithi134 ceph-mon[36153]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:58 smithi134 ceph-mon[36153]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-16T05:32:59.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2937186221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:59.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3257914625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:59.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:58 smithi082 ceph-mon[36195]: pgmap v301: 129 pgs: 129 active+clean; 601 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-16T05:32:59.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:58 smithi082 ceph-mon[36195]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:59.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:58 smithi082 ceph-mon[36195]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:32:59.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:58 smithi082 ceph-mon[36195]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-16T05:32:59.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2937186221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:32:59.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3257914625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:32:59.212 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:32:59.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:32:59 smithi032 ceph-mon[32351]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:32:59 smithi134 ceph-mon[36153]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:32:59 smithi082 ceph-mon[36195]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:00.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:00 smithi032 ceph-mon[32351]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:00.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:00 smithi032 ceph-mon[32351]: pgmap v302: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-16T05:33:01.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 -- 172.21.15.32:0/2263614650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc103610 msgr2=0x7f99dc109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 --2- 172.21.15.32:0/2263614650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc103610 0x7f99dc109ec0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f99d400b0d0 tx=0x7f99d400b3e0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 -- 172.21.15.32:0/2263614650 shutdown_connections 2024-04-16T05:33:01.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 --2- 172.21.15.32:0/2263614650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc103610 0x7f99dc109ec0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 --2- 172.21.15.32:0/2263614650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc101a70 0x7f99dc101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 --2- 172.21.15.32:0/2263614650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99dc102c70 0x7f99dc1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 -- 172.21.15.32:0/2263614650 >> 172.21.15.32:0/2263614650 conn(0x7f99dc0fd010 msgr2=0x7f99dc0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 -- 172.21.15.32:0/2263614650 shutdown_connections 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.104+0000 7f99e2322700 1 -- 172.21.15.32:0/2263614650 wait complete. 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99e2322700 1 Processor -- start 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99e2322700 1 -- start start 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99e2322700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc101a70 0x7f99dc06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99e2322700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102c70 0x7f99dc19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99e2322700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99dc103610 0x7f99dc19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99e2322700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc10d640 con 0x7f99dc101a70 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99e2322700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc10d4c0 con 0x7f99dc102c70 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99e2322700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc10d7c0 con 0x7f99dc103610 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc101a70 0x7f99dc06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc101a70 0x7f99dc06bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52470/0 (socket says 172.21.15.32:52470) 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.105+0000 7f99dbfff700 1 -- 172.21.15.32:0/3542081260 learned_addr learned my addr 172.21.15.32:0/3542081260 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99e08bf700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99dc103610 0x7f99dc19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99db7fe700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102c70 0x7f99dc19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99dbfff700 1 -- 172.21.15.32:0/3542081260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99dc103610 msgr2=0x7f99dc19ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99dbfff700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99dc103610 0x7f99dc19ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99dbfff700 1 -- 172.21.15.32:0/3542081260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102c70 msgr2=0x7f99dc19d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99dbfff700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102c70 0x7f99dc19d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99dbfff700 1 -- 172.21.15.32:0/3542081260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc1a0870 con 0x7f99dc101a70 2024-04-16T05:33:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99db7fe700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102c70 0x7f99dc19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99dbfff700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc101a70 0x7f99dc06bd80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d980 tx=0x7f99cc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99d97fa700 1 -- 172.21.15.32:0/3542081260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc017030 con 0x7f99dc101a70 2024-04-16T05:33:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc1a0b60 con 0x7f99dc101a70 2024-04-16T05:33:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99d97fa700 1 -- 172.21.15.32:0/3542081260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc004500 con 0x7f99dc101a70 2024-04-16T05:33:01.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99d97fa700 1 -- 172.21.15.32:0/3542081260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc003b20 con 0x7f99dc101a70 2024-04-16T05:33:01.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.106+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc1a10a0 con 0x7f99dc101a70 2024-04-16T05:33:01.161 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:00 smithi082 ceph-mon[36195]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:01.161 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:00 smithi082 ceph-mon[36195]: pgmap v302: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-16T05:33:01.161 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:00 smithi134 ceph-mon[36153]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:33:01.161 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:00 smithi134 ceph-mon[36153]: pgmap v302: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-16T05:33:01.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.107+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc051030 con 0x7f99dc101a70 2024-04-16T05:33:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.108+0000 7f99d97fa700 1 -- 172.21.15.32:0/3542081260 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f99cc01a070 con 0x7f99dc101a70 2024-04-16T05:33:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.109+0000 7f99d97fa700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f99c40a3d80 0x7f99c40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.109+0000 7f99db7fe700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f99c40a3d80 0x7f99c40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.109+0000 7f99d97fa700 1 -- 172.21.15.32:0/3542081260 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f99cc0d0220 con 0x7f99dc101a70 2024-04-16T05:33:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.110+0000 7f99db7fe700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f99c40a3d80 0x7f99c40a6240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99d0005fd0 tx=0x7f99d0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.114+0000 7f99d97fa700 1 -- 172.21.15.32:0/3542081260 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99cc0232f0 con 0x7f99dc101a70 2024-04-16T05:33:01.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.406+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f99dc063700 con 0x7f99dc101a70 2024-04-16T05:33:01.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.409+0000 7f99d97fa700 1 -- 172.21.15.32:0/3542081260 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f99cc09c800 con 0x7f99dc101a70 2024-04-16T05:33:01.407 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-16T05:33:01.407 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCNDR5mAQlPGBAAOKgQvIFdaPowUO7BqK1Hew== 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f99c40a3d80 msgr2=0x7f99c40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f99c40a3d80 0x7f99c40a6240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99d0005fd0 tx=0x7f99d0005dc0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc101a70 msgr2=0x7f99dc06bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc101a70 0x7f99dc06bd80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d980 tx=0x7f99cc00dd40 comp rx=0 tx=0).stop 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 shutdown_connections 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f99c40a3d80 0x7f99c40a6240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99dc103610 0x7f99dc19ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99dc101a70 0x7f99dc06bd80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 --2- 172.21.15.32:0/3542081260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102c70 0x7f99dc19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 >> 172.21.15.32:0/3542081260 conn(0x7f99dc0fd010 msgr2=0x7f99dc105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 shutdown_connections 2024-04-16T05:33:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:01.411+0000 7f99e2322700 1 -- 172.21.15.32:0/3542081260 wait complete. 2024-04-16T05:33:01.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3542081260' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T05:33:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3542081260' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T05:33:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3542081260' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T05:33:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3542081260' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T05:33:02.142 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:33:02.142 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-16T05:33:02.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-16T05:33:02.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3542081260' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T05:33:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3542081260' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T05:33:02.188 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-16T05:33:02.188 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-16T05:33:02.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph mgr dump --format=json 2024-04-16T05:33:02.477 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:02.903 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:02 smithi032 ceph-mon[32351]: pgmap v303: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-16T05:33:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:02 smithi134 ceph-mon[36153]: pgmap v303: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-16T05:33:03.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:02 smithi082 ceph-mon[36195]: pgmap v303: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-16T05:33:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2386969770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c108b70 msgr2=0x7f8f2c108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2386969770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c108b70 0x7f8f2c108f50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009990 tx=0x7f8f28009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2386969770 shutdown_connections 2024-04-16T05:33:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2386969770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c06b1a0 0x7f8f2c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2386969770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c108b70 0x7f8f2c108f50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2386969770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f2c06a800 0x7f8f2c06ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2386969770 >> 172.21.15.32:0/2386969770 conn(0x7f8f2c0789a0 msgr2=0x7f8f2c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2386969770 shutdown_connections 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.255+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2386969770 wait complete. 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f33e8b700 1 Processor -- start 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f33e8b700 1 -- start start 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f33e8b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c06a800 0x7f8f2c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f33e8b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c06b1a0 0x7f8f2c199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f33e8b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f2c108b70 0x7f8f2c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f33e8b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c112e30 con 0x7f8f2c06a800 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f33e8b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c112cb0 con 0x7f8f2c108b70 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f33e8b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c112fb0 con 0x7f8f2c06b1a0 2024-04-16T05:33:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f31c27700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c06a800 0x7f8f2c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f31c27700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c06a800 0x7f8f2c199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52496/0 (socket says 172.21.15.32:52496) 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.256+0000 7f8f31c27700 1 -- 172.21.15.32:0/2379912402 learned_addr learned my addr 172.21.15.32:0/2379912402 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f31426700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c06b1a0 0x7f8f2c199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f32428700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f2c108b70 0x7f8f2c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f31c27700 1 -- 172.21.15.32:0/2379912402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c06b1a0 msgr2=0x7f8f2c199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f31c27700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c06b1a0 0x7f8f2c199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f31c27700 1 -- 172.21.15.32:0/2379912402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f2c108b70 msgr2=0x7f8f2c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f31c27700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f2c108b70 0x7f8f2c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f31c27700 1 -- 172.21.15.32:0/2379912402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c19e300 con 0x7f8f2c06a800 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f31426700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c06b1a0 0x7f8f2c199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f31c27700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c06a800 0x7f8f2c199340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f28000c00 tx=0x7f8f280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f22ffd700 1 -- 172.21.15.32:0/2379912402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28027070 con 0x7f8f2c06a800 2024-04-16T05:33:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f22ffd700 1 -- 172.21.15.32:0/2379912402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f28004050 con 0x7f8f2c06a800 2024-04-16T05:33:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f22ffd700 1 -- 172.21.15.32:0/2379912402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2800f460 con 0x7f8f2c06a800 2024-04-16T05:33:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c19e590 con 0x7f8f2c06a800 2024-04-16T05:33:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.257+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c1a5e70 con 0x7f8f2c06a800 2024-04-16T05:33:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.258+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2c0697e0 con 0x7f8f2c06a800 2024-04-16T05:33:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.259+0000 7f8f22ffd700 1 -- 172.21.15.32:0/2379912402 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8f2802a070 con 0x7f8f2c06a800 2024-04-16T05:33:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.260+0000 7f8f22ffd700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f180a3d50 0x7f8f180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.260+0000 7f8f31426700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f180a3d50 0x7f8f180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.260+0000 7f8f22ffd700 1 -- 172.21.15.32:0/2379912402 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f8f280d16c0 con 0x7f8f2c06a800 2024-04-16T05:33:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.260+0000 7f8f31426700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f180a3d50 0x7f8f180a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c079010 tx=0x7f8f1c00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:04.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.265+0000 7f8f22ffd700 1 -- 172.21.15.32:0/2379912402 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f2805eff0 con 0x7f8f2c06a800 2024-04-16T05:33:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.533+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8f2c02dfe0 con 0x7f8f2c06a800 2024-04-16T05:33:04.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.538+0000 7f8f22ffd700 1 -- 172.21.15.32:0/2379912402 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f8f28013070 con 0x7f8f2c06a800 2024-04-16T05:33:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f180a3d50 msgr2=0x7f8f180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f180a3d50 0x7f8f180a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c079010 tx=0x7f8f1c00b5f0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c06a800 msgr2=0x7f8f2c199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c06a800 0x7f8f2c199340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f28000c00 tx=0x7f8f280047b0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 shutdown_connections 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f180a3d50 0x7f8f180a6210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f2c108b70 0x7f8f2c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f2c06a800 0x7f8f2c199340 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 --2- 172.21.15.32:0/2379912402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c06b1a0 0x7f8f2c199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 >> 172.21.15.32:0/2379912402 conn(0x7f8f2c0789a0 msgr2=0x7f8f2c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 shutdown_connections 2024-04-16T05:33:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:04.541+0000 7f8f33e8b700 1 -- 172.21.15.32:0/2379912402 wait complete. 2024-04-16T05:33:04.916 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:04 smithi032 ceph-mon[32351]: pgmap v304: 129 pgs: 129 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-16T05:33:04.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:04 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2379912402' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T05:33:05.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:04 smithi134 ceph-mon[36153]: pgmap v304: 129 pgs: 129 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-16T05:33:05.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:04 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2379912402' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T05:33:05.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:04 smithi082 ceph-mon[36195]: pgmap v304: 129 pgs: 129 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-16T05:33:05.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:04 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2379912402' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T05:33:05.483 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2875882657},{"type":"v1","addr":"172.21.15.32:6801","nonce":2875882657}]},"active_addr":"172.21.15.32:6801/2875882657","active_change":"2024-04-16T05:25:32.817181+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.32:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1356969979}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":558122612}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1729232468}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2362160984}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3699058289}]}]} 2024-04-16T05:33:05.487 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-16T05:33:05.488 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-16T05:33:05.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd dump --format=json 2024-04-16T05:33:05.742 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:06.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:06 smithi032 ceph-mon[32351]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-16T05:33:07.128 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:06 smithi134 ceph-mon[36153]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-16T05:33:07.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:06 smithi082 ceph-mon[36195]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-16T05:33:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.622+0000 7fa8241cd700 1 -- 172.21.15.32:0/1966222417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c1013d0 msgr2=0x7fa81c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.622+0000 7fa8241cd700 1 --2- 172.21.15.32:0/1966222417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c1013d0 0x7fa81c1017b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa8180099c0 tx=0x7fa818009cd0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.622+0000 7fa8241cd700 1 -- 172.21.15.32:0/1966222417 shutdown_connections 2024-04-16T05:33:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.622+0000 7fa8241cd700 1 --2- 172.21.15.32:0/1966222417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81c1064b0 0x7fa81c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.622+0000 7fa8241cd700 1 --2- 172.21.15.32:0/1966222417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c1013d0 0x7fa81c1017b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.622+0000 7fa8241cd700 1 --2- 172.21.15.32:0/1966222417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa81c101d80 0x7fa81c105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.622+0000 7fa8241cd700 1 -- 172.21.15.32:0/1966222417 >> 172.21.15.32:0/1966222417 conn(0x7fa81c077ff0 msgr2=0x7fa81c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 -- 172.21.15.32:0/1966222417 shutdown_connections 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 -- 172.21.15.32:0/1966222417 wait complete. 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 Processor -- start 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 -- start start 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81c1013d0 0x7fa81c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c101d80 0x7fa81c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa81c1064b0 0x7fa81c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c111d30 con 0x7fa81c101d80 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c111bb0 con 0x7fa81c1064b0 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.623+0000 7fa8241cd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c111eb0 con 0x7fa81c1013d0 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c101d80 0x7fa81c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c101d80 0x7fa81c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52514/0 (socket says 172.21.15.32:52514) 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 -- 172.21.15.32:0/2943663306 learned_addr learned my addr 172.21.15.32:0/2943663306 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821f69700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81c1013d0 0x7fa81c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa82276a700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa81c1064b0 0x7fa81c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 -- 172.21.15.32:0/2943663306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81c1013d0 msgr2=0x7fa81c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81c1013d0 0x7fa81c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 -- 172.21.15.32:0/2943663306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa81c1064b0 msgr2=0x7fa81c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa81c1064b0 0x7fa81c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 -- 172.21.15.32:0/2943663306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c1a7790 con 0x7fa81c101d80 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821f69700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81c1013d0 0x7fa81c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa82276a700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa81c1064b0 0x7fa81c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa821768700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c101d80 0x7fa81c198460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa81000e970 tx=0x7fa81000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa80effd700 1 -- 172.21.15.32:0/2943663306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81000cbd0 con 0x7fa81c101d80 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa80effd700 1 -- 172.21.15.32:0/2943663306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81000cd70 con 0x7fa81c101d80 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c1a7a80 con 0x7fa81c101d80 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa80effd700 1 -- 172.21.15.32:0/2943663306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810010430 con 0x7fa81c101d80 2024-04-16T05:33:07.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.624+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c1a7fc0 con 0x7fa81c101d80 2024-04-16T05:33:07.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.625+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81c0697e0 con 0x7fa81c101d80 2024-04-16T05:33:07.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.627+0000 7fa80effd700 1 -- 172.21.15.32:0/2943663306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa810004750 con 0x7fa81c101d80 2024-04-16T05:33:07.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.627+0000 7fa80effd700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa8080a3cd0 0x7fa8080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:07.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.627+0000 7fa821f69700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa8080a3cd0 0x7fa8080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:07.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.627+0000 7fa80effd700 1 -- 172.21.15.32:0/2943663306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa8100d0d40 con 0x7fa81c101d80 2024-04-16T05:33:07.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.628+0000 7fa821f69700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa8080a3cd0 0x7fa8080a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa818000c00 tx=0x7fa818005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:07.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.631+0000 7fa80effd700 1 -- 172.21.15.32:0/2943663306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa810053ff0 con 0x7fa81c101d80 2024-04-16T05:33:07.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.837+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa81c02dfe0 con 0x7fa81c101d80 2024-04-16T05:33:07.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.839+0000 7fa80effd700 1 -- 172.21.15.32:0/2943663306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7fa81009d380 con 0x7fa81c101d80 2024-04-16T05:33:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:33:07.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa8080a3cd0 msgr2=0x7fa8080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa8080a3cd0 0x7fa8080a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa818000c00 tx=0x7fa818005fd0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c101d80 msgr2=0x7fa81c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c101d80 0x7fa81c198460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa81000e970 tx=0x7fa81000ed30 comp rx=0 tx=0).stop 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 shutdown_connections 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa8080a3cd0 0x7fa8080a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa81c1064b0 0x7fa81c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81c1013d0 0x7fa81c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 --2- 172.21.15.32:0/2943663306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa81c101d80 0x7fa81c198460 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 >> 172.21.15.32:0/2943663306 conn(0x7fa81c077ff0 msgr2=0x7fa81c104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 shutdown_connections 2024-04-16T05:33:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:07.841+0000 7fa8241cd700 1 -- 172.21.15.32:0/2943663306 wait complete. 2024-04-16T05:33:08.575 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-16T05:33:08.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd dump --format=json 2024-04-16T05:33:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:08.948 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:08 smithi032 ceph-mon[32351]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-04-16T05:33:08.948 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:08 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2943663306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:08 smithi134 ceph-mon[36153]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-04-16T05:33:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:08 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2943663306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:09.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:08 smithi082 ceph-mon[36195]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-04-16T05:33:09.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:08 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2943663306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:10.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.617+0000 7fd9215a2700 1 -- 172.21.15.32:0/3762480252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101320 msgr2=0x7fd91c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.617+0000 7fd9215a2700 1 --2- 172.21.15.32:0/3762480252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101320 0x7fd91c1017a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd90400aad0 tx=0x7fd90400ade0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.617+0000 7fd9215a2700 1 -- 172.21.15.32:0/3762480252 shutdown_connections 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.617+0000 7fd9215a2700 1 --2- 172.21.15.32:0/3762480252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c101ce0 0x7fd91c10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.617+0000 7fd9215a2700 1 --2- 172.21.15.32:0/3762480252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c107180 0x7fd91c107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.617+0000 7fd9215a2700 1 --2- 172.21.15.32:0/3762480252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101320 0x7fd91c1017a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.617+0000 7fd9215a2700 1 -- 172.21.15.32:0/3762480252 >> 172.21.15.32:0/3762480252 conn(0x7fd91c076ca0 msgr2=0x7fd91c0770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.617+0000 7fd9215a2700 1 -- 172.21.15.32:0/3762480252 shutdown_connections 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 -- 172.21.15.32:0/3762480252 wait complete. 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 Processor -- start 2024-04-16T05:33:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 -- start start 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c101320 0x7fd91c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101ce0 0x7fd91c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c107180 0x7fd91c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c110b90 con 0x7fd91c101ce0 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c110a10 con 0x7fd91c107180 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.618+0000 7fd9215a2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c110d10 con 0x7fd91c101320 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101ce0 0x7fd91c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101ce0 0x7fd91c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38076/0 (socket says 172.21.15.32:38076) 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 -- 172.21.15.32:0/2142428360 learned_addr learned my addr 172.21.15.32:0/2142428360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd91bfff700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c101320 0x7fd91c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd920da1700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c107180 0x7fd91c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 -- 172.21.15.32:0/2142428360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c101320 msgr2=0x7fd91c1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c101320 0x7fd91c1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 -- 172.21.15.32:0/2142428360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c107180 msgr2=0x7fd91c19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c107180 0x7fd91c19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 -- 172.21.15.32:0/2142428360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c19e200 con 0x7fd91c101ce0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd913fff700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101ce0 0x7fd91c199730 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd90400c690 tx=0x7fd90400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd919ffb700 1 -- 172.21.15.32:0/2142428360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904023020 con 0x7fd91c101ce0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c19e490 con 0x7fd91c101ce0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd919ffb700 1 -- 172.21.15.32:0/2142428360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd90400ed70 con 0x7fd91c101ce0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd919ffb700 1 -- 172.21.15.32:0/2142428360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904003c40 con 0x7fd91c101ce0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.619+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c1a5d70 con 0x7fd91c101ce0 2024-04-16T05:33:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.620+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91c102810 con 0x7fd91c101ce0 2024-04-16T05:33:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.622+0000 7fd919ffb700 1 -- 172.21.15.32:0/2142428360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd904026020 con 0x7fd91c101ce0 2024-04-16T05:33:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.622+0000 7fd919ffb700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd9080a3cd0 0x7fd9080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.622+0000 7fd919ffb700 1 -- 172.21.15.32:0/2142428360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd9040d2360 con 0x7fd91c101ce0 2024-04-16T05:33:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.622+0000 7fd91bfff700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd9080a3cd0 0x7fd9080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.623+0000 7fd91bfff700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd9080a3cd0 0x7fd9080a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009be0 tx=0x7fd90c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.626+0000 7fd919ffb700 1 -- 172.21.15.32:0/2142428360 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd90402d020 con 0x7fd91c101ce0 2024-04-16T05:33:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.833+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd91c0697e0 con 0x7fd91c101ce0 2024-04-16T05:33:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.835+0000 7fd919ffb700 1 -- 172.21.15.32:0/2142428360 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24092 (secure 0 0 0) 0x7fd90409ea30 con 0x7fd91c101ce0 2024-04-16T05:33:10.834 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:33:10.834 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:32:39.296582+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd9080a3cd0 msgr2=0x7fd9080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd9080a3cd0 0x7fd9080a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009be0 tx=0x7fd90c009260 comp rx=0 tx=0).stop 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101ce0 msgr2=0x7fd91c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101ce0 0x7fd91c199730 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd90400c690 tx=0x7fd90400cdd0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 shutdown_connections 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd9080a3cd0 0x7fd9080a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c107180 0x7fd91c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c101320 0x7fd91c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 --2- 172.21.15.32:0/2142428360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c101ce0 0x7fd91c199730 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 >> 172.21.15.32:0/2142428360 conn(0x7fd91c076ca0 msgr2=0x7fd91c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 shutdown_connections 2024-04-16T05:33:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:10.837+0000 7fd9215a2700 1 -- 172.21.15.32:0/2142428360 wait complete. 2024-04-16T05:33:10.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:10 smithi032 ceph-mon[32351]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-16T05:33:11.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:10 smithi134 ceph-mon[36153]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-16T05:33:11.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:10 smithi082 ceph-mon[36195]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-16T05:33:11.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-16T05:33:11.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-16T05:33:11.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-16T05:33:11.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-16T05:33:11.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-16T05:33:11.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-16T05:33:11.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-16T05:33:11.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-16T05:33:11.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-16T05:33:11.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-16T05:33:11.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-16T05:33:11.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-16T05:33:11.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:11 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2142428360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:11 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2142428360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:11 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2142428360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:33:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:12.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:12 smithi032 ceph-mon[32351]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-16T05:33:13.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:12 smithi134 ceph-mon[36153]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-16T05:33:13.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:12 smithi082 ceph-mon[36195]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-16T05:33:14.118 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.118 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.867+0000 7ff9de175700 1 -- 172.21.15.32:0/535968548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8102b40 msgr2=0x7ff9d8102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.867+0000 7ff9de175700 1 --2- 172.21.15.32:0/535968548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8102b40 0x7ff9d8102fc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009990 tx=0x7ff9cc009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 -- 172.21.15.32:0/535968548 shutdown_connections 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 --2- 172.21.15.32:0/535968548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9d8103500 0x7ff9d810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 --2- 172.21.15.32:0/535968548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9d8108b90 0x7ff9d8108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 --2- 172.21.15.32:0/535968548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8102b40 0x7ff9d8102fc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 -- 172.21.15.32:0/535968548 >> 172.21.15.32:0/535968548 conn(0x7ff9d8079c50 msgr2=0x7ff9d807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 -- 172.21.15.32:0/535968548 shutdown_connections 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 -- 172.21.15.32:0/535968548 wait complete. 2024-04-16T05:33:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 Processor -- start 2024-04-16T05:33:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.868+0000 7ff9de175700 1 -- start start 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9de175700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9d8102b40 0x7ff9d81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9de175700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9d8103500 0x7ff9d8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9de175700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8108b90 0x7ff9d819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9de175700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8112e00 con 0x7ff9d8108b90 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9de175700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8112c80 con 0x7ff9d8102b40 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9de175700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8112f80 con 0x7ff9d8103500 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8108b90 0x7ff9d819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8108b90 0x7ff9d819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38098/0 (socket says 172.21.15.32:38098) 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 -- 172.21.15.32:0/4003579970 learned_addr learned my addr 172.21.15.32:0/4003579970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d77fe700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9d8102b40 0x7ff9d81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d6ffd700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9d8103500 0x7ff9d8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 -- 172.21.15.32:0/4003579970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9d8103500 msgr2=0x7ff9d8199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9d8103500 0x7ff9d8199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 -- 172.21.15.32:0/4003579970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9d8102b40 msgr2=0x7ff9d81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9d8102b40 0x7ff9d81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 -- 172.21.15.32:0/4003579970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d819e290 con 0x7ff9d8108b90 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d7fff700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8108b90 0x7ff9d819db50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0007ee0 tx=0x7ff9d000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000dbd0 con 0x7ff9d8108b90 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d819e580 con 0x7ff9d8108b90 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.869+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d000dd70 con 0x7ff9d8108b90 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.870+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0005a60 con 0x7ff9d8108b90 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.870+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81a5e00 con 0x7ff9d8108b90 2024-04-16T05:33:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.870+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9d8063200 con 0x7ff9d8108b90 2024-04-16T05:33:14.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.872+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff9d001a020 con 0x7ff9d8108b90 2024-04-16T05:33:14.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.872+0000 7ff9d4ff9700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff9c00a3d50 0x7ff9c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.872+0000 7ff9d77fe700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff9c00a3d50 0x7ff9c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.872+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff9d00d0590 con 0x7ff9d8108b90 2024-04-16T05:33:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.873+0000 7ff9d4ff9700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] conn(0x7ff9c00aea10 0x7ff9c00b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.873+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 --> [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9d0012ec0 con 0x7ff9c00aea10 2024-04-16T05:33:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.873+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff9d00d09e0 con 0x7ff9d8108b90 2024-04-16T05:33:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.873+0000 7ff9d77fe700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff9c00a3d50 0x7ff9c00a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8005950 tx=0x7ff9c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.873+0000 7ff9d6ffd700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] conn(0x7ff9c00aea10 0x7ff9c00b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:14.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.873+0000 7ff9d6ffd700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] conn(0x7ff9c00aea10 0x7ff9c00b0e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:14.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.874+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 <== osd.2 v2:172.21.15.134:6800/2834744702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9d0012ec0 con 0x7ff9c00aea10 2024-04-16T05:33:14.880 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.906+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 --> [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9d8063600 con 0x7ff9c00aea10 2024-04-16T05:33:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.907+0000 7ff9d4ff9700 1 -- 172.21.15.32:0/4003579970 <== osd.2 v2:172.21.15.134:6800/2834744702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff9d8063600 con 0x7ff9c00aea10 2024-04-16T05:33:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 >> [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] conn(0x7ff9c00aea10 msgr2=0x7ff9c00b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9de175700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] conn(0x7ff9c00aea10 0x7ff9c00b0e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff9c00a3d50 msgr2=0x7ff9c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9de175700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff9c00a3d50 0x7ff9c00a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8005950 tx=0x7ff9c800b410 comp rx=0 tx=0).stop 2024-04-16T05:33:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8108b90 msgr2=0x7ff9d819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9de175700 1 --2- 172.21.15.32:0/4003579970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d8108b90 0x7ff9d819db50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0007ee0 tx=0x7ff9d000d3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9d7fff700 1 -- 172.21.15.32:0/4003579970 reap_dead start 2024-04-16T05:33:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 shutdown_connections 2024-04-16T05:33:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.909+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 >> 172.21.15.32:0/4003579970 conn(0x7ff9d8079c50 msgr2=0x7ff9d8101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.910+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 shutdown_connections 2024-04-16T05:33:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:14.911+0000 7ff9de175700 1 -- 172.21.15.32:0/4003579970 wait complete. 2024-04-16T05:33:14.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:14 smithi032 ceph-mon[32351]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-16T05:33:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:14 smithi134 ceph-mon[36153]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-16T05:33:15.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:14 smithi082 ceph-mon[36195]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-16T05:33:16.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:16 smithi134 ceph-mon[36153]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-16T05:33:16.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:16 smithi032 ceph-mon[32351]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-16T05:33:17.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:16 smithi082 ceph-mon[36195]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-16T05:33:18.912 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:18 smithi032 ceph-mon[32351]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T05:33:18.943 INFO:teuthology.orchestra.run.smithi032.stdout:77309411395 2024-04-16T05:33:18.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-16T05:33:19.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:18 smithi134 ceph-mon[36153]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T05:33:19.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:18 smithi082 ceph-mon[36195]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T05:33:20.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:20 smithi032 ceph-mon[32351]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T05:33:21.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:20 smithi134 ceph-mon[36153]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T05:33:21.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:20 smithi082 ceph-mon[36195]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T05:33:23.086 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:22 smithi032 ceph-mon[32351]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:33:23.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:22 smithi134 ceph-mon[36153]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:33:23.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:22 smithi082 ceph-mon[36195]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:33:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 -- 172.21.15.32:0/956486166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d4101a20 msgr2=0x7f76d4101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 --2- 172.21.15.32:0/956486166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d4101a20 0x7f76d4101ea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009990 tx=0x7f76c4009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 -- 172.21.15.32:0/956486166 shutdown_connections 2024-04-16T05:33:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 --2- 172.21.15.32:0/956486166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d41023e0 0x7f76d410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 --2- 172.21.15.32:0/956486166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d41079d0 0x7f76d4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 --2- 172.21.15.32:0/956486166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d4101a20 0x7f76d4101ea0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 -- 172.21.15.32:0/956486166 >> 172.21.15.32:0/956486166 conn(0x7f76d4077a60 msgr2=0x7f76d4077e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 -- 172.21.15.32:0/956486166 shutdown_connections 2024-04-16T05:33:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.926+0000 7f76dc103700 1 -- 172.21.15.32:0/956486166 wait complete. 2024-04-16T05:33:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76dc103700 1 Processor -- start 2024-04-16T05:33:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76dc103700 1 -- start start 2024-04-16T05:33:23.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76dc103700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d4101a20 0x7f76d419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:23.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76dc103700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d41023e0 0x7f76d419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76dc103700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d41079d0 0x7f76d419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76dc103700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111d60 con 0x7f76d41079d0 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76dc103700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111be0 con 0x7f76d41023e0 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76dc103700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111ee0 con 0x7f76d4101a20 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76da6a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d41079d0 0x7f76d419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76da6a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d41079d0 0x7f76d419fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33468/0 (socket says 172.21.15.32:33468) 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76da6a0700 1 -- 172.21.15.32:0/122508632 learned_addr learned my addr 172.21.15.32:0/122508632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76d9e9f700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d4101a20 0x7f76d419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76d969e700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d41023e0 0x7f76d419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76da6a0700 1 -- 172.21.15.32:0/122508632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d4101a20 msgr2=0x7f76d419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76da6a0700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d4101a20 0x7f76d419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76da6a0700 1 -- 172.21.15.32:0/122508632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d41023e0 msgr2=0x7f76d419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76da6a0700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d41023e0 0x7f76d419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:23.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.927+0000 7f76da6a0700 1 -- 172.21.15.32:0/122508632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d41a0480 con 0x7f76d41079d0 2024-04-16T05:33:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.928+0000 7f76da6a0700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d41079d0 0x7f76d419fd40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f76cc007d70 tx=0x7f76cc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.928+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc016020 con 0x7f76d41079d0 2024-04-16T05:33:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.928+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc00ac90 con 0x7f76d41079d0 2024-04-16T05:33:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.928+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41a0770 con 0x7f76d41079d0 2024-04-16T05:33:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.928+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0115c0 con 0x7f76d41079d0 2024-04-16T05:33:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.928+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a7ff0 con 0x7f76d41079d0 2024-04-16T05:33:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.929+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76d4101490 con 0x7f76d41079d0 2024-04-16T05:33:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.930+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f76cc019020 con 0x7f76d41079d0 2024-04-16T05:33:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.931+0000 7f76caffd700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76c00a3cd0 0x7f76c00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.931+0000 7f76d9e9f700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76c00a3cd0 0x7f76c00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.931+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f76cc0d0ba0 con 0x7f76d41079d0 2024-04-16T05:33:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.931+0000 7f76caffd700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] conn(0x7f76c00ae990 0x7f76c00b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.931+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 --> [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76cc018e70 con 0x7f76c00ae990 2024-04-16T05:33:23.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.931+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f76cc0d0ff0 con 0x7f76d41079d0 2024-04-16T05:33:23.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.932+0000 7f76d9e9f700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76c00a3cd0 0x7f76c00a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76d0005950 tx=0x7f76d000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:23.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.932+0000 7f76d969e700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] conn(0x7f76c00ae990 0x7f76c00b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:23.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.932+0000 7f76d969e700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] conn(0x7f76c00ae990 0x7f76c00b0dd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:23.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.933+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 <== osd.8 v2:172.21.15.134:6816/2280284325 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76cc018e70 con 0x7f76c00ae990 2024-04-16T05:33:23.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.949+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 --> [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f76d4103110 con 0x7f76c00ae990 2024-04-16T05:33:23.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76caffd700 1 -- 172.21.15.32:0/122508632 <== osd.8 v2:172.21.15.134:6816/2280284325 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f76d4103110 con 0x7f76c00ae990 2024-04-16T05:33:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 >> [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] conn(0x7f76c00ae990 msgr2=0x7f76c00b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] conn(0x7f76c00ae990 0x7f76c00b0dd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76c00a3cd0 msgr2=0x7f76c00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76c00a3cd0 0x7f76c00a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76d0005950 tx=0x7f76d000a300 comp rx=0 tx=0).stop 2024-04-16T05:33:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d41079d0 msgr2=0x7f76d419fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 --2- 172.21.15.32:0/122508632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76d41079d0 0x7f76d419fd40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f76cc007d70 tx=0x7f76cc00a340 comp rx=0 tx=0).stop 2024-04-16T05:33:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76da6a0700 1 -- 172.21.15.32:0/122508632 reap_dead start 2024-04-16T05:33:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 shutdown_connections 2024-04-16T05:33:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 >> 172.21.15.32:0/122508632 conn(0x7f76d4077a60 msgr2=0x7f76d40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.950+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 shutdown_connections 2024-04-16T05:33:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:23.951+0000 7f76dc103700 1 -- 172.21.15.32:0/122508632 wait complete. 2024-04-16T05:33:24.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:24 smithi032 ceph-mon[32351]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T05:33:25.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:24 smithi134 ceph-mon[36153]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T05:33:25.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:24 smithi082 ceph-mon[36195]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 -- 172.21.15.32:0/1845599254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 msgr2=0x7f6bdc10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 --2- 172.21.15.32:0/1845599254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 0x7f6bdc10f9c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400b0a0 tx=0x7f6bd400b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 -- 172.21.15.32:0/1845599254 shutdown_connections 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 --2- 172.21.15.32:0/1845599254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 0x7f6bdc10f9c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 --2- 172.21.15.32:0/1845599254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bdc108b80 0x7f6bdc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 --2- 172.21.15.32:0/1845599254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bdc102b70 0x7f6bdc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 -- 172.21.15.32:0/1845599254 >> 172.21.15.32:0/1845599254 conn(0x7f6bdc079bc0 msgr2=0x7f6bdc079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 -- 172.21.15.32:0/1845599254 shutdown_connections 2024-04-16T05:33:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.255+0000 7f6be1e14700 1 -- 172.21.15.32:0/1845599254 wait complete. 2024-04-16T05:33:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6be1e14700 1 Processor -- start 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6be1e14700 1 -- start start 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6be1e14700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bdc102b70 0x7f6bdc074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6be1e14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 0x7f6bdc075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6be1e14700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bdc108b80 0x7f6bdc075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6be1e14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc112e00 con 0x7f6bdc103530 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6be1e14700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc112c80 con 0x7f6bdc102b70 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6be1e14700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc112f80 con 0x7f6bdc108b80 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6bd2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 0x7f6bdc075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6bd2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 0x7f6bdc075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33482/0 (socket says 172.21.15.32:33482) 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6bd2ffd700 1 -- 172.21.15.32:0/1443851426 learned_addr learned my addr 172.21.15.32:0/1443851426 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.256+0000 7f6bdb7fe700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bdc102b70 0x7f6bdc074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.257+0000 7f6bdbfff700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bdc108b80 0x7f6bdc075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.257+0000 7f6bd2ffd700 1 -- 172.21.15.32:0/1443851426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bdc108b80 msgr2=0x7f6bdc075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.257+0000 7f6bd2ffd700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bdc108b80 0x7f6bdc075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.257+0000 7f6bd2ffd700 1 -- 172.21.15.32:0/1443851426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bdc102b70 msgr2=0x7f6bdc074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.257+0000 7f6bd2ffd700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bdc102b70 0x7f6bdc074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.257+0000 7f6bd2ffd700 1 -- 172.21.15.32:0/1443851426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc1a7730 con 0x7f6bdc103530 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.257+0000 7f6bd2ffd700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 0x7f6bdc075330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400d950 tx=0x7f6bc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.258+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4016070 con 0x7f6bdc103530 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.258+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc400fd70 con 0x7f6bdc103530 2024-04-16T05:33:25.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.258+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400e610 con 0x7f6bdc103530 2024-04-16T05:33:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.258+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc1a7a20 con 0x7f6bdc103530 2024-04-16T05:33:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.258+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a7f10 con 0x7f6bdc103530 2024-04-16T05:33:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.259+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bdc0631c0 con 0x7f6bdc103530 2024-04-16T05:33:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.261+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6bc4019070 con 0x7f6bdc103530 2024-04-16T05:33:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.262+0000 7f6bd97fa700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bc80a3d50 0x7f6bc80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.262+0000 7f6bdb7fe700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bc80a3d50 0x7f6bc80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.262+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6bc40d0700 con 0x7f6bdc103530 2024-04-16T05:33:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.262+0000 7f6bd97fa700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] conn(0x7f6bc80aea10 0x7f6bc80b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.262+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 --> [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bc4014e80 con 0x7f6bc80aea10 2024-04-16T05:33:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.262+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f6bc40d0b50 con 0x7f6bdc103530 2024-04-16T05:33:25.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.262+0000 7f6bdb7fe700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bc80a3d50 0x7f6bc80a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc005950 tx=0x7f6bcc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:25.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.262+0000 7f6bdbfff700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] conn(0x7f6bc80aea10 0x7f6bc80b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:25.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.263+0000 7f6bdbfff700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] conn(0x7f6bc80aea10 0x7f6bc80b0e50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:25.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.264+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 <== osd.11 v2:172.21.15.134:6824/1766870366 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc4014e80 con 0x7f6bc80aea10 2024-04-16T05:33:25.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.280+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 --> [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bdc063690 con 0x7f6bc80aea10 2024-04-16T05:33:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6bd97fa700 1 -- 172.21.15.32:0/1443851426 <== osd.11 v2:172.21.15.134:6824/1766870366 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bdc063690 con 0x7f6bc80aea10 2024-04-16T05:33:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 >> [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] conn(0x7f6bc80aea10 msgr2=0x7f6bc80b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] conn(0x7f6bc80aea10 0x7f6bc80b0e50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bc80a3d50 msgr2=0x7f6bc80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6bc80a3d50 0x7f6bc80a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc005950 tx=0x7f6bcc00a300 comp rx=0 tx=0).stop 2024-04-16T05:33:25.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 msgr2=0x7f6bdc075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:25.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 --2- 172.21.15.32:0/1443851426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bdc103530 0x7f6bdc075330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400d950 tx=0x7f6bc400dc60 comp rx=0 tx=0).stop 2024-04-16T05:33:25.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6bdbfff700 1 -- 172.21.15.32:0/1443851426 reap_dead start 2024-04-16T05:33:25.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 shutdown_connections 2024-04-16T05:33:25.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 >> 172.21.15.32:0/1443851426 conn(0x7f6bdc079bc0 msgr2=0x7f6bdc101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:25.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.281+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 shutdown_connections 2024-04-16T05:33:25.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:25.282+0000 7f6be1e14700 1 -- 172.21.15.32:0/1443851426 wait complete. 2024-04-16T05:33:26.405 INFO:teuthology.orchestra.run.smithi032.stdout:343597383703 2024-04-16T05:33:26.405 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-16T05:33:26.952 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:26 smithi032 ceph-mon[32351]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:33:27.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:26 smithi134 ceph-mon[36153]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:33:27.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:26 smithi082 ceph-mon[36195]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:33:27.279 INFO:teuthology.orchestra.run.smithi032.stdout:253403070502 2024-04-16T05:33:27.279 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-16T05:33:28.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.299+0000 7f48a8982700 1 -- 172.21.15.32:0/1651990074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a41023d0 msgr2=0x7f48a410e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.299+0000 7f48a8982700 1 --2- 172.21.15.32:0/1651990074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a41023d0 0x7f48a410e8e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f489c00b0a0 tx=0x7f489c00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 -- 172.21.15.32:0/1651990074 shutdown_connections 2024-04-16T05:33:28.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 --2- 172.21.15.32:0/1651990074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a41023d0 0x7f48a410e8e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 --2- 172.21.15.32:0/1651990074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a4107a10 0x7f48a4107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 --2- 172.21.15.32:0/1651990074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a4101a10 0x7f48a4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 -- 172.21.15.32:0/1651990074 >> 172.21.15.32:0/1651990074 conn(0x7f48a4078a90 msgr2=0x7f48a4078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:28.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 -- 172.21.15.32:0/1651990074 shutdown_connections 2024-04-16T05:33:28.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 -- 172.21.15.32:0/1651990074 wait complete. 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 Processor -- start 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.300+0000 7f48a8982700 1 -- start start 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a8982700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a4101a10 0x7f48a40fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a8982700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a41023d0 0x7f48a40fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a8982700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a4107a10 0x7f48a40ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a8982700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4111df0 con 0x7f48a4101a10 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a8982700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4111c70 con 0x7f48a41023d0 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a8982700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4111f70 con 0x7f48a4107a10 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a4101a10 0x7f48a40fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a4101a10 0x7f48a40fea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33502/0 (socket says 172.21.15.32:33502) 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 -- 172.21.15.32:0/751061303 learned_addr learned my addr 172.21.15.32:0/751061303 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a1d9b700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a41023d0 0x7f48a40fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a2d9d700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a4107a10 0x7f48a40ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 -- 172.21.15.32:0/751061303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a4107a10 msgr2=0x7f48a40ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a4107a10 0x7f48a40ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 -- 172.21.15.32:0/751061303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a41023d0 msgr2=0x7f48a40fef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a41023d0 0x7f48a40fef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 -- 172.21.15.32:0/751061303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a41a7880 con 0x7f48a4101a10 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a2d9d700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a4107a10 0x7f48a40ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a259c700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a4101a10 0x7f48a40fea40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f489400d950 tx=0x7f489400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894016070 con 0x7f48a4101a10 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a41a7b70 con 0x7f48a4101a10 2024-04-16T05:33:28.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4894009e20 con 0x7f48a4101a10 2024-04-16T05:33:28.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.301+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489400f620 con 0x7f48a4101a10 2024-04-16T05:33:28.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.302+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a41a80b0 con 0x7f48a4101a10 2024-04-16T05:33:28.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.302+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48a40631c0 con 0x7f48a4101a10 2024-04-16T05:33:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.304+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4894019070 con 0x7f48a4101a10 2024-04-16T05:33:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.305+0000 7f489b7fe700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f48900a3cd0 0x7f48900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.305+0000 7f48a1d9b700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f48900a3cd0 0x7f48900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.305+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f48940d08e0 con 0x7f48a4101a10 2024-04-16T05:33:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.305+0000 7f489b7fe700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] conn(0x7f48900ae990 0x7f48900b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.305+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 --> [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4894010e50 con 0x7f48900ae990 2024-04-16T05:33:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.305+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f48940d0c90 con 0x7f48a4101a10 2024-04-16T05:33:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.305+0000 7f48a1d9b700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f48900a3cd0 0x7f48900a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f48a4079090 tx=0x7f488c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:28.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.305+0000 7f48a2d9d700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] conn(0x7f48900ae990 0x7f48900b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:28.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.306+0000 7f48a2d9d700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] conn(0x7f48900ae990 0x7f48900b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:28.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.306+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 <== osd.1 v2:172.21.15.82:6800/4042955203 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4894010e50 con 0x7f48900ae990 2024-04-16T05:33:28.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.323+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 --> [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48a40635c0 con 0x7f48900ae990 2024-04-16T05:33:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.323+0000 7f489b7fe700 1 -- 172.21.15.32:0/751061303 <== osd.1 v2:172.21.15.82:6800/4042955203 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f48a40635c0 con 0x7f48900ae990 2024-04-16T05:33:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.323+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 >> [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] conn(0x7f48900ae990 msgr2=0x7f48900b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.323+0000 7f48a8982700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] conn(0x7f48900ae990 0x7f48900b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f48900a3cd0 msgr2=0x7f48900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a8982700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f48900a3cd0 0x7f48900a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f48a4079090 tx=0x7f488c009190 comp rx=0 tx=0).stop 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a4101a10 msgr2=0x7f48a40fea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a8982700 1 --2- 172.21.15.32:0/751061303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a4101a10 0x7f48a40fea40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f489400d950 tx=0x7f489400dc60 comp rx=0 tx=0).stop 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a2d9d700 1 -- 172.21.15.32:0/751061303 reap_dead start 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 shutdown_connections 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 >> 172.21.15.32:0/751061303 conn(0x7f48a4078a90 msgr2=0x7f48a406b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 shutdown_connections 2024-04-16T05:33:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:28.324+0000 7f48a8982700 1 -- 172.21.15.32:0/751061303 wait complete. 2024-04-16T05:33:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:28 smithi032 ceph-mon[32351]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T05:33:29.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:28 smithi134 ceph-mon[36153]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T05:33:29.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:28 smithi082 ceph-mon[36195]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T05:33:30.064 INFO:teuthology.orchestra.run.smithi032.stdout:55834574922 2024-04-16T05:33:30.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-16T05:33:30.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.265+0000 7f0dbe939700 1 -- 172.21.15.32:0/3447930805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8101a80 msgr2=0x7f0db8101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:30.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 --2- 172.21.15.32:0/3447930805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8101a80 0x7f0db8101f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009990 tx=0x7f0da0009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 -- 172.21.15.32:0/3447930805 shutdown_connections 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 --2- 172.21.15.32:0/3447930805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8102440 0x7f0db810e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 --2- 172.21.15.32:0/3447930805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0db81079f0 0x7f0db8107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 --2- 172.21.15.32:0/3447930805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8101a80 0x7f0db8101f00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 -- 172.21.15.32:0/3447930805 >> 172.21.15.32:0/3447930805 conn(0x7f0db8077a40 msgr2=0x7f0db8077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 -- 172.21.15.32:0/3447930805 shutdown_connections 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 -- 172.21.15.32:0/3447930805 wait complete. 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 Processor -- start 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.266+0000 7f0dbe939700 1 -- start start 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0dbe939700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8101a80 0x7f0db8198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0dbe939700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8102440 0x7f0db8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0dbe939700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0db81079f0 0x7f0db819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:30.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0dbe939700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8111d80 con 0x7f0db8102440 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0dbe939700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8111c00 con 0x7f0db8101a80 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0dbe939700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8111f00 con 0x7f0db81079f0 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8102440 0x7f0db8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8102440 0x7f0db8198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44196/0 (socket says 172.21.15.32:44196) 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 -- 172.21.15.32:0/2662301568 learned_addr learned my addr 172.21.15.32:0/2662301568 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0dbced6700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0db81079f0 0x7f0db819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0db7fff700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8101a80 0x7f0db8198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 -- 172.21.15.32:0/2662301568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0db81079f0 msgr2=0x7f0db819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0db81079f0 0x7f0db819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 -- 172.21.15.32:0/2662301568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8101a80 msgr2=0x7f0db8198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8101a80 0x7f0db8198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 -- 172.21.15.32:0/2662301568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db819d260 con 0x7f0db8102440 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.267+0000 7f0daffff700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8102440 0x7f0db8198790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0da0000c00 tx=0x7f0da0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.268+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0027070 con 0x7f0db8102440 2024-04-16T05:33:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.268+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da0004050 con 0x7f0db8102440 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.268+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da000f460 con 0x7f0db8102440 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.268+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db819d4f0 con 0x7f0db8102440 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.268+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81a4dd0 con 0x7f0db8102440 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.268+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0db8063200 con 0x7f0db8102440 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.270+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0da002a070 con 0x7f0db8102440 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.270+0000 7f0db5ffb700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0da40a3d50 0x7f0da40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.270+0000 7f0db7fff700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0da40a3d50 0x7f0da40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.270+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f0da00d1e60 con 0x7f0db8102440 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.271+0000 7f0db5ffb700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] conn(0x7f0da40aea10 0x7f0da40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.271+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 --> [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0da000b0b0 con 0x7f0da40aea10 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.271+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f0da00d22b0 con 0x7f0db8102440 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.271+0000 7f0db7fff700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0da40a3d50 0x7f0da40a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009a10 tx=0x7f0da8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.271+0000 7f0dbced6700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] conn(0x7f0da40aea10 0x7f0da40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.271+0000 7f0dbced6700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] conn(0x7f0da40aea10 0x7f0da40b0e50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.272+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 <== osd.7 v2:172.21.15.82:6816/529517800 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0da000b0b0 con 0x7f0da40aea10 2024-04-16T05:33:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.294+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 --> [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0db8063600 con 0x7f0da40aea10 2024-04-16T05:33:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2662301568 <== osd.7 v2:172.21.15.82:6816/529517800 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0db8063600 con 0x7f0da40aea10 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 >> [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] conn(0x7f0da40aea10 msgr2=0x7f0da40b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbe939700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] conn(0x7f0da40aea10 0x7f0da40b0e50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0da40a3d50 msgr2=0x7f0da40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbe939700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0da40a3d50 0x7f0da40a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009a10 tx=0x7f0da8009190 comp rx=0 tx=0).stop 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8102440 msgr2=0x7f0db8198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbe939700 1 --2- 172.21.15.32:0/2662301568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db8102440 0x7f0db8198790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0da0000c00 tx=0x7f0da0003680 comp rx=0 tx=0).stop 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbced6700 1 -- 172.21.15.32:0/2662301568 reap_dead start 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 shutdown_connections 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.295+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 >> 172.21.15.32:0/2662301568 conn(0x7f0db8077a40 msgr2=0x7f0db80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.296+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 shutdown_connections 2024-04-16T05:33:30.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.296+0000 7f0dbe939700 1 -- 172.21.15.32:0/2662301568 wait complete. 2024-04-16T05:33:30.885 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:30 smithi032 ceph-mon[32351]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-16T05:33:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3847404194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac81023e0 msgr2=0x7f6ac810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 --2- 172.21.15.32:0/3847404194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac81023e0 0x7f6ac810e8a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000b0a0 tx=0x7f6ac000b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3847404194 shutdown_connections 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 --2- 172.21.15.32:0/3847404194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac81023e0 0x7f6ac810e8a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 --2- 172.21.15.32:0/3847404194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac81079d0 0x7f6ac8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 --2- 172.21.15.32:0/3847404194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ac8101a20 0x7f6ac8101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3847404194 >> 172.21.15.32:0/3847404194 conn(0x7f6ac80789c0 msgr2=0x7f6ac8078dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3847404194 shutdown_connections 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3847404194 wait complete. 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.102+0000 7f6ad06c3700 1 Processor -- start 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ad06c3700 1 -- start start 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ad06c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac8101a20 0x7f6ac819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ad06c3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac81023e0 0x7f6ac819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ad06c3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ac81079d0 0x7f6ac81a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ad06c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac8111d30 con 0x7f6ac8101a20 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ad06c3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac8111bb0 con 0x7f6ac81023e0 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ad06c3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac8111eb0 con 0x7f6ac81079d0 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac8101a20 0x7f6ac819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac8101a20 0x7f6ac819a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44222/0 (socket says 172.21.15.32:44222) 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 -- 172.21.15.32:0/3551648067 learned_addr learned my addr 172.21.15.32:0/3551648067 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6acdc5e700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac81023e0 0x7f6ac819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 -- 172.21.15.32:0/3551648067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ac81079d0 msgr2=0x7f6ac81a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ac81079d0 0x7f6ac81a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 -- 172.21.15.32:0/3551648067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac81023e0 msgr2=0x7f6ac819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac81023e0 0x7f6ac819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 -- 172.21.15.32:0/3551648067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac81a5690 con 0x7f6ac8101a20 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6ace45f700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac8101a20 0x7f6ac819a0d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c920 tx=0x7f6ac400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.103+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4007b00 con 0x7f6ac8101a20 2024-04-16T05:33:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.104+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac81a58c0 con 0x7f6ac8101a20 2024-04-16T05:33:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.104+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ac4004d10 con 0x7f6ac8101a20 2024-04-16T05:33:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.104+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400f450 con 0x7f6ac8101a20 2024-04-16T05:33:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.104+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac81a5dd0 con 0x7f6ac8101a20 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.104+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ac80631c0 con 0x7f6ac8101a20 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.106+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ac4004750 con 0x7f6ac8101a20 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.106+0000 7f6abb7fe700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6ab40a3cd0 0x7f6ab40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.107+0000 7f6acdc5e700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6ab40a3cd0 0x7f6ab40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.107+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6ac40d0910 con 0x7f6ac8101a20 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.107+0000 7f6abb7fe700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] conn(0x7f6ab40ae980 0x7f6ab40b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.107+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 --> [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ac4012ec0 con 0x7f6ab40ae980 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.107+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f6ac40d0d60 con 0x7f6ac8101a20 2024-04-16T05:33:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.107+0000 7f6acec60700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] conn(0x7f6ab40ae980 0x7f6ab40b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.107+0000 7f6acdc5e700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6ab40a3cd0 0x7f6ab40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6abc005950 tx=0x7f6abc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.107+0000 7f6acec60700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] conn(0x7f6ab40ae980 0x7f6ab40b0dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.108+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 <== osd.3 v2:172.21.15.32:6810/3929583360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ac4012ec0 con 0x7f6ab40ae980 2024-04-16T05:33:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.126+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 --> [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ac8063640 con 0x7f6ab40ae980 2024-04-16T05:33:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.126+0000 7f6abb7fe700 1 -- 172.21.15.32:0/3551648067 <== osd.3 v2:172.21.15.32:6810/3929583360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6ac8063640 con 0x7f6ab40ae980 2024-04-16T05:33:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.126+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] conn(0x7f6ab40ae980 msgr2=0x7f6ab40b0dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.126+0000 7f6ad06c3700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] conn(0x7f6ab40ae980 0x7f6ab40b0dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6ab40a3cd0 msgr2=0x7f6ab40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6ad06c3700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f6ab40a3cd0 0x7f6ab40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6abc005950 tx=0x7f6abc0058e0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac8101a20 msgr2=0x7f6ac819a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6ad06c3700 1 --2- 172.21.15.32:0/3551648067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ac8101a20 0x7f6ac819a0d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c920 tx=0x7f6ac400cce0 comp rx=0 tx=0).stop 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6acec60700 1 -- 172.21.15.32:0/3551648067 reap_dead start 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 shutdown_connections 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 >> 172.21.15.32:0/3551648067 conn(0x7f6ac80789c0 msgr2=0x7f6ac80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 shutdown_connections 2024-04-16T05:33:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.127+0000 7f6ad06c3700 1 -- 172.21.15.32:0/3551648067 wait complete. 2024-04-16T05:33:31.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:30 smithi134 ceph-mon[36153]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-16T05:33:31.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:30 smithi082 ceph-mon[36195]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-16T05:33:32.199 INFO:teuthology.orchestra.run.smithi032.stdout:107374182464 2024-04-16T05:33:32.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 -- 172.21.15.32:0/3347419139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378108b80 msgr2=0x7f2378108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 --2- 172.21.15.32:0/3347419139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378108b80 0x7f2378108f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2368009990 tx=0x7f2368009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 -- 172.21.15.32:0/3347419139 shutdown_connections 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 --2- 172.21.15.32:0/3347419139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2378103530 0x7f237810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 --2- 172.21.15.32:0/3347419139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378108b80 0x7f2378108f60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 --2- 172.21.15.32:0/3347419139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2378102b70 0x7f2378102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 -- 172.21.15.32:0/3347419139 >> 172.21.15.32:0/3347419139 conn(0x7f2378079bc0 msgr2=0x7f2378079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 -- 172.21.15.32:0/3347419139 shutdown_connections 2024-04-16T05:33:32.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.423+0000 7f237cd92700 1 -- 172.21.15.32:0/3347419139 wait complete. 2024-04-16T05:33:32.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.424+0000 7f237cd92700 1 Processor -- start 2024-04-16T05:33:32.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.424+0000 7f237cd92700 1 -- start start 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.424+0000 7f237cd92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378102b70 0x7f237810f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237cd92700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2378103530 0x7f237810c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237cd92700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2378108b80 0x7f237810cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237cd92700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2378076560 con 0x7f2378102b70 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237cd92700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f23780763e0 con 0x7f2378108b80 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237cd92700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23780766e0 con 0x7f2378103530 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378102b70 0x7f237810f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378102b70 0x7f237810f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44242/0 (socket says 172.21.15.32:44242) 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 -- 172.21.15.32:0/1334395419 learned_addr learned my addr 172.21.15.32:0/1334395419 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f2376d9d700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2378108b80 0x7f237810cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 -- 172.21.15.32:0/1334395419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2378103530 msgr2=0x7f237810c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f2375d9b700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2378103530 0x7f237810c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2378103530 0x7f237810c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 -- 172.21.15.32:0/1334395419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2378108b80 msgr2=0x7f237810cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2378108b80 0x7f237810cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 -- 172.21.15.32:0/1334395419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237810d2e0 con 0x7f2378102b70 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f2376d9d700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2378108b80 0x7f237810cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f2375d9b700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2378103530 0x7f237810c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f237659c700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378102b70 0x7f237810f620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2368000c00 tx=0x7f23680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.425+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368027070 con 0x7f2378102b70 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.426+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23781abf70 con 0x7f2378102b70 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.426+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23781ac310 con 0x7f2378102b70 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.426+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2368004050 con 0x7f2378102b70 2024-04-16T05:33:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.426+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236800f460 con 0x7f2378102b70 2024-04-16T05:33:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.426+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2358001030 con 0x7f2378102b70 2024-04-16T05:33:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.428+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f236802a070 con 0x7f2378102b70 2024-04-16T05:33:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.429+0000 7f236f7fe700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f23640a3d50 0x7f23640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.429+0000 7f2375d9b700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f23640a3d50 0x7f23640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.429+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f23680d1cd0 con 0x7f2378102b70 2024-04-16T05:33:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.429+0000 7f236f7fe700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] conn(0x7f23640aea10 0x7f23640b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.429+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 --> [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23680043d0 con 0x7f23640aea10 2024-04-16T05:33:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.429+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f23680d2120 con 0x7f2378102b70 2024-04-16T05:33:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.430+0000 7f2376d9d700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] conn(0x7f23640aea10 0x7f23640b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.430+0000 7f2375d9b700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f23640a3d50 0x7f23640a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2360005fd0 tx=0x7f2360005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.430+0000 7f2376d9d700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] conn(0x7f23640aea10 0x7f23640b0e50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.431+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 <== osd.0 v2:172.21.15.32:6802/4225872342 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23680043d0 con 0x7f23640aea10 2024-04-16T05:33:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.447+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 --> [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2358002e60 con 0x7f23640aea10 2024-04-16T05:33:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f236f7fe700 1 -- 172.21.15.32:0/1334395419 <== osd.0 v2:172.21.15.32:6802/4225872342 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2358002e60 con 0x7f23640aea10 2024-04-16T05:33:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] conn(0x7f23640aea10 msgr2=0x7f23640b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] conn(0x7f23640aea10 0x7f23640b0e50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f23640a3d50 msgr2=0x7f23640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f23640a3d50 0x7f23640a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2360005fd0 tx=0x7f2360005dc0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378102b70 msgr2=0x7f237810f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 --2- 172.21.15.32:0/1334395419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2378102b70 0x7f237810f620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2368000c00 tx=0x7f23680047b0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f2376d9d700 1 -- 172.21.15.32:0/1334395419 reap_dead start 2024-04-16T05:33:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 shutdown_connections 2024-04-16T05:33:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 >> 172.21.15.32:0/1334395419 conn(0x7f2378079bc0 msgr2=0x7f23781013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 shutdown_connections 2024-04-16T05:33:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:32.448+0000 7f237cd92700 1 -- 172.21.15.32:0/1334395419 wait complete. 2024-04-16T05:33:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 -- 172.21.15.32:0/303512903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381023e0 msgr2=0x7f523810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 --2- 172.21.15.32:0/303512903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381023e0 0x7f523810e8a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f523000b0a0 tx=0x7f523000b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 -- 172.21.15.32:0/303512903 shutdown_connections 2024-04-16T05:33:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 --2- 172.21.15.32:0/303512903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381023e0 0x7f523810e8a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 --2- 172.21.15.32:0/303512903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52381079d0 0x7f5238107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 --2- 172.21.15.32:0/303512903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5238101a20 0x7f5238101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 -- 172.21.15.32:0/303512903 >> 172.21.15.32:0/303512903 conn(0x7f52380789a0 msgr2=0x7f5238078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 -- 172.21.15.32:0/303512903 shutdown_connections 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 -- 172.21.15.32:0/303512903 wait complete. 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 Processor -- start 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.991+0000 7f523e91d700 1 -- start start 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523e91d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5238101a20 0x7f52381035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523e91d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52381023e0 0x7f5238103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523e91d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381079d0 0x7f5238104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523e91d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5238111dc0 con 0x7f52381079d0 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523e91d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5238111c40 con 0x7f52381023e0 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523e91d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5238111f40 con 0x7f5238101a20 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381079d0 0x7f5238104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381079d0 0x7f5238104020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44214/0 (socket says 172.21.15.32:44214) 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 -- 172.21.15.32:0/752594717 learned_addr learned my addr 172.21.15.32:0/752594717 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f522ffff700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52381023e0 0x7f5238103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 -- 172.21.15.32:0/752594717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5238101a20 msgr2=0x7f52381035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f5237fff700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5238101a20 0x7f52381035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5238101a20 0x7f52381035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 -- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52381023e0 msgr2=0x7f5238103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52381023e0 0x7f5238103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 -- 172.21.15.32:0/752594717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52381a7830 con 0x7f52381079d0 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f5237fff700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5238101a20 0x7f52381035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f522ffff700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52381023e0 0x7f5238103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523ceba700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381079d0 0x7f5238104020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52300062a0 tx=0x7f523000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230021070 con 0x7f52381079d0 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5230007db0 con 0x7f52381079d0 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52381a7ac0 con 0x7f52381079d0 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230005460 con 0x7f52381079d0 2024-04-16T05:33:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.992+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52381a7f20 con 0x7f52381079d0 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.993+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52380631c0 con 0x7f52381079d0 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.995+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5230024070 con 0x7f52381079d0 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.995+0000 7f5235ffb700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52240a3cd0 0x7f52240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.996+0000 7f5237fff700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52240a3cd0 0x7f52240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.996+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f52300d1f00 con 0x7f52381079d0 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.996+0000 7f5235ffb700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] conn(0x7f52240ae860 0x7f52240b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.996+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 --> [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52240b1370 con 0x7f52240ae860 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.996+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f52300d2350 con 0x7f52381079d0 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.996+0000 7f5237fff700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52240a3cd0 0x7f52240a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5228007900 tx=0x7f5228008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.996+0000 7f522ffff700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] conn(0x7f52240ae860 0x7f52240b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.997+0000 7f522ffff700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] conn(0x7f52240ae860 0x7f52240b0ca0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:30.998+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 <== osd.10 v2:172.21.15.82:6824/4010044592 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52240b1370 con 0x7f52240ae860 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.015+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 --> [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5238063640 con 0x7f52240ae860 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f5235ffb700 1 -- 172.21.15.32:0/752594717 <== osd.10 v2:172.21.15.82:6824/4010044592 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5238063640 con 0x7f52240ae860 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] conn(0x7f52240ae860 msgr2=0x7f52240b0ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] conn(0x7f52240ae860 0x7f52240b0ca0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52240a3cd0 msgr2=0x7f52240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52240a3cd0 0x7f52240a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5228007900 tx=0x7f5228008040 comp rx=0 tx=0).stop 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381079d0 msgr2=0x7f5238104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 --2- 172.21.15.32:0/752594717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52381079d0 0x7f5238104020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52300062a0 tx=0x7f523000bf10 comp rx=0 tx=0).stop 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523ceba700 1 -- 172.21.15.32:0/752594717 reap_dead start 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 shutdown_connections 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 >> 172.21.15.32:0/752594717 conn(0x7f52380789a0 msgr2=0x7f523806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 shutdown_connections 2024-04-16T05:33:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:31.016+0000 7f523e91d700 1 -- 172.21.15.32:0/752594717 wait complete. 2024-04-16T05:33:32.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:32 smithi032 ceph-mon[32351]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-16T05:33:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:32 smithi134 ceph-mon[36153]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-16T05:33:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:32 smithi082 ceph-mon[36195]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-16T05:33:33.388 INFO:teuthology.orchestra.run.smithi032.stdout:309237645341 2024-04-16T05:33:33.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 -- 172.21.15.32:0/3518899530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0107bb0 msgr2=0x7f0ab0107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 --2- 172.21.15.32:0/3518899530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0107bb0 0x7f0ab0107f90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009990 tx=0x7f0aa0009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 -- 172.21.15.32:0/3518899530 shutdown_connections 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 --2- 172.21.15.32:0/3518899530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab0101e50 0x7f0ab010f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 --2- 172.21.15.32:0/3518899530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0107bb0 0x7f0ab0107f90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 --2- 172.21.15.32:0/3518899530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab0101490 0x7f0ab0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 -- 172.21.15.32:0/3518899530 >> 172.21.15.32:0/3518899530 conn(0x7f0ab0076d10 msgr2=0x7f0ab0077120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 -- 172.21.15.32:0/3518899530 shutdown_connections 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.810+0000 7f0ab68f3700 1 -- 172.21.15.32:0/3518899530 wait complete. 2024-04-16T05:33:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab68f3700 1 Processor -- start 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab68f3700 1 -- start start 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab68f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0101490 0x7f0ab019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab68f3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab0101e50 0x7f0ab019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab68f3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab0107bb0 0x7f0ab019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab68f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab0112e30 con 0x7f0ab0101490 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab68f3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab0112cb0 con 0x7f0ab0101e50 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab68f3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab0112fb0 con 0x7f0ab0107bb0 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0aaffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0101490 0x7f0ab019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0aaffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0101490 0x7f0ab019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44260/0 (socket says 172.21.15.32:44260) 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0aaffff700 1 -- 172.21.15.32:0/1248864328 learned_addr learned my addr 172.21.15.32:0/1248864328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0ab4e90700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab0107bb0 0x7f0ab019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.811+0000 7f0aa7fff700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab0101e50 0x7f0ab019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aaffff700 1 -- 172.21.15.32:0/1248864328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab0107bb0 msgr2=0x7f0ab019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aaffff700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab0107bb0 0x7f0ab019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aaffff700 1 -- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab0101e50 msgr2=0x7f0ab019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aaffff700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab0101e50 0x7f0ab019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aaffff700 1 -- 172.21.15.32:0/1248864328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab01a0470 con 0x7f0ab0101490 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aa7fff700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab0101e50 0x7f0ab019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0ab4e90700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab0107bb0 0x7f0ab019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aaffff700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0101490 0x7f0ab019b460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0000c00 tx=0x7f0aa000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0027070 con 0x7f0ab0101490 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab01a0700 con 0x7f0ab0101490 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aa0003d70 con 0x7f0ab0101490 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa000f6c0 con 0x7f0ab0101490 2024-04-16T05:33:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.812+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab01a7fe0 con 0x7f0ab0101490 2024-04-16T05:33:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.813+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ab01007b0 con 0x7f0ab0101490 2024-04-16T05:33:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.815+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0aa002a070 con 0x7f0ab0101490 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.815+0000 7f0aadffb700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0a9c0a3cd0 0x7f0a9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.816+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0aa00d2340 con 0x7f0ab0101490 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.816+0000 7f0aa7fff700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0a9c0a3cd0 0x7f0a9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.816+0000 7f0aadffb700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] conn(0x7f0a9c0aee30 0x7f0a9c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.816+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 --> [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0aa00043d0 con 0x7f0a9c0aee30 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.816+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0aa00d2790 con 0x7f0ab0101490 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.816+0000 7f0aa7fff700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0a9c0a3cd0 0x7f0a9c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0104740 tx=0x7f0a98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.816+0000 7f0ab4e90700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] conn(0x7f0a9c0aee30 0x7f0a9c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.816+0000 7f0ab4e90700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] conn(0x7f0a9c0aee30 0x7f0a9c0b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.817+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 <== osd.4 v2:172.21.15.82:6808/3071889784 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0aa00043d0 con 0x7f0a9c0aee30 2024-04-16T05:33:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.834+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 --> [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ab006ad40 con 0x7f0a9c0aee30 2024-04-16T05:33:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.834+0000 7f0aadffb700 1 -- 172.21.15.32:0/1248864328 <== osd.4 v2:172.21.15.82:6808/3071889784 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0ab006ad40 con 0x7f0a9c0aee30 2024-04-16T05:33:33.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] conn(0x7f0a9c0aee30 msgr2=0x7f0a9c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] conn(0x7f0a9c0aee30 0x7f0a9c0b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0a9c0a3cd0 msgr2=0x7f0a9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0a9c0a3cd0 0x7f0a9c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0104740 tx=0x7f0a98009260 comp rx=0 tx=0).stop 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0101490 msgr2=0x7f0ab019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 --2- 172.21.15.32:0/1248864328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab0101490 0x7f0ab019b460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0000c00 tx=0x7f0aa000bab0 comp rx=0 tx=0).stop 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab4e90700 1 -- 172.21.15.32:0/1248864328 reap_dead start 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 shutdown_connections 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 >> 172.21.15.32:0/1248864328 conn(0x7f0ab0076d10 msgr2=0x7f0ab00ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 shutdown_connections 2024-04-16T05:33:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:33.835+0000 7f0ab68f3700 1 -- 172.21.15.32:0/1248864328 wait complete. 2024-04-16T05:33:33.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-04-16T05:33:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:33 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-04-16T05:33:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-16T05:33:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-16T05:33:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-16T05:33:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-16T05:33:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-16T05:33:34.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T05:33:34.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-16T05:33:34.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-04-16T05:33:34.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-04-16T05:33:34.037 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:33 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-04-16T05:33:34.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:33 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-04-16T05:33:34.726 INFO:teuthology.orchestra.run.smithi032.stdout:34359738448 2024-04-16T05:33:34.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-16T05:33:34.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-04-16T05:33:34.964 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-04-16T05:33:34.964 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-04-16T05:33:34.964 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:34 smithi032 ceph-mon[32351]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T05:33:35.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-04-16T05:33:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:34 smithi134 ceph-mon[36153]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-04-16T05:33:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-04-16T05:33:35.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:34 smithi082 ceph-mon[36195]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 -- 172.21.15.32:0/482018930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76810f590 msgr2=0x7fd768111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 --2- 172.21.15.32:0/482018930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76810f590 0x7fd768111980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd76000b0a0 tx=0x7fd76000b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 -- 172.21.15.32:0/482018930 shutdown_connections 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 --2- 172.21.15.32:0/482018930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76810f590 0x7fd768111980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 --2- 172.21.15.32:0/482018930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd76806b090 0x7fd76806b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 --2- 172.21.15.32:0/482018930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd76806b9b0 0x7fd76810f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 -- 172.21.15.32:0/482018930 >> 172.21.15.32:0/482018930 conn(0x7fd768078960 msgr2=0x7fd768078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 -- 172.21.15.32:0/482018930 shutdown_connections 2024-04-16T05:33:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.472+0000 7fd76d6b0700 1 -- 172.21.15.32:0/482018930 wait complete. 2024-04-16T05:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd76d6b0700 1 Processor -- start 2024-04-16T05:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd76d6b0700 1 -- start start 2024-04-16T05:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd76d6b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76806b090 0x7fd76819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd76d6b0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd76806b9b0 0x7fd76819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd76d6b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd76810f590 0x7fd76819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd76d6b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd768115000 con 0x7fd76806b090 2024-04-16T05:33:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd76d6b0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd768114e80 con 0x7fd76810f590 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd76d6b0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd768115180 con 0x7fd76806b9b0 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76806b090 0x7fd76819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76806b090 0x7fd76819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44278/0 (socket says 172.21.15.32:44278) 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd766ffd700 1 -- 172.21.15.32:0/2852460997 learned_addr learned my addr 172.21.15.32:0/2852460997 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd7667fc700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd76806b9b0 0x7fd76819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.473+0000 7fd7677fe700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd76810f590 0x7fd76819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd766ffd700 1 -- 172.21.15.32:0/2852460997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd76806b9b0 msgr2=0x7fd76819b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd766ffd700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd76806b9b0 0x7fd76819b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd766ffd700 1 -- 172.21.15.32:0/2852460997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd76810f590 msgr2=0x7fd76819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd766ffd700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd76810f590 0x7fd76819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd766ffd700 1 -- 172.21.15.32:0/2852460997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7681a0470 con 0x7fd76806b090 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd7667fc700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd76806b9b0 0x7fd76819b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:33:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd766ffd700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76806b090 0x7fd76819b4b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd75800e9a0 tx=0x7fd75800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75800ca10 con 0x7fd76806b090 2024-04-16T05:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7681a0760 con 0x7fd76806b090 2024-04-16T05:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd75800cbb0 con 0x7fd76806b090 2024-04-16T05:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758018680 con 0x7fd76806b090 2024-04-16T05:33:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.474+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7681a7fe0 con 0x7fd76806b090 2024-04-16T05:33:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.475+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd768063240 con 0x7fd76806b090 2024-04-16T05:33:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.477+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd758010430 con 0x7fd76806b090 2024-04-16T05:33:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.477+0000 7fd75ffff700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd7540a3cd0 0x7fd7540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.477+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd7580d1320 con 0x7fd76806b090 2024-04-16T05:33:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.477+0000 7fd7667fc700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd7540a3cd0 0x7fd7540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.477+0000 7fd75ffff700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] conn(0x7fd7540aee30 0x7fd7540b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.477+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 --> [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd758027e50 con 0x7fd7540aee30 2024-04-16T05:33:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.477+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd7580d1770 con 0x7fd76806b090 2024-04-16T05:33:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.478+0000 7fd7677fe700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] conn(0x7fd7540aee30 0x7fd7540b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.478+0000 7fd7667fc700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd7540a3cd0 0x7fd7540a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd768102e60 tx=0x7fd750009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.478+0000 7fd7677fe700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] conn(0x7fd7540aee30 0x7fd7540b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.479+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 <== osd.6 v2:172.21.15.32:6818/1227886227 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd758027e50 con 0x7fd7540aee30 2024-04-16T05:33:35.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.495+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 --> [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd768063640 con 0x7fd7540aee30 2024-04-16T05:33:35.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd75ffff700 1 -- 172.21.15.32:0/2852460997 <== osd.6 v2:172.21.15.32:6818/1227886227 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd768063640 con 0x7fd7540aee30 2024-04-16T05:33:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] conn(0x7fd7540aee30 msgr2=0x7fd7540b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd76d6b0700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] conn(0x7fd7540aee30 0x7fd7540b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd7540a3cd0 msgr2=0x7fd7540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd76d6b0700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd7540a3cd0 0x7fd7540a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd768102e60 tx=0x7fd750009260 comp rx=0 tx=0).stop 2024-04-16T05:33:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76806b090 msgr2=0x7fd76819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd76d6b0700 1 --2- 172.21.15.32:0/2852460997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd76806b090 0x7fd76819b4b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd75800e9a0 tx=0x7fd75800ecb0 comp rx=0 tx=0).stop 2024-04-16T05:33:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd7677fe700 1 -- 172.21.15.32:0/2852460997 reap_dead start 2024-04-16T05:33:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 shutdown_connections 2024-04-16T05:33:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.496+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 >> 172.21.15.32:0/2852460997 conn(0x7fd768078960 msgr2=0x7fd7681109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.497+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 shutdown_connections 2024-04-16T05:33:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:35.497+0000 7fd76d6b0700 1 -- 172.21.15.32:0/2852460997 wait complete. 2024-04-16T05:33:36.024 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:35 smithi032 ceph-mon[32351]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T05:33:36.024 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:35 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:33:36.058 INFO:teuthology.orchestra.run.smithi032.stdout:141733920828 2024-04-16T05:33:36.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-16T05:33:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:35 smithi134 ceph-mon[36153]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T05:33:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:35 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:33:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:35 smithi082 ceph-mon[36195]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T05:33:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:35 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:33:36.786 INFO:teuthology.orchestra.run.smithi032.stdout:219043332140 2024-04-16T05:33:36.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-16T05:33:37.004 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:36 smithi032 ceph-mon[32351]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T05:33:37.004 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:36 smithi032 ceph-mon[32351]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-16T05:33:37.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:36 smithi134 ceph-mon[36153]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T05:33:37.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:36 smithi134 ceph-mon[36153]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-16T05:33:37.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:36 smithi082 ceph-mon[36195]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T05:33:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:36 smithi082 ceph-mon[36195]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-16T05:33:37.494 INFO:teuthology.orchestra.run.smithi032.stdout:197568495666 2024-04-16T05:33:37.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-16T05:33:37.529 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 -- 172.21.15.32:0/3625806600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738107ec0 msgr2=0x7fe73810fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 --2- 172.21.15.32:0/3625806600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738107ec0 0x7fe73810fa40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe73000b0d0 tx=0x7fe73000b3e0 comp rx=0 tx=0).stop 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 -- 172.21.15.32:0/3625806600 shutdown_connections 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 --2- 172.21.15.32:0/3625806600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738107ec0 0x7fe73810fa40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 --2- 172.21.15.32:0/3625806600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7381012a0 0x7fe738101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 --2- 172.21.15.32:0/3625806600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe738101c50 0x7fe738107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 -- 172.21.15.32:0/3625806600 >> 172.21.15.32:0/3625806600 conn(0x7fe738077ff0 msgr2=0x7fe7380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 -- 172.21.15.32:0/3625806600 shutdown_connections 2024-04-16T05:33:37.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 -- 172.21.15.32:0/3625806600 wait complete. 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 Processor -- start 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.980+0000 7fe73f9c3700 1 -- start start 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73f9c3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7381012a0 0x7fe738102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73f9c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738101c50 0x7fe738102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73f9c3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe738107ec0 0x7fe738102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73f9c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe738112e70 con 0x7fe738101c50 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73f9c3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe738112cf0 con 0x7fe7381012a0 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73f9c3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe738112ff0 con 0x7fe738107ec0 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73cf5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738101c50 0x7fe738102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73d75f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7381012a0 0x7fe738102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73cf5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738101c50 0x7fe738102760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44286/0 (socket says 172.21.15.32:44286) 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73cf5e700 1 -- 172.21.15.32:0/1500376655 learned_addr learned my addr 172.21.15.32:0/1500376655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73df60700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe738107ec0 0x7fe738102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73d75f700 1 -- 172.21.15.32:0/1500376655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe738107ec0 msgr2=0x7fe738102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73d75f700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe738107ec0 0x7fe738102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73d75f700 1 -- 172.21.15.32:0/1500376655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738101c50 msgr2=0x7fe738102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73d75f700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738101c50 0x7fe738102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73d75f700 1 -- 172.21.15.32:0/1500376655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7381a7870 con 0x7fe7381012a0 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73cf5e700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe738101c50 0x7fe738102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.981+0000 7fe73d75f700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7381012a0 0x7fe738102220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe73400e9f0 tx=0x7fe73400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:37.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.982+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734004220 con 0x7fe7381012a0 2024-04-16T05:33:37.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.982+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7381a7b60 con 0x7fe7381012a0 2024-04-16T05:33:37.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.982+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7381a80a0 con 0x7fe7381012a0 2024-04-16T05:33:37.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.982+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe734004d10 con 0x7fe7381012a0 2024-04-16T05:33:37.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.982+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73401c070 con 0x7fe7381012a0 2024-04-16T05:33:37.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.982+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe71c001030 con 0x7fe7381012a0 2024-04-16T05:33:37.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.984+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe7340043c0 con 0x7fe7381012a0 2024-04-16T05:33:37.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.984+0000 7fe72e7fc700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe7240a3cd0 0x7fe7240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:37.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.984+0000 7fe73cf5e700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe7240a3cd0 0x7fe7240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:37.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.984+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7340d0990 con 0x7fe7381012a0 2024-04-16T05:33:37.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.984+0000 7fe72e7fc700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] conn(0x7fe7240aee30 0x7fe7240b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:37.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.984+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 --> [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe734019e90 con 0x7fe7240aee30 2024-04-16T05:33:37.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.984+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe7340d0d40 con 0x7fe7381012a0 2024-04-16T05:33:37.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.985+0000 7fe73cf5e700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe7240a3cd0 0x7fe7240a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe738105b10 tx=0x7fe728009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:37.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.985+0000 7fe73df60700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] conn(0x7fe7240aee30 0x7fe7240b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:37.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.985+0000 7fe73df60700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] conn(0x7fe7240aee30 0x7fe7240b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:37.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:37.986+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 <== osd.5 v2:172.21.15.134:6808/1189980878 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe734019e90 con 0x7fe7240aee30 2024-04-16T05:33:38.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.004+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 --> [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe71c002e60 con 0x7fe7240aee30 2024-04-16T05:33:38.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe72e7fc700 1 -- 172.21.15.32:0/1500376655 <== osd.5 v2:172.21.15.134:6808/1189980878 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe71c002e60 con 0x7fe7240aee30 2024-04-16T05:33:38.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 >> [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] conn(0x7fe7240aee30 msgr2=0x7fe7240b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73f9c3700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] conn(0x7fe7240aee30 0x7fe7240b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:38.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe7240a3cd0 msgr2=0x7fe7240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73f9c3700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe7240a3cd0 0x7fe7240a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe738105b10 tx=0x7fe728009260 comp rx=0 tx=0).stop 2024-04-16T05:33:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7381012a0 msgr2=0x7fe738102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73f9c3700 1 --2- 172.21.15.32:0/1500376655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7381012a0 0x7fe738102220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe73400e9f0 tx=0x7fe73400ed00 comp rx=0 tx=0).stop 2024-04-16T05:33:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73df60700 1 -- 172.21.15.32:0/1500376655 reap_dead start 2024-04-16T05:33:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 shutdown_connections 2024-04-16T05:33:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.005+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 >> 172.21.15.32:0/1500376655 conn(0x7fe738077ff0 msgr2=0x7fe7380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.006+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 shutdown_connections 2024-04-16T05:33:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.006+0000 7fe73f9c3700 1 -- 172.21.15.32:0/1500376655 wait complete. 2024-04-16T05:33:38.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:37 smithi032 ceph-mon[32351]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-16T05:33:38.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:37 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 -- 172.21.15.32:0/988958367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9181019f0 msgr2=0x7fe918101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 --2- 172.21.15.32:0/988958367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9181019f0 0x7fe918101e70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009990 tx=0x7fe90c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 -- 172.21.15.32:0/988958367 shutdown_connections 2024-04-16T05:33:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 --2- 172.21.15.32:0/988958367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9181023b0 0x7fe91810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 --2- 172.21.15.32:0/988958367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe918107a40 0x7fe918107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 --2- 172.21.15.32:0/988958367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9181019f0 0x7fe918101e70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 -- 172.21.15.32:0/988958367 >> 172.21.15.32:0/988958367 conn(0x7fe9180789a0 msgr2=0x7fe918078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 -- 172.21.15.32:0/988958367 shutdown_connections 2024-04-16T05:33:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.254+0000 7fe91e719700 1 -- 172.21.15.32:0/988958367 wait complete. 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91e719700 1 Processor -- start 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91e719700 1 -- start start 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91e719700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9181019f0 0x7fe91819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91e719700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9181023b0 0x7fe91819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91e719700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe918107a40 0x7fe9181a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91e719700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe918111d50 con 0x7fe918107a40 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91e719700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe918111bd0 con 0x7fe9181019f0 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91e719700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe918111ed0 con 0x7fe9181023b0 2024-04-16T05:33:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91ccb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe918107a40 0x7fe9181a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91ccb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe918107a40 0x7fe9181a51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44294/0 (socket says 172.21.15.32:44294) 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe91ccb6700 1 -- 172.21.15.32:0/2101217608 learned_addr learned my addr 172.21.15.32:0/2101217608 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe9177fe700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9181023b0 0x7fe91819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.255+0000 7fe917fff700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9181019f0 0x7fe91819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe91ccb6700 1 -- 172.21.15.32:0/2101217608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9181023b0 msgr2=0x7fe91819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe91ccb6700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9181023b0 0x7fe91819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe91ccb6700 1 -- 172.21.15.32:0/2101217608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9181019f0 msgr2=0x7fe91819a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe91ccb6700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9181019f0 0x7fe91819a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe91ccb6700 1 -- 172.21.15.32:0/2101217608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9181a56f0 con 0x7fe918107a40 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe9177fe700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9181023b0 0x7fe91819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe91ccb6700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe918107a40 0x7fe9181a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe91000b970 tx=0x7fe91000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910019070 con 0x7fe918107a40 2024-04-16T05:33:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9181a5920 con 0x7fe918107a40 2024-04-16T05:33:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91000ce10 con 0x7fe918107a40 2024-04-16T05:33:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910007370 con 0x7fe918107a40 2024-04-16T05:33:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.256+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9181a5e30 con 0x7fe918107a40 2024-04-16T05:33:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.257+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe918063180 con 0x7fe918107a40 2024-04-16T05:33:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.259+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe91001c070 con 0x7fe918107a40 2024-04-16T05:33:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.259+0000 7fe9157fa700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9000a3cd0 0x7fe9000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.259+0000 7fe917fff700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9000a3cd0 0x7fe9000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.259+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe9100d03c0 con 0x7fe918107a40 2024-04-16T05:33:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.259+0000 7fe9157fa700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] conn(0x7fe9000aee30 0x7fe9000b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.259+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 --> [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9000b19a0 con 0x7fe9000aee30 2024-04-16T05:33:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.259+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe9100d0770 con 0x7fe918107a40 2024-04-16T05:33:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.260+0000 7fe9177fe700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] conn(0x7fe9000aee30 0x7fe9000b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.260+0000 7fe917fff700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9000a3cd0 0x7fe9000a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe908005950 tx=0x7fe90800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.260+0000 7fe9177fe700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] conn(0x7fe9000aee30 0x7fe9000b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.261+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 <== osd.9 v2:172.21.15.32:6826/192041066 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9000b19a0 con 0x7fe9000aee30 2024-04-16T05:33:38.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.277+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 --> [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe918063600 con 0x7fe9000aee30 2024-04-16T05:33:38.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.277+0000 7fe9157fa700 1 -- 172.21.15.32:0/2101217608 <== osd.9 v2:172.21.15.32:6826/192041066 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe918063600 con 0x7fe9000aee30 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] conn(0x7fe9000aee30 msgr2=0x7fe9000b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] conn(0x7fe9000aee30 0x7fe9000b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9000a3cd0 msgr2=0x7fe9000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe9000a3cd0 0x7fe9000a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe908005950 tx=0x7fe90800a300 comp rx=0 tx=0).stop 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe918107a40 msgr2=0x7fe9181a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 --2- 172.21.15.32:0/2101217608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe918107a40 0x7fe9181a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe91000b970 tx=0x7fe91000bc80 comp rx=0 tx=0).stop 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91ccb6700 1 -- 172.21.15.32:0/2101217608 reap_dead start 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 shutdown_connections 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 >> 172.21.15.32:0/2101217608 conn(0x7fe9180789a0 msgr2=0x7fe9180ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 shutdown_connections 2024-04-16T05:33:38.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:38.278+0000 7fe91e719700 1 -- 172.21.15.32:0/2101217608 wait complete. 2024-04-16T05:33:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:37 smithi134 ceph-mon[36153]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-16T05:33:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:37 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:37 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:37 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:37 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:37 smithi082 ceph-mon[36195]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-16T05:33:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:37 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:37 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:37 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:38.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:37 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:33:39.197 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:38 smithi032 ceph-mon[32351]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-04-16T05:33:39.198 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:38 smithi032 ceph-mon[32351]: Cluster is now healthy 2024-04-16T05:33:39.228 INFO:teuthology.orchestra.run.smithi032.stdout:287762808867 2024-04-16T05:33:39.228 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-16T05:33:39.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:38 smithi134 ceph-mon[36153]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-04-16T05:33:39.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:38 smithi134 ceph-mon[36153]: Cluster is now healthy 2024-04-16T05:33:39.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:38 smithi082 ceph-mon[36195]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-04-16T05:33:39.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:38 smithi082 ceph-mon[36195]: Cluster is now healthy 2024-04-16T05:33:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:39.988 INFO:teuthology.orchestra.run.smithi032.stdout:163208757304 2024-04-16T05:33:39.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-16T05:33:40.188 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:39 smithi032 ceph-mon[32351]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:40.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:39 smithi134 ceph-mon[36153]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:40.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:39 smithi082 ceph-mon[36195]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:41 smithi032 ceph-mon[32351]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:42.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:41 smithi134 ceph-mon[36153]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:42.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:41 smithi082 ceph-mon[36195]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:43.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 -- 172.21.15.32:0/3982325215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c108b70 msgr2=0x7fa10c108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:43.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 --2- 172.21.15.32:0/3982325215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c108b70 0x7fa10c108f50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00b0a0 tx=0x7fa0fc00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:43.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 -- 172.21.15.32:0/3982325215 shutdown_connections 2024-04-16T05:33:43.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 --2- 172.21.15.32:0/3982325215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10c06b1a0 0x7fa10c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:43.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 --2- 172.21.15.32:0/3982325215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c108b70 0x7fa10c108f50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:43.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 --2- 172.21.15.32:0/3982325215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa10c06a800 0x7fa10c06ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:43.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 -- 172.21.15.32:0/3982325215 >> 172.21.15.32:0/3982325215 conn(0x7fa10c0789a0 msgr2=0x7fa10c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:43.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 -- 172.21.15.32:0/3982325215 shutdown_connections 2024-04-16T05:33:43.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.903+0000 7fa112f0b700 1 -- 172.21.15.32:0/3982325215 wait complete. 2024-04-16T05:33:43.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa112f0b700 1 Processor -- start 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa112f0b700 1 -- start start 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa112f0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c06a800 0x7fa10c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa112f0b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa10c06b1a0 0x7fa10c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa112f0b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10c108b70 0x7fa10c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa112f0b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c112e30 con 0x7fa10c06a800 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa112f0b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c112cb0 con 0x7fa10c108b70 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa112f0b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c112fb0 con 0x7fa10c06b1a0 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa110ca7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c06a800 0x7fa10c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa110ca7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c06a800 0x7fa10c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53130/0 (socket says 172.21.15.32:53130) 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa110ca7700 1 -- 172.21.15.32:0/2454699832 learned_addr learned my addr 172.21.15.32:0/2454699832 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa1114a8700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10c108b70 0x7fa10c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa10bfff700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa10c06b1a0 0x7fa10c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa110ca7700 1 -- 172.21.15.32:0/2454699832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa10c06b1a0 msgr2=0x7fa10c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa110ca7700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa10c06b1a0 0x7fa10c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa110ca7700 1 -- 172.21.15.32:0/2454699832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10c108b70 msgr2=0x7fa10c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.904+0000 7fa110ca7700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10c108b70 0x7fa10c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa110ca7700 1 -- 172.21.15.32:0/2454699832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c19e290 con 0x7fa10c06a800 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa1114a8700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10c108b70 0x7fa10c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa10bfff700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa10c06b1a0 0x7fa10c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa110ca7700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c06a800 0x7fa10c199280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc000f80 tx=0x7fa0fc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa109ffb700 1 -- 172.21.15.32:0/2454699832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc021070 con 0x7fa10c06a800 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c19e520 con 0x7fa10c06a800 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa109ffb700 1 -- 172.21.15.32:0/2454699832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc007db0 con 0x7fa10c06a800 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa109ffb700 1 -- 172.21.15.32:0/2454699832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc005550 con 0x7fa10c06a800 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.905+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c1a5e00 con 0x7fa10c06a800 2024-04-16T05:33:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.906+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10c0697e0 con 0x7fa10c06a800 2024-04-16T05:33:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.907+0000 7fa109ffb700 1 -- 172.21.15.32:0/2454699832 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa0fc024070 con 0x7fa10c06a800 2024-04-16T05:33:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.908+0000 7fa109ffb700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa0f80a3cd0 0x7fa0f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.908+0000 7fa10bfff700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa0f80a3cd0 0x7fa0f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.908+0000 7fa109ffb700 1 -- 172.21.15.32:0/2454699832 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa0fc00f020 con 0x7fa10c06a800 2024-04-16T05:33:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.908+0000 7fa10bfff700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa0f80a3cd0 0x7fa0f80a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa10c078ea0 tx=0x7fa0f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:43.913+0000 7fa109ffb700 1 -- 172.21.15.32:0/2454699832 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0fc02b020 con 0x7fa10c06a800 2024-04-16T05:33:44.080 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:43 smithi032 ceph-mon[32351]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.125+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa10c02dfe0 con 0x7fa10c06a800 2024-04-16T05:33:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.125+0000 7fa109ffb700 1 -- 172.21.15.32:0/2454699832 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0fc09e660 con 0x7fa10c06a800 2024-04-16T05:33:44.124 INFO:teuthology.orchestra.run.smithi032.stdout:77309411401 2024-04-16T05:33:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.127+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa0f80a3cd0 msgr2=0x7fa0f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa0f80a3cd0 0x7fa0f80a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa10c078ea0 tx=0x7fa0f4008040 comp rx=0 tx=0).stop 2024-04-16T05:33:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c06a800 msgr2=0x7fa10c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c06a800 0x7fa10c199280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc000f80 tx=0x7fa0fc00bf10 comp rx=0 tx=0).stop 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 shutdown_connections 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fa0f80a3cd0 0x7fa0f80a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10c108b70 0x7fa10c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10c06a800 0x7fa10c199280 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 --2- 172.21.15.32:0/2454699832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa10c06b1a0 0x7fa10c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 >> 172.21.15.32:0/2454699832 conn(0x7fa10c0789a0 msgr2=0x7fa10c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 shutdown_connections 2024-04-16T05:33:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:44.128+0000 7fa112f0b700 1 -- 172.21.15.32:0/2454699832 wait complete. 2024-04-16T05:33:44.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:43 smithi134 ceph-mon[36153]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:44.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:43 smithi082 ceph-mon[36195]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-04-16T05:33:45.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:44 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2454699832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T05:33:45.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:44 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2454699832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T05:33:45.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:44 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2454699832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 -- 172.21.15.32:0/696618248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 msgr2=0x7f742810e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 --2- 172.21.15.32:0/696618248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 0x7f742810e8d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f742000b0d0 tx=0x7f742000b3e0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 -- 172.21.15.32:0/696618248 shutdown_connections 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 --2- 172.21.15.32:0/696618248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 0x7f742810e8d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 --2- 172.21.15.32:0/696618248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7428107a70 0x7f7428107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 --2- 172.21.15.32:0/696618248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7428101a20 0x7f7428101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 -- 172.21.15.32:0/696618248 >> 172.21.15.32:0/696618248 conn(0x7f7428078a30 msgr2=0x7f7428078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 -- 172.21.15.32:0/696618248 shutdown_connections 2024-04-16T05:33:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.583+0000 7f742c8c7700 1 -- 172.21.15.32:0/696618248 wait complete. 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742c8c7700 1 Processor -- start 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742c8c7700 1 -- start start 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742c8c7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7428101a20 0x7f7428199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742c8c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 0x7f7428199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742c8c7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7428107a70 0x7f742819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742c8c7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7428111d20 con 0x7f74281023e0 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742c8c7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7428111ba0 con 0x7f7428107a70 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742c8c7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7428111ea0 con 0x7f7428101a20 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 0x7f7428199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 0x7f7428199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53146/0 (socket says 172.21.15.32:53146) 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 -- 172.21.15.32:0/3039378742 learned_addr learned my addr 172.21.15.32:0/3039378742 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f742659c700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7428101a20 0x7f7428199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7426d9d700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7428107a70 0x7f742819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 -- 172.21.15.32:0/3039378742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7428101a20 msgr2=0x7f7428199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7428101a20 0x7f7428199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 -- 172.21.15.32:0/3039378742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7428107a70 msgr2=0x7f742819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7428107a70 0x7f742819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 -- 172.21.15.32:0/3039378742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742819e210 con 0x7f74281023e0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7426d9d700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7428107a70 0x7f742819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.584+0000 7f7425d9b700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 0x7f7428199740 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f741c00c920 tx=0x7f741c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.585+0000 7f74177fe700 1 -- 172.21.15.32:0/3039378742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c007b00 con 0x7f74281023e0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.585+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742819e500 con 0x7f74281023e0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.585+0000 7f74177fe700 1 -- 172.21.15.32:0/3039378742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741c004d10 con 0x7f74281023e0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.585+0000 7f74177fe700 1 -- 172.21.15.32:0/3039378742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c005630 con 0x7f74281023e0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.585+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74281a5d80 con 0x7f74281023e0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.586+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7428103140 con 0x7f74281023e0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.587+0000 7f74177fe700 1 -- 172.21.15.32:0/3039378742 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f741c004750 con 0x7f74281023e0 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.587+0000 7f74177fe700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74100a3cd0 0x7f74100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:45.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.587+0000 7f742659c700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74100a3cd0 0x7f74100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.588+0000 7f74177fe700 1 -- 172.21.15.32:0/3039378742 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f741c0d0470 con 0x7f74281023e0 2024-04-16T05:33:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.588+0000 7f742659c700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74100a3cd0 0x7f74100a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7418007b60 tx=0x7f74180073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.592+0000 7f74177fe700 1 -- 172.21.15.32:0/3039378742 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f741c014030 con 0x7f74281023e0 2024-04-16T05:33:45.601 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:45.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.813+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f742802dfe0 con 0x7f74281023e0 2024-04-16T05:33:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.813+0000 7f74177fe700 1 -- 172.21.15.32:0/3039378742 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f741c09c990 con 0x7f74281023e0 2024-04-16T05:33:45.812 INFO:teuthology.orchestra.run.smithi032.stdout:343597383707 2024-04-16T05:33:45.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74100a3cd0 msgr2=0x7f74100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74100a3cd0 0x7f74100a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7418007b60 tx=0x7f74180073d0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 msgr2=0x7f7428199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:45.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 0x7f7428199740 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f741c00c920 tx=0x7f741c00cc30 comp rx=0 tx=0).stop 2024-04-16T05:33:45.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 shutdown_connections 2024-04-16T05:33:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74100a3cd0 0x7f74100a6190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7428107a70 0x7f742819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7428101a20 0x7f7428199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 --2- 172.21.15.32:0/3039378742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74281023e0 0x7f7428199740 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 >> 172.21.15.32:0/3039378742 conn(0x7f7428078a30 msgr2=0x7f742806b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 shutdown_connections 2024-04-16T05:33:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:45.815+0000 7f742c8c7700 1 -- 172.21.15.32:0/3039378742 wait complete. 2024-04-16T05:33:46.144 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:45 smithi032 ceph-mon[32351]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:46.145 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:45 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3039378742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T05:33:46.301 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:45 smithi082 ceph-mon[36195]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:46.301 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:45 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3039378742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T05:33:46.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:45 smithi134 ceph-mon[36153]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:46.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:45 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3039378742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T05:33:46.585 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 -- 172.21.15.32:0/3383871781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1079d0 msgr2=0x7f59dc107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 --2- 172.21.15.32:0/3383871781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1079d0 0x7f59dc107db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00b0a0 tx=0x7f59cc00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 -- 172.21.15.32:0/3383871781 shutdown_connections 2024-04-16T05:33:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 --2- 172.21.15.32:0/3383871781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59dc1023e0 0x7f59dc10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 --2- 172.21.15.32:0/3383871781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1079d0 0x7f59dc107db0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 --2- 172.21.15.32:0/3383871781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc101a20 0x7f59dc101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 -- 172.21.15.32:0/3383871781 >> 172.21.15.32:0/3383871781 conn(0x7f59dc077a60 msgr2=0x7f59dc077e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 -- 172.21.15.32:0/3383871781 shutdown_connections 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.627+0000 7f59e2128700 1 -- 172.21.15.32:0/3383871781 wait complete. 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.628+0000 7f59e2128700 1 Processor -- start 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.628+0000 7f59e2128700 1 -- start start 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.628+0000 7f59e2128700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc101a20 0x7f59dc1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.628+0000 7f59e2128700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1023e0 0x7f59dc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.628+0000 7f59e2128700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59dc1079d0 0x7f59dc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59e2128700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc111d30 con 0x7f59dc1023e0 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59e2128700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc111bb0 con 0x7f59dc1079d0 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59e2128700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc111eb0 con 0x7f59dc101a20 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1023e0 0x7f59dc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1023e0 0x7f59dc198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53160/0 (socket says 172.21.15.32:53160) 2024-04-16T05:33:47.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 -- 172.21.15.32:0/2752906648 learned_addr learned my addr 172.21.15.32:0/2752906648 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59db7fe700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc101a20 0x7f59dc1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59dbfff700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59dc1079d0 0x7f59dc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 -- 172.21.15.32:0/2752906648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc101a20 msgr2=0x7f59dc1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc101a20 0x7f59dc1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 -- 172.21.15.32:0/2752906648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59dc1079d0 msgr2=0x7f59dc19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59dc1079d0 0x7f59dc19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 -- 172.21.15.32:0/2752906648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc19d1f0 con 0x7f59dc1023e0 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59daffd700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1023e0 0x7f59dc198720 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59d000d980 tx=0x7f59d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59d8ff9700 1 -- 172.21.15.32:0/2752906648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0017070 con 0x7f59dc1023e0 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59d8ff9700 1 -- 172.21.15.32:0/2752906648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59d0005020 con 0x7f59dc1023e0 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc19d4e0 con 0x7f59dc1023e0 2024-04-16T05:33:47.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59d8ff9700 1 -- 172.21.15.32:0/2752906648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d000fbf0 con 0x7f59dc1023e0 2024-04-16T05:33:47.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.629+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc1a4d60 con 0x7f59dc1023e0 2024-04-16T05:33:47.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.630+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc109fd0 con 0x7f59dc1023e0 2024-04-16T05:33:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.632+0000 7f59d8ff9700 1 -- 172.21.15.32:0/2752906648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f59d00051d0 con 0x7f59dc1023e0 2024-04-16T05:33:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.632+0000 7f59d8ff9700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59c40a3cd0 0x7f59c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.632+0000 7f59db7fe700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59c40a3cd0 0x7f59c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.632+0000 7f59d8ff9700 1 -- 172.21.15.32:0/2752906648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f59d00d0aa0 con 0x7f59dc1023e0 2024-04-16T05:33:47.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.633+0000 7f59db7fe700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59c40a3cd0 0x7f59c40a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009e60 tx=0x7f59cc00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.636+0000 7f59d8ff9700 1 -- 172.21.15.32:0/2752906648 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59d0020020 con 0x7f59dc1023e0 2024-04-16T05:33:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.849+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f59dc199570 con 0x7f59dc1023e0 2024-04-16T05:33:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.850+0000 7f59d8ff9700 1 -- 172.21.15.32:0/2752906648 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f59dc199570 con 0x7f59dc1023e0 2024-04-16T05:33:47.849 INFO:teuthology.orchestra.run.smithi032.stdout:253403070506 2024-04-16T05:33:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59c40a3cd0 msgr2=0x7f59c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59c40a3cd0 0x7f59c40a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009e60 tx=0x7f59cc00b7b0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1023e0 msgr2=0x7f59dc198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1023e0 0x7f59dc198720 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59d000d980 tx=0x7f59d000dc90 comp rx=0 tx=0).stop 2024-04-16T05:33:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 shutdown_connections 2024-04-16T05:33:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f59c40a3cd0 0x7f59c40a6190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59dc1079d0 0x7f59dc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc101a20 0x7f59dc1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 --2- 172.21.15.32:0/2752906648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59dc1023e0 0x7f59dc198720 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 >> 172.21.15.32:0/2752906648 conn(0x7f59dc077a60 msgr2=0x7f59dc0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.854+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 shutdown_connections 2024-04-16T05:33:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:47.855+0000 7f59e2128700 1 -- 172.21.15.32:0/2752906648 wait complete. 2024-04-16T05:33:48.182 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:47 smithi032 ceph-mon[32351]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:48.182 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:47 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2752906648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T05:33:48.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:47 smithi134 ceph-mon[36153]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:48.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:47 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2752906648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T05:33:48.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:47 smithi082 ceph-mon[36195]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:48.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:47 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2752906648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T05:33:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:33:50.036 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:50 smithi032 ceph-mon[32351]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:50.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:50 smithi134 ceph-mon[36153]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:50.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:50 smithi082 ceph-mon[36195]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T05:33:50.573 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070502 got 253403070506 for osd.8 2024-04-16T05:33:50.573 DEBUG:teuthology.parallel:result is None 2024-04-16T05:33:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 -- 172.21.15.32:0/1584909658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0394107a40 msgr2=0x7f0394107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 --2- 172.21.15.32:0/1584909658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0394107a40 0x7f0394107e20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0384009990 tx=0x7f0384009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 -- 172.21.15.32:0/1584909658 shutdown_connections 2024-04-16T05:33:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 --2- 172.21.15.32:0/1584909658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03941023b0 0x7f039410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 --2- 172.21.15.32:0/1584909658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0394107a40 0x7f0394107e20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 --2- 172.21.15.32:0/1584909658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03941019f0 0x7f0394101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 -- 172.21.15.32:0/1584909658 >> 172.21.15.32:0/1584909658 conn(0x7f03940789a0 msgr2=0x7f0394078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 -- 172.21.15.32:0/1584909658 shutdown_connections 2024-04-16T05:33:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.764+0000 7f0399b8d700 1 -- 172.21.15.32:0/1584909658 wait complete. 2024-04-16T05:33:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0399b8d700 1 Processor -- start 2024-04-16T05:33:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0399b8d700 1 -- start start 2024-04-16T05:33:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0399b8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03941019f0 0x7f03941035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0399b8d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03941023b0 0x7f0394103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0399b8d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0394107a40 0x7f0394104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0399b8d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0394111cf0 con 0x7f03941019f0 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0399b8d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0394111b70 con 0x7f0394107a40 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0399b8d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0394111e70 con 0x7f03941023b0 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f03937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03941019f0 0x7f03941035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f03937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03941019f0 0x7f03941035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47238/0 (socket says 172.21.15.32:47238) 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f03937fe700 1 -- 172.21.15.32:0/2683233549 learned_addr learned my addr 172.21.15.32:0/2683233549 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.765+0000 7f0392ffd700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03941023b0 0x7f0394103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f0393fff700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0394107a40 0x7f0394104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f03937fe700 1 -- 172.21.15.32:0/2683233549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03941023b0 msgr2=0x7f0394103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f03937fe700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03941023b0 0x7f0394103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f03937fe700 1 -- 172.21.15.32:0/2683233549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0394107a40 msgr2=0x7f0394104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f03937fe700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0394107a40 0x7f0394104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f03937fe700 1 -- 172.21.15.32:0/2683233549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03941a7750 con 0x7f03941019f0 2024-04-16T05:33:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f0393fff700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0394107a40 0x7f0394104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f03937fe700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03941019f0 0x7f03941035a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0384000c00 tx=0x7f038400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f0390ff9700 1 -- 172.21.15.32:0/2683233549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384027070 con 0x7f03941019f0 2024-04-16T05:33:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f0390ff9700 1 -- 172.21.15.32:0/2683233549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0384003d70 con 0x7f03941019f0 2024-04-16T05:33:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03941a79e0 con 0x7f03941019f0 2024-04-16T05:33:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f0390ff9700 1 -- 172.21.15.32:0/2683233549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038400f6c0 con 0x7f03941019f0 2024-04-16T05:33:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.766+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03941a7e30 con 0x7f03941019f0 2024-04-16T05:33:50.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.767+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03940697e0 con 0x7f03941019f0 2024-04-16T05:33:50.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.768+0000 7f0390ff9700 1 -- 172.21.15.32:0/2683233549 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f038402a070 con 0x7f03941019f0 2024-04-16T05:33:50.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.769+0000 7f0390ff9700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03800a3cd0 0x7f03800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:50.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.769+0000 7f0392ffd700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03800a3cd0 0x7f03800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:50.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.769+0000 7f0390ff9700 1 -- 172.21.15.32:0/2683233549 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f03840d19a0 con 0x7f03941019f0 2024-04-16T05:33:50.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.770+0000 7f0392ffd700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03800a3cd0 0x7f03800a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0394078ec0 tx=0x7f037c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:50.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.774+0000 7f0390ff9700 1 -- 172.21.15.32:0/2683233549 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f038405eff0 con 0x7f03941019f0 2024-04-16T05:33:50.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.995+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f03941a81c0 con 0x7f03941019f0 2024-04-16T05:33:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.995+0000 7f0390ff9700 1 -- 172.21.15.32:0/2683233549 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0384013070 con 0x7f03941019f0 2024-04-16T05:33:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:309237645345 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03800a3cd0 msgr2=0x7f03800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03800a3cd0 0x7f03800a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0394078ec0 tx=0x7f037c00a380 comp rx=0 tx=0).stop 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03941019f0 msgr2=0x7f03941035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03941019f0 0x7f03941035a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0384000c00 tx=0x7f038400bab0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 shutdown_connections 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f03800a3cd0 0x7f03800a6190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0394107a40 0x7f0394104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03941019f0 0x7f03941035a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 --2- 172.21.15.32:0/2683233549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03941023b0 0x7f0394103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 >> 172.21.15.32:0/2683233549 conn(0x7f03940789a0 msgr2=0x7f0394100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 shutdown_connections 2024-04-16T05:33:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:50.997+0000 7f0399b8d700 1 -- 172.21.15.32:0/2683233549 wait complete. 2024-04-16T05:33:51.165 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:51 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2683233549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T05:33:51.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:51 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2683233549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T05:33:51.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:51 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2683233549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T05:33:52.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:52 smithi032 ceph-mon[32351]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:33:52.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:52 smithi134 ceph-mon[36153]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:33:52.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:52 smithi082 ceph-mon[36195]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:33:53.261 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411401 for osd.2 2024-04-16T05:33:53.261 DEBUG:teuthology.parallel:result is None 2024-04-16T05:33:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 -- 172.21.15.32:0/2374985615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a01019f0 msgr2=0x7f47a0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 --2- 172.21.15.32:0/2374985615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a01019f0 0x7f47a0101e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4790009990 tx=0x7f4790009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 -- 172.21.15.32:0/2374985615 shutdown_connections 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 --2- 172.21.15.32:0/2374985615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47a01023b0 0x7f47a010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 --2- 172.21.15.32:0/2374985615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a0107a40 0x7f47a0107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 --2- 172.21.15.32:0/2374985615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a01019f0 0x7f47a0101e70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 -- 172.21.15.32:0/2374985615 >> 172.21.15.32:0/2374985615 conn(0x7f47a00789a0 msgr2=0x7f47a0078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 -- 172.21.15.32:0/2374985615 shutdown_connections 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.443+0000 7f47a7a48700 1 -- 172.21.15.32:0/2374985615 wait complete. 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a7a48700 1 Processor -- start 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a7a48700 1 -- start start 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a7a48700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a01019f0 0x7f47a01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a7a48700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47a01023b0 0x7f47a0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a7a48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a0107a40 0x7f47a019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a7a48700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0111cf0 con 0x7f47a0107a40 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a7a48700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0111b70 con 0x7f47a01019f0 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a7a48700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0111e70 con 0x7f47a01023b0 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a5fe5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a0107a40 0x7f47a019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a5fe5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a0107a40 0x7f47a019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47256/0 (socket says 172.21.15.32:47256) 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a5fe5700 1 -- 172.21.15.32:0/3786830314 learned_addr learned my addr 172.21.15.32:0/3786830314 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a57e4700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a01019f0 0x7f47a01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a4fe3700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47a01023b0 0x7f47a0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a5fe5700 1 -- 172.21.15.32:0/3786830314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47a01023b0 msgr2=0x7f47a0199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a5fe5700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47a01023b0 0x7f47a0199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a5fe5700 1 -- 172.21.15.32:0/3786830314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a01019f0 msgr2=0x7f47a01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.444+0000 7f47a5fe5700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a01019f0 0x7f47a01991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.445+0000 7f47a5fe5700 1 -- 172.21.15.32:0/3786830314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a019e1f0 con 0x7f47a0107a40 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.445+0000 7f47a4fe3700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47a01023b0 0x7f47a0199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.445+0000 7f47a5fe5700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a0107a40 0x7f47a019dab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f479800ef30 tx=0x7f479800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.445+0000 7f47967fc700 1 -- 172.21.15.32:0/3786830314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980092e0 con 0x7f47a0107a40 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.445+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a019e480 con 0x7f47a0107a40 2024-04-16T05:33:53.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.445+0000 7f47967fc700 1 -- 172.21.15.32:0/3786830314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798004bb0 con 0x7f47a0107a40 2024-04-16T05:33:53.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.445+0000 7f47967fc700 1 -- 172.21.15.32:0/3786830314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798004280 con 0x7f47a0107a40 2024-04-16T05:33:53.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.445+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01a5d60 con 0x7f47a0107a40 2024-04-16T05:33:53.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.446+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a00697e0 con 0x7f47a0107a40 2024-04-16T05:33:53.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.447+0000 7f47967fc700 1 -- 172.21.15.32:0/3786830314 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f479801b030 con 0x7f47a0107a40 2024-04-16T05:33:53.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.448+0000 7f47967fc700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f478c0a3cd0 0x7f478c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:53.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.448+0000 7f47a57e4700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f478c0a3cd0 0x7f478c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:53.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.448+0000 7f47967fc700 1 -- 172.21.15.32:0/3786830314 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f47980d0eb0 con 0x7f47a0107a40 2024-04-16T05:33:53.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.448+0000 7f47a57e4700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f478c0a3cd0 0x7f478c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f479c005fd0 tx=0x7f479c00af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:53.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.452+0000 7f47967fc700 1 -- 172.21.15.32:0/3786830314 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4798022020 con 0x7f47a0107a40 2024-04-16T05:33:53.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.659+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f47a019a540 con 0x7f47a0107a40 2024-04-16T05:33:53.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.660+0000 7f47967fc700 1 -- 172.21.15.32:0/3786830314 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4798010040 con 0x7f47a0107a40 2024-04-16T05:33:53.659 INFO:teuthology.orchestra.run.smithi032.stdout:34359738452 2024-04-16T05:33:53.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f478c0a3cd0 msgr2=0x7f478c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f478c0a3cd0 0x7f478c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f479c005fd0 tx=0x7f479c00af20 comp rx=0 tx=0).stop 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a0107a40 msgr2=0x7f47a019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a0107a40 0x7f47a019dab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f479800ef30 tx=0x7f479800c450 comp rx=0 tx=0).stop 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 shutdown_connections 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f478c0a3cd0 0x7f478c0a6190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47a0107a40 0x7f47a019dab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a01019f0 0x7f47a01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 --2- 172.21.15.32:0/3786830314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47a01023b0 0x7f47a0199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 >> 172.21.15.32:0/3786830314 conn(0x7f47a00789a0 msgr2=0x7f47a006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 shutdown_connections 2024-04-16T05:33:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:53.663+0000 7f47a7a48700 1 -- 172.21.15.32:0/3786830314 wait complete. 2024-04-16T05:33:54.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:54 smithi032 ceph-mon[32351]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:33:54.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:54 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3786830314' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T05:33:54.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:54 smithi134 ceph-mon[36153]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:33:54.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:54 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3786830314' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T05:33:54.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:54 smithi082 ceph-mon[36195]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:33:54.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:54 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3786830314' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T05:33:54.444 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645341 got 309237645345 for osd.10 2024-04-16T05:33:54.444 DEBUG:teuthology.parallel:result is None 2024-04-16T05:33:56.372 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:56 smithi032 ceph-mon[32351]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:33:56.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:56 smithi134 ceph-mon[36153]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:33:56.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:56 smithi082 ceph-mon[36195]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:33:57.881 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383707 for osd.11 2024-04-16T05:33:57.882 DEBUG:teuthology.parallel:result is None 2024-04-16T05:33:58.102 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:58 smithi032 ceph-mon[32351]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:33:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.335+0000 7f661b325700 1 -- 172.21.15.32:0/1134715115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614102bb0 msgr2=0x7f661410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.335+0000 7f661b325700 1 --2- 172.21.15.32:0/1134715115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614102bb0 0x7f661410e8a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f660c00b0a0 tx=0x7f660c00b3b0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.335+0000 7f661b325700 1 -- 172.21.15.32:0/1134715115 shutdown_connections 2024-04-16T05:33:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.335+0000 7f661b325700 1 --2- 172.21.15.32:0/1134715115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614102bb0 0x7f661410e8a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.335+0000 7f661b325700 1 --2- 172.21.15.32:0/1134715115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6614107a60 0x7f6614107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.335+0000 7f661b325700 1 --2- 172.21.15.32:0/1134715115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66141021f0 0x7f6614102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.335+0000 7f661b325700 1 -- 172.21.15.32:0/1134715115 >> 172.21.15.32:0/1134715115 conn(0x7f6614078090 msgr2=0x7f66140ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 -- 172.21.15.32:0/1134715115 shutdown_connections 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 -- 172.21.15.32:0/1134715115 wait complete. 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 Processor -- start 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 -- start start 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66141021f0 0x7f66141991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6614102bb0 0x7f6614199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614107a60 0x7f661419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6614111ce0 con 0x7f6614107a60 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6614111b60 con 0x7f66141021f0 2024-04-16T05:33:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.336+0000 7f661b325700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6614111e60 con 0x7f6614102bb0 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614107a60 0x7f661419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614107a60 0x7f661419da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47264/0 (socket says 172.21.15.32:47264) 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 -- 172.21.15.32:0/3201199480 learned_addr learned my addr 172.21.15.32:0/3201199480 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66188c0700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6614102bb0 0x7f6614199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66190c1700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66141021f0 0x7f66141991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 -- 172.21.15.32:0/3201199480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6614102bb0 msgr2=0x7f6614199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6614102bb0 0x7f6614199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 -- 172.21.15.32:0/3201199480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66141021f0 msgr2=0x7f66141991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66141021f0 0x7f66141991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 -- 172.21.15.32:0/3201199480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661419e180 con 0x7f6614107a60 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66188c0700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6614102bb0 0x7f6614199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f66198c2700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614107a60 0x7f661419da40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f660c0062a0 tx=0x7f660c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f660a7fc700 1 -- 172.21.15.32:0/3201199480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c021070 con 0x7f6614107a60 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f660a7fc700 1 -- 172.21.15.32:0/3201199480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f660c007c00 con 0x7f6614107a60 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661419e410 con 0x7f6614107a60 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f660a7fc700 1 -- 172.21.15.32:0/3201199480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c01d450 con 0x7f6614107a60 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.337+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66141a5cf0 con 0x7f6614107a60 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.338+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6614103200 con 0x7f6614107a60 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.340+0000 7f660a7fc700 1 -- 172.21.15.32:0/3201199480 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f660c004000 con 0x7f6614107a60 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.341+0000 7f660a7fc700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f66000a3d50 0x7f66000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.341+0000 7f66190c1700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f66000a3d50 0x7f66000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.341+0000 7f660a7fc700 1 -- 172.21.15.32:0/3201199480 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f660c00f020 con 0x7f6614107a60 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.341+0000 7f66190c1700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f66000a3d50 0x7f66000a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6610005950 tx=0x7f661000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.344+0000 7f660a7fc700 1 -- 172.21.15.32:0/3201199480 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f660c02a020 con 0x7f6614107a60 2024-04-16T05:33:58.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:58 smithi134 ceph-mon[36153]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:33:58.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:58 smithi082 ceph-mon[36195]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:33:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.553+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f66141a5ec0 con 0x7f6614107a60 2024-04-16T05:33:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.554+0000 7f660a7fc700 1 -- 172.21.15.32:0/3201199480 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f660c09e3c0 con 0x7f6614107a60 2024-04-16T05:33:58.553 INFO:teuthology.orchestra.run.smithi032.stdout:107374182469 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f66000a3d50 msgr2=0x7f66000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f66000a3d50 0x7f66000a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6610005950 tx=0x7f661000b410 comp rx=0 tx=0).stop 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614107a60 msgr2=0x7f661419da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614107a60 0x7f661419da40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f660c0062a0 tx=0x7f660c004d70 comp rx=0 tx=0).stop 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 shutdown_connections 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f66000a3d50 0x7f66000a6210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6614107a60 0x7f661419da40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66141021f0 0x7f66141991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 --2- 172.21.15.32:0/3201199480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6614102bb0 0x7f6614199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 >> 172.21.15.32:0/3201199480 conn(0x7f6614078090 msgr2=0x7f66140ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 shutdown_connections 2024-04-16T05:33:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:58.556+0000 7f661b325700 1 -- 172.21.15.32:0/3201199480 wait complete. 2024-04-16T05:33:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.020+0000 7ff7306a4700 1 -- 172.21.15.32:0/3780762594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 msgr2=0x7ff728101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.020+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3780762594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 0x7ff728101560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff718009990 tx=0x7ff718009ca0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 -- 172.21.15.32:0/3780762594 shutdown_connections 2024-04-16T05:33:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3780762594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff728101aa0 0x7ff728104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3780762594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7281058e0 0x7ff728105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3780762594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 0x7ff728101560 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 -- 172.21.15.32:0/3780762594 >> 172.21.15.32:0/3780762594 conn(0x7ff728078090 msgr2=0x7ff7280ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 -- 172.21.15.32:0/3780762594 shutdown_connections 2024-04-16T05:33:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 -- 172.21.15.32:0/3780762594 wait complete. 2024-04-16T05:33:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 Processor -- start 2024-04-16T05:33:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.021+0000 7ff7306a4700 1 -- start start 2024-04-16T05:33:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff7306a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 0x7ff7281a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff7306a4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff728101aa0 0x7ff7281a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff7306a4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7281058e0 0x7ff72819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff7306a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff728119470 con 0x7ff7281010e0 2024-04-16T05:33:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff7306a4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281192f0 con 0x7ff728101aa0 2024-04-16T05:33:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff7306a4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281195f0 con 0x7ff7281058e0 2024-04-16T05:33:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff72ec41700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7281058e0 0x7ff72819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff72e440700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 0x7ff7281a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff72ec41700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7281058e0 0x7ff72819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:56358/0 (socket says 172.21.15.32:56358) 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff72ec41700 1 -- 172.21.15.32:0/3683973057 learned_addr learned my addr 172.21.15.32:0/3683973057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.022+0000 7ff72e440700 1 -- 172.21.15.32:0/3683973057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7281058e0 msgr2=0x7ff72819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.023+0000 7ff72dc3f700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff728101aa0 0x7ff7281a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.023+0000 7ff72e440700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7281058e0 0x7ff72819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.023+0000 7ff72e440700 1 -- 172.21.15.32:0/3683973057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff728101aa0 msgr2=0x7ff7281a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.023+0000 7ff72e440700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff728101aa0 0x7ff7281a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.023+0000 7ff72e440700 1 -- 172.21.15.32:0/3683973057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72819fff0 con 0x7ff7281010e0 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.023+0000 7ff72ec41700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7281058e0 0x7ff72819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.023+0000 7ff72e440700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 0x7ff7281a5640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff72400d750 tx=0x7ff72400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.023+0000 7ff72dc3f700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff728101aa0 0x7ff7281a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:33:59.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.025+0000 7ff71f7fe700 1 -- 172.21.15.32:0/3683973057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724016070 con 0x7ff7281010e0 2024-04-16T05:33:59.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.025+0000 7ff71f7fe700 1 -- 172.21.15.32:0/3683973057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff724004d10 con 0x7ff7281010e0 2024-04-16T05:33:59.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.025+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7281a02e0 con 0x7ff7281010e0 2024-04-16T05:33:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.026+0000 7ff71f7fe700 1 -- 172.21.15.32:0/3683973057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72400f690 con 0x7ff7281010e0 2024-04-16T05:33:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.026+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7281ac5c0 con 0x7ff7281010e0 2024-04-16T05:33:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.027+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7281020f0 con 0x7ff7281010e0 2024-04-16T05:33:59.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.029+0000 7ff71f7fe700 1 -- 172.21.15.32:0/3683973057 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff724004750 con 0x7ff7281010e0 2024-04-16T05:33:59.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.029+0000 7ff71f7fe700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff7140a3d50 0x7ff7140a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:59.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.030+0000 7ff71f7fe700 1 -- 172.21.15.32:0/3683973057 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff7240d04c0 con 0x7ff7281010e0 2024-04-16T05:33:59.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.030+0000 7ff72dc3f700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff7140a3d50 0x7ff7140a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:59.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.031+0000 7ff72dc3f700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff7140a3d50 0x7ff7140a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff718000c00 tx=0x7ff718005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.035+0000 7ff71f7fe700 1 -- 172.21.15.32:0/3683973057 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff72401f020 con 0x7ff7281010e0 2024-04-16T05:33:59.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:33:59 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3201199480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T05:33:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.250+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff7281a0c40 con 0x7ff7281010e0 2024-04-16T05:33:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.251+0000 7ff71f7fe700 1 -- 172.21.15.32:0/3683973057 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff72409c9e0 con 0x7ff7281010e0 2024-04-16T05:33:59.250 INFO:teuthology.orchestra.run.smithi032.stdout:163208757308 2024-04-16T05:33:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff7140a3d50 msgr2=0x7ff7140a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff7140a3d50 0x7ff7140a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff718000c00 tx=0x7ff718005fb0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 msgr2=0x7ff7281a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 0x7ff7281a5640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff72400d750 tx=0x7ff72400da60 comp rx=0 tx=0).stop 2024-04-16T05:33:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 shutdown_connections 2024-04-16T05:33:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff7140a3d50 0x7ff7140a6210 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7281058e0 0x7ff72819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7281010e0 0x7ff7281a5640 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 --2- 172.21.15.32:0/3683973057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff728101aa0 0x7ff7281a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 >> 172.21.15.32:0/3683973057 conn(0x7ff728078090 msgr2=0x7ff7281025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 shutdown_connections 2024-04-16T05:33:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.253+0000 7ff7306a4700 1 -- 172.21.15.32:0/3683973057 wait complete. 2024-04-16T05:33:59.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:33:59 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3201199480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T05:33:59.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:33:59 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3201199480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T05:33:59.492 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738452 for osd.0 2024-04-16T05:33:59.492 DEBUG:teuthology.parallel:result is None 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 -- 172.21.15.32:0/46006606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81025a0 msgr2=0x7fc4d8102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 --2- 172.21.15.32:0/46006606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81025a0 0x7fc4d8102a20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80099c0 tx=0x7fc4c8009cd0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 -- 172.21.15.32:0/46006606 shutdown_connections 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 --2- 172.21.15.32:0/46006606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8102f60 0x7fc4d810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 --2- 172.21.15.32:0/46006606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4d8107cc0 0x7fc4d81080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 --2- 172.21.15.32:0/46006606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81025a0 0x7fc4d8102a20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 -- 172.21.15.32:0/46006606 >> 172.21.15.32:0/46006606 conn(0x7fc4d8076d10 msgr2=0x7fc4d8077120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 -- 172.21.15.32:0/46006606 shutdown_connections 2024-04-16T05:33:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.782+0000 7fc4e0135700 1 -- 172.21.15.32:0/46006606 wait complete. 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4e0135700 1 Processor -- start 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4e0135700 1 -- start start 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4e0135700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4d81025a0 0x7fc4d819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4e0135700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d8102f60 0x7fc4d819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4e0135700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8107cc0 0x7fc4d81a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4e0135700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8111e30 con 0x7fc4d8102f60 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4e0135700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8111cb0 con 0x7fc4d8107cc0 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4e0135700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8111fb0 con 0x7fc4d81025a0 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dd6d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d8102f60 0x7fc4d819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dd6d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d8102f60 0x7fc4d819a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59086/0 (socket says 172.21.15.32:59086) 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dd6d0700 1 -- 172.21.15.32:0/1275872929 learned_addr learned my addr 172.21.15.32:0/1275872929 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dded1700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4d81025a0 0x7fc4d819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dd6d0700 1 -- 172.21.15.32:0/1275872929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4d81025a0 msgr2=0x7fc4d819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dd6d0700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4d81025a0 0x7fc4d819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4de6d2700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8107cc0 0x7fc4d81a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dd6d0700 1 -- 172.21.15.32:0/1275872929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8107cc0 msgr2=0x7fc4d81a5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dd6d0700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8107cc0 0x7fc4d81a5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.783+0000 7fc4dd6d0700 1 -- 172.21.15.32:0/1275872929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d81a5890 con 0x7fc4d8102f60 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.784+0000 7fc4dd6d0700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d8102f60 0x7fc4d819a810 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80059d0 tx=0x7fc4c80038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.784+0000 7fc4ceffd700 1 -- 172.21.15.32:0/1275872929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8027070 con 0x7fc4d8102f60 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.784+0000 7fc4ceffd700 1 -- 172.21.15.32:0/1275872929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c8004050 con 0x7fc4d8102f60 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.784+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d81a5a60 con 0x7fc4d8102f60 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.784+0000 7fc4ceffd700 1 -- 172.21.15.32:0/1275872929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8022db0 con 0x7fc4d8102f60 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.784+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d81a5e50 con 0x7fc4d8102f60 2024-04-16T05:33:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.785+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d80697e0 con 0x7fc4d8102f60 2024-04-16T05:33:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.787+0000 7fc4ceffd700 1 -- 172.21.15.32:0/1275872929 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc4c802a070 con 0x7fc4d8102f60 2024-04-16T05:33:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.788+0000 7fc4ceffd700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc4c40a3cd0 0x7fc4c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:33:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.788+0000 7fc4dded1700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc4c40a3cd0 0x7fc4c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:33:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.788+0000 7fc4ceffd700 1 -- 172.21.15.32:0/1275872929 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc4c80d1d30 con 0x7fc4d8102f60 2024-04-16T05:33:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.788+0000 7fc4dded1700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc4c40a3cd0 0x7fc4c40a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4007b60 tx=0x7fc4d40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:33:59.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:33:59.791+0000 7fc4ceffd700 1 -- 172.21.15.32:0/1275872929 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4c8031020 con 0x7fc4d8102f60 2024-04-16T05:33:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.000+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc4d802e1a0 con 0x7fc4d8102f60 2024-04-16T05:33:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.001+0000 7fc4ceffd700 1 -- 172.21.15.32:0/1275872929 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4c8013070 con 0x7fc4d8102f60 2024-04-16T05:33:59.999 INFO:teuthology.orchestra.run.smithi032.stdout:55834574928 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc4c40a3cd0 msgr2=0x7fc4c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc4c40a3cd0 0x7fc4c40a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4007b60 tx=0x7fc4d40073d0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d8102f60 msgr2=0x7fc4d819a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d8102f60 0x7fc4d819a810 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80059d0 tx=0x7fc4c80038b0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 shutdown_connections 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc4c40a3cd0 0x7fc4c40a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8107cc0 0x7fc4d81a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4d81025a0 0x7fc4d819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 --2- 172.21.15.32:0/1275872929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d8102f60 0x7fc4d819a810 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 >> 172.21.15.32:0/1275872929 conn(0x7fc4d8076d10 msgr2=0x7fc4d80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:00.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 shutdown_connections 2024-04-16T05:34:00.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.003+0000 7fc4e0135700 1 -- 172.21.15.32:0/1275872929 wait complete. 2024-04-16T05:34:00.254 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:00 smithi032 ceph-mon[32351]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:00.254 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3683973057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T05:34:00.254 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1275872929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T05:34:00.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.258+0000 7fbd9817e700 1 -- 172.21.15.32:0/3669225216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 msgr2=0x7fbd90102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.258+0000 7fbd9817e700 1 --2- 172.21.15.32:0/3669225216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 0x7fbd90102fc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009990 tx=0x7fbd80009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 -- 172.21.15.32:0/3669225216 shutdown_connections 2024-04-16T05:34:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 --2- 172.21.15.32:0/3669225216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd90103500 0x7fbd9010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 --2- 172.21.15.32:0/3669225216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd90108b90 0x7fbd90108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 --2- 172.21.15.32:0/3669225216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 0x7fbd90102fc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 -- 172.21.15.32:0/3669225216 >> 172.21.15.32:0/3669225216 conn(0x7fbd90079c50 msgr2=0x7fbd9007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 -- 172.21.15.32:0/3669225216 shutdown_connections 2024-04-16T05:34:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 -- 172.21.15.32:0/3669225216 wait complete. 2024-04-16T05:34:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 Processor -- start 2024-04-16T05:34:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 -- start start 2024-04-16T05:34:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.259+0000 7fbd9817e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 0x7fbd90199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9817e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd90103500 0x7fbd90199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9817e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd90108b90 0x7fbd9019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9817e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbd90112df0 con 0x7fbd90102b40 2024-04-16T05:34:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9817e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbd90112c70 con 0x7fbd90103500 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9817e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd90112f70 con 0x7fbd90108b90 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 0x7fbd90199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 0x7fbd90199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59100/0 (socket says 172.21.15.32:59100) 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 -- 172.21.15.32:0/446119449 learned_addr learned my addr 172.21.15.32:0/446119449 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95719700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd90103500 0x7fbd90199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9671b700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd90108b90 0x7fbd9019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 -- 172.21.15.32:0/446119449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd90108b90 msgr2=0x7fbd9019dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd90108b90 0x7fbd9019dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 -- 172.21.15.32:0/446119449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd90103500 msgr2=0x7fbd90199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd90103500 0x7fbd90199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 -- 172.21.15.32:0/446119449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9019e330 con 0x7fbd90102b40 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9671b700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd90108b90 0x7fbd9019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd95f1a700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 0x7fbd90199320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00d950 tx=0x7fbd8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd86ffd700 1 -- 172.21.15.32:0/446119449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c016070 con 0x7fbd90102b40 2024-04-16T05:34:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd86ffd700 1 -- 172.21.15.32:0/446119449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd8c009e20 con 0x7fbd90102b40 2024-04-16T05:34:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9019e620 con 0x7fbd90102b40 2024-04-16T05:34:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd86ffd700 1 -- 172.21.15.32:0/446119449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00f620 con 0x7fbd90102b40 2024-04-16T05:34:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.260+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd901a5ea0 con 0x7fbd90102b40 2024-04-16T05:34:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.261+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd900697e0 con 0x7fbd90102b40 2024-04-16T05:34:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.263+0000 7fbd86ffd700 1 -- 172.21.15.32:0/446119449 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd8c019070 con 0x7fbd90102b40 2024-04-16T05:34:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.264+0000 7fbd86ffd700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbd7c0a3cd0 0x7fbd7c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.264+0000 7fbd95719700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbd7c0a3cd0 0x7fbd7c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.264+0000 7fbd86ffd700 1 -- 172.21.15.32:0/446119449 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbd8c0d0400 con 0x7fbd90102b40 2024-04-16T05:34:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.264+0000 7fbd95719700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbd7c0a3cd0 0x7fbd7c0a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000b5c0 tx=0x7fbd80005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:00.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.267+0000 7fbd86ffd700 1 -- 172.21.15.32:0/446119449 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd8c09c920 con 0x7fbd90102b40 2024-04-16T05:34:00.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:00 smithi134 ceph-mon[36153]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:00.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3683973057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T05:34:00.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1275872929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T05:34:00.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:00 smithi082 ceph-mon[36195]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:00.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3683973057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T05:34:00.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1275872929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T05:34:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.487+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbd9002dfe0 con 0x7fbd90102b40 2024-04-16T05:34:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.487+0000 7fbd86ffd700 1 -- 172.21.15.32:0/446119449 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd8c09c220 con 0x7fbd90102b40 2024-04-16T05:34:00.486 INFO:teuthology.orchestra.run.smithi032.stdout:197568495671 2024-04-16T05:34:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbd7c0a3cd0 msgr2=0x7fbd7c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbd7c0a3cd0 0x7fbd7c0a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000b5c0 tx=0x7fbd80005960 comp rx=0 tx=0).stop 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 msgr2=0x7fbd90199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 0x7fbd90199320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00d950 tx=0x7fbd8c00dc60 comp rx=0 tx=0).stop 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 shutdown_connections 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbd7c0a3cd0 0x7fbd7c0a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd90108b90 0x7fbd9019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd90102b40 0x7fbd90199320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 --2- 172.21.15.32:0/446119449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd90103500 0x7fbd90199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 >> 172.21.15.32:0/446119449 conn(0x7fbd90079c50 msgr2=0x7fbd901018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 shutdown_connections 2024-04-16T05:34:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:00.490+0000 7fbd9817e700 1 -- 172.21.15.32:0/446119449 wait complete. 2024-04-16T05:34:01.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757304 got 163208757308 for osd.5 2024-04-16T05:34:01.007 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:01.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/446119449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T05:34:01.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/446119449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T05:34:01.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/446119449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T05:34:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 -- 172.21.15.32:0/698446762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 msgr2=0x7f501810f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 --2- 172.21.15.32:0/698446762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 0x7f501810f9b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f501000b0a0 tx=0x7f501000b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 -- 172.21.15.32:0/698446762 shutdown_connections 2024-04-16T05:34:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 --2- 172.21.15.32:0/698446762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 0x7f501810f9b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 --2- 172.21.15.32:0/698446762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5018108b70 0x7f5018108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 --2- 172.21.15.32:0/698446762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f501806a800 0x7f501806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 -- 172.21.15.32:0/698446762 >> 172.21.15.32:0/698446762 conn(0x7f50180789a0 msgr2=0x7f5018078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 -- 172.21.15.32:0/698446762 shutdown_connections 2024-04-16T05:34:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.427+0000 7f501eb9f700 1 -- 172.21.15.32:0/698446762 wait complete. 2024-04-16T05:34:01.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501eb9f700 1 Processor -- start 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501eb9f700 1 -- start start 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501eb9f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f501806a800 0x7f5018101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501eb9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 0x7f5018102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501eb9f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5018108b70 0x7f5018102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501eb9f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5018112dd0 con 0x7f501806b1a0 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501eb9f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5018112c50 con 0x7f5018108b70 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501eb9f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5018112f50 con 0x7f501806a800 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f5017fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 0x7f5018102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f5017fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 0x7f5018102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59122/0 (socket says 172.21.15.32:59122) 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f5017fff700 1 -- 172.21.15.32:0/883632673 learned_addr learned my addr 172.21.15.32:0/883632673 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501c93b700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f501806a800 0x7f5018101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f501d13c700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5018108b70 0x7f5018102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f5017fff700 1 -- 172.21.15.32:0/883632673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f501806a800 msgr2=0x7f5018101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f5017fff700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f501806a800 0x7f5018101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f5017fff700 1 -- 172.21.15.32:0/883632673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5018108b70 msgr2=0x7f5018102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f5017fff700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5018108b70 0x7f5018102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.428+0000 7f5017fff700 1 -- 172.21.15.32:0/883632673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50181a7790 con 0x7f501806b1a0 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.429+0000 7f5017fff700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 0x7f5018102000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f500c00c920 tx=0x7f500c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.429+0000 7f5015ffb700 1 -- 172.21.15.32:0/883632673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c007b00 con 0x7f501806b1a0 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.429+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50181a7a80 con 0x7f501806b1a0 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.429+0000 7f5015ffb700 1 -- 172.21.15.32:0/883632673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c004d10 con 0x7f501806b1a0 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.429+0000 7f5015ffb700 1 -- 172.21.15.32:0/883632673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c005630 con 0x7f501806b1a0 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.429+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50181a8010 con 0x7f501806b1a0 2024-04-16T05:34:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.430+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50180697e0 con 0x7f501806b1a0 2024-04-16T05:34:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.431+0000 7f5015ffb700 1 -- 172.21.15.32:0/883632673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f500c004750 con 0x7f501806b1a0 2024-04-16T05:34:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.431+0000 7f5015ffb700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50000a3cd0 0x7f50000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.432+0000 7f501c93b700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50000a3cd0 0x7f50000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.432+0000 7f5015ffb700 1 -- 172.21.15.32:0/883632673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f500c0d0470 con 0x7f501806b1a0 2024-04-16T05:34:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.432+0000 7f501c93b700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50000a3cd0 0x7f50000a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f50080045b0 tx=0x7f500800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.436+0000 7f5015ffb700 1 -- 172.21.15.32:0/883632673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f500c014030 con 0x7f501806b1a0 2024-04-16T05:34:01.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.646+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f501802dfe0 con 0x7f501806b1a0 2024-04-16T05:34:01.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.646+0000 7f5015ffb700 1 -- 172.21.15.32:0/883632673 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f500c09c990 con 0x7f501806b1a0 2024-04-16T05:34:01.645 INFO:teuthology.orchestra.run.smithi032.stdout:287762808872 2024-04-16T05:34:01.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50000a3cd0 msgr2=0x7f50000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:01.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50000a3cd0 0x7f50000a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f50080045b0 tx=0x7f500800b3c0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 msgr2=0x7f5018102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:01.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 0x7f5018102000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f500c00c920 tx=0x7f500c00cc30 comp rx=0 tx=0).stop 2024-04-16T05:34:01.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 shutdown_connections 2024-04-16T05:34:01.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f50000a3cd0 0x7f50000a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5018108b70 0x7f5018102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f501806a800 0x7f5018101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 --2- 172.21.15.32:0/883632673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501806b1a0 0x7f5018102000 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 >> 172.21.15.32:0/883632673 conn(0x7f50180789a0 msgr2=0x7f50180ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.648+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 shutdown_connections 2024-04-16T05:34:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:01.649+0000 7f501eb9f700 1 -- 172.21.15.32:0/883632673 wait complete. 2024-04-16T05:34:02.085 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574928 for osd.1 2024-04-16T05:34:02.085 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:02.356 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:02 smithi032 ceph-mon[32351]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:02.356 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:02 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/883632673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T05:34:02.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:02 smithi134 ceph-mon[36153]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:02.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:02 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/883632673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T05:34:02.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:02 smithi082 ceph-mon[36195]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:02.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:02 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/883632673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T05:34:02.506 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182464 got 107374182469 for osd.3 2024-04-16T05:34:02.507 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:03.154 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808872 for osd.9 2024-04-16T05:34:03.154 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:03.161 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:04.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:04 smithi134 ceph-mon[36153]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:04.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:04 smithi134 ceph-mon[36153]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-16T05:34:04.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:04 smithi032 ceph-mon[32351]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:04.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:04 smithi032 ceph-mon[32351]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-16T05:34:04.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:04 smithi082 ceph-mon[36195]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:04.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:04 smithi082 ceph-mon[36195]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-16T05:34:04.922 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495666 got 197568495671 for osd.6 2024-04-16T05:34:04.923 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 -- 172.21.15.32:0/950971462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 msgr2=0x7f77a8101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/950971462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 0x7f77a8101ea0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f779c009990 tx=0x7f779c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 -- 172.21.15.32:0/950971462 shutdown_connections 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/950971462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a81023e0 0x7f77a810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/950971462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77a81079d0 0x7f77a8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/950971462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 0x7f77a8101ea0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 -- 172.21.15.32:0/950971462 >> 172.21.15.32:0/950971462 conn(0x7f77a80789c0 msgr2=0x7f77a8078dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 -- 172.21.15.32:0/950971462 shutdown_connections 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 -- 172.21.15.32:0/950971462 wait complete. 2024-04-16T05:34:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 Processor -- start 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.595+0000 7f77ae6c1700 1 -- start start 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77ae6c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 0x7f77a8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77ae6c1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77a81023e0 0x7f77a81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77ae6c1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a81079d0 0x7f77a819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77ae6c1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8111d60 con 0x7f77a8101a20 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77ae6c1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8111be0 con 0x7f77a81023e0 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77ae6c1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8111ee0 con 0x7f77a81079d0 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 0x7f77a8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 0x7f77a8199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59148/0 (socket says 172.21.15.32:59148) 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 -- 172.21.15.32:0/2453164854 learned_addr learned my addr 172.21.15.32:0/2453164854 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a77fe700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77a81023e0 0x7f77a81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77acc5e700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a81079d0 0x7f77a819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 -- 172.21.15.32:0/2453164854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a81079d0 msgr2=0x7f77a819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a81079d0 0x7f77a819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 -- 172.21.15.32:0/2453164854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77a81023e0 msgr2=0x7f77a81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77a81023e0 0x7f77a81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 -- 172.21.15.32:0/2453164854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a819e290 con 0x7f77a8101a20 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a77fe700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77a81023e0 0x7f77a81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a7fff700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 0x7f77a8199280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f779800c920 tx=0x7f779800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a57fa700 1 -- 172.21.15.32:0/2453164854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798007b00 con 0x7f77a8101a20 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a57fa700 1 -- 172.21.15.32:0/2453164854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798004d10 con 0x7f77a8101a20 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a819e580 con 0x7f77a8101a20 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.596+0000 7f77a57fa700 1 -- 172.21.15.32:0/2453164854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779800f450 con 0x7f77a8101a20 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.597+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a81a5e00 con 0x7f77a8101a20 2024-04-16T05:34:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.597+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a80697e0 con 0x7f77a8101a20 2024-04-16T05:34:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.599+0000 7f77a57fa700 1 -- 172.21.15.32:0/2453164854 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7798004750 con 0x7f77a8101a20 2024-04-16T05:34:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.600+0000 7f77a57fa700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f77900a3cc0 0x7f77900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.600+0000 7f77a57fa700 1 -- 172.21.15.32:0/2453164854 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f77980d0680 con 0x7f77a8101a20 2024-04-16T05:34:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.600+0000 7f77a77fe700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f77900a3cc0 0x7f77900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.600+0000 7f77a77fe700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f77900a3cc0 0x7f77900a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f779c006010 tx=0x7f779c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:05.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.605+0000 7f77a57fa700 1 -- 172.21.15.32:0/2453164854 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7798014030 con 0x7f77a8101a20 2024-04-16T05:34:05.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.815+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f77a802dfe0 con 0x7f77a8101a20 2024-04-16T05:34:05.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.815+0000 7f77a57fa700 1 -- 172.21.15.32:0/2453164854 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f779809cd00 con 0x7f77a8101a20 2024-04-16T05:34:05.814 INFO:teuthology.orchestra.run.smithi032.stdout:141733920834 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f77900a3cc0 msgr2=0x7f77900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f77900a3cc0 0x7f77900a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f779c006010 tx=0x7f779c00b540 comp rx=0 tx=0).stop 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 msgr2=0x7f77a8199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 0x7f77a8199280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f779800c920 tx=0x7f779800cce0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 shutdown_connections 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f77900a3cc0 0x7f77900a6180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a81079d0 0x7f77a819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a8101a20 0x7f77a8199280 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 --2- 172.21.15.32:0/2453164854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77a81023e0 0x7f77a81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:05.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 >> 172.21.15.32:0/2453164854 conn(0x7f77a80789c0 msgr2=0x7f77a806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 shutdown_connections 2024-04-16T05:34:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:05.817+0000 7f77ae6c1700 1 -- 172.21.15.32:0/2453164854 wait complete. 2024-04-16T05:34:06.214 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:06 smithi032 ceph-mon[32351]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:34:06.214 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:06 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2453164854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T05:34:06.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:06 smithi134 ceph-mon[36153]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:34:06.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:06 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2453164854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T05:34:06.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:06 smithi082 ceph-mon[36195]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:34:06.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:06 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2453164854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T05:34:06.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920834 for osd.4 2024-04-16T05:34:06.667 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.933+0000 7f76ee4e9700 1 -- 172.21.15.32:0/2075949323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 msgr2=0x7f76e810f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.933+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/2075949323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 0x7f76e810f9b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b0a0 tx=0x7f76e000b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 -- 172.21.15.32:0/2075949323 shutdown_connections 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/2075949323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 0x7f76e810f9b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/2075949323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76e8108b70 0x7f76e8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/2075949323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e806a800 0x7f76e806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 -- 172.21.15.32:0/2075949323 >> 172.21.15.32:0/2075949323 conn(0x7f76e80789a0 msgr2=0x7f76e8078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 -- 172.21.15.32:0/2075949323 shutdown_connections 2024-04-16T05:34:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 -- 172.21.15.32:0/2075949323 wait complete. 2024-04-16T05:34:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 Processor -- start 2024-04-16T05:34:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.934+0000 7f76ee4e9700 1 -- start start 2024-04-16T05:34:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76ee4e9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e806a800 0x7f76e8101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76ee4e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 0x7f76e8102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76ee4e9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76e8108b70 0x7f76e8102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76ee4e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112dd0 con 0x7f76e806b1a0 2024-04-16T05:34:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76ee4e9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112c50 con 0x7f76e806a800 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76ee4e9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112f50 con 0x7f76e8108b70 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 0x7f76e8102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 0x7f76e8102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59162/0 (socket says 172.21.15.32:59162) 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 -- 172.21.15.32:0/3863666981 learned_addr learned my addr 172.21.15.32:0/3863666981 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e7fff700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e806a800 0x7f76e8101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76eca86700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76e8108b70 0x7f76e8102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 -- 172.21.15.32:0/3863666981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76e8108b70 msgr2=0x7f76e8102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76e8108b70 0x7f76e8102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 -- 172.21.15.32:0/3863666981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e806a800 msgr2=0x7f76e8101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e806a800 0x7f76e8101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 -- 172.21.15.32:0/3863666981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81a7860 con 0x7f76e806b1a0 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e77fe700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 0x7f76e8102000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e9a0 tx=0x7f76dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e57fa700 1 -- 172.21.15.32:0/3863666981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00ca10 con 0x7f76e806b1a0 2024-04-16T05:34:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76e57fa700 1 -- 172.21.15.32:0/3863666981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76dc00cbb0 con 0x7f76e806b1a0 2024-04-16T05:34:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.935+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81a7b50 con 0x7f76e806b1a0 2024-04-16T05:34:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.936+0000 7f76e57fa700 1 -- 172.21.15.32:0/3863666981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc018680 con 0x7f76e806b1a0 2024-04-16T05:34:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.936+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a8090 con 0x7f76e806b1a0 2024-04-16T05:34:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.937+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e80697e0 con 0x7f76e806b1a0 2024-04-16T05:34:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.938+0000 7f76e57fa700 1 -- 172.21.15.32:0/3863666981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f76dc010430 con 0x7f76e806b1a0 2024-04-16T05:34:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.939+0000 7f76e57fa700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76d00a3c40 0x7f76d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.939+0000 7f76e57fa700 1 -- 172.21.15.32:0/3863666981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f76dc0d0dc0 con 0x7f76e806b1a0 2024-04-16T05:34:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.939+0000 7f76e7fff700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76d00a3c40 0x7f76d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.940+0000 7f76e7fff700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76d00a3c40 0x7f76d00a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009be0 tx=0x7f76d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:06.943+0000 7f76e57fa700 1 -- 172.21.15.32:0/3863666981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76dc09d440 con 0x7f76e806b1a0 2024-04-16T05:34:07.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.157+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f76e802dfe0 con 0x7f76e806b1a0 2024-04-16T05:34:07.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.157+0000 7f76e57fa700 1 -- 172.21.15.32:0/3863666981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f76dc09cd40 con 0x7f76e806b1a0 2024-04-16T05:34:07.156 INFO:teuthology.orchestra.run.smithi032.stdout:219043332148 2024-04-16T05:34:07.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.159+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76d00a3c40 msgr2=0x7f76d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:07.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.159+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76d00a3c40 0x7f76d00a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009be0 tx=0x7f76d8009260 comp rx=0 tx=0).stop 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.159+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 msgr2=0x7f76e8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.159+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 0x7f76e8102000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e9a0 tx=0x7f76dc00ecb0 comp rx=0 tx=0).stop 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.160+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 shutdown_connections 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.160+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f76d00a3c40 0x7f76d00a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.160+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76e8108b70 0x7f76e8102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.160+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e806a800 0x7f76e8101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.160+0000 7f76ee4e9700 1 --2- 172.21.15.32:0/3863666981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76e806b1a0 0x7f76e8102000 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.160+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 >> 172.21.15.32:0/3863666981 conn(0x7f76e80789a0 msgr2=0x7f76e80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.160+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 shutdown_connections 2024-04-16T05:34:07.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:07.160+0000 7f76ee4e9700 1 -- 172.21.15.32:0/3863666981 wait complete. 2024-04-16T05:34:07.881 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332140 got 219043332148 for osd.7 2024-04-16T05:34:07.881 DEBUG:teuthology.parallel:result is None 2024-04-16T05:34:07.881 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-16T05:34:07.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph pg dump --format=json 2024-04-16T05:34:08.140 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:08.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:08 smithi032 ceph-mon[32351]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:34:08.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:08.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:08 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:08.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:08 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3863666981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T05:34:08.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:08 smithi134 ceph-mon[36153]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:34:08.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:08.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:08 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:08.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:08 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3863666981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T05:34:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:08 smithi082 ceph-mon[36195]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:34:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:08 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:08.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:08 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3863666981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T05:34:09.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:09.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:09.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:09 smithi134 ceph-mon[36153]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:09.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:09.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:09.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:09 smithi082 ceph-mon[36195]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:09.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:09.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:09.423 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:09 smithi032 ceph-mon[32351]: from='mgr.14152 172.21.15.32:0/1470028372' entity='mgr.x' 2024-04-16T05:34:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 -- 172.21.15.32:0/1399189691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102e50 msgr2=0x7f66b010f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1399189691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102e50 0x7f66b010f090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009990 tx=0x7f66a0009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 -- 172.21.15.32:0/1399189691 shutdown_connections 2024-04-16T05:34:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1399189691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b010f5d0 0x7f66b01119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1399189691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b0102530 0x7f66b0102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1399189691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102e50 0x7f66b010f090 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 -- 172.21.15.32:0/1399189691 >> 172.21.15.32:0/1399189691 conn(0x7f66b0077a60 msgr2=0x7f66b0077e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 -- 172.21.15.32:0/1399189691 shutdown_connections 2024-04-16T05:34:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 -- 172.21.15.32:0/1399189691 wait complete. 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 Processor -- start 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.015+0000 7f66b74b8700 1 -- start start 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b74b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102530 0x7f66b019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b74b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b0102e50 0x7f66b019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b74b8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b010f5d0 0x7f66b019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b74b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01150f0 con 0x7f66b0102530 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b74b8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0114f70 con 0x7f66b0102e50 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b74b8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0115270 con 0x7f66b010f5d0 2024-04-16T05:34:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102530 0x7f66b019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102530 0x7f66b019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37554/0 (socket says 172.21.15.32:37554) 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 -- 172.21.15.32:0/1029881802 learned_addr learned my addr 172.21.15.32:0/1029881802 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b4a53700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b0102e50 0x7f66b019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5a55700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b010f5d0 0x7f66b019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 -- 172.21.15.32:0/1029881802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b010f5d0 msgr2=0x7f66b019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b010f5d0 0x7f66b019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 -- 172.21.15.32:0/1029881802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b0102e50 msgr2=0x7f66b019baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b0102e50 0x7f66b019baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 -- 172.21.15.32:0/1029881802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b01a0570 con 0x7f66b0102530 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.016+0000 7f66b5254700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102530 0x7f66b019b560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00d950 tx=0x7f66ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.017+0000 7f66a67fc700 1 -- 172.21.15.32:0/1029881802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac016070 con 0x7f66b0102530 2024-04-16T05:34:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.017+0000 7f66a67fc700 1 -- 172.21.15.32:0/1029881802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ac00fd70 con 0x7f66b0102530 2024-04-16T05:34:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.017+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b01a0860 con 0x7f66b0102530 2024-04-16T05:34:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.017+0000 7f66a67fc700 1 -- 172.21.15.32:0/1029881802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00e610 con 0x7f66b0102530 2024-04-16T05:34:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.017+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b01a80e0 con 0x7f66b0102530 2024-04-16T05:34:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.018+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b00697e0 con 0x7f66b0102530 2024-04-16T05:34:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.019+0000 7f66a67fc700 1 -- 172.21.15.32:0/1029881802 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f66ac019070 con 0x7f66b0102530 2024-04-16T05:34:10.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.019+0000 7f66a67fc700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f669c0a3cc0 0x7f669c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:10.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.019+0000 7f66b4a53700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f669c0a3cc0 0x7f669c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:10.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.020+0000 7f66a67fc700 1 -- 172.21.15.32:0/1029881802 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f66ac0d0170 con 0x7f66b0102530 2024-04-16T05:34:10.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.020+0000 7f66b4a53700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f669c0a3cc0 0x7f669c0a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f66a000b5c0 tx=0x7f66a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.024+0000 7f66a67fc700 1 -- 172.21.15.32:0/1029881802 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66ac09c7f0 con 0x7f66b0102530 2024-04-16T05:34:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.231+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66b019c8c0 con 0x7f669c0a3cc0 2024-04-16T05:34:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.241+0000 7f66a67fc700 1 -- 172.21.15.32:0/1029881802 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340920 (secure 0 0 0) 0x7f66b019c8c0 con 0x7f669c0a3cc0 2024-04-16T05:34:10.245 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:10.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f669c0a3cc0 msgr2=0x7f669c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f669c0a3cc0 0x7f669c0a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f66a000b5c0 tx=0x7f66a0005960 comp rx=0 tx=0).stop 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102530 msgr2=0x7f66b019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102530 0x7f66b019b560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00d950 tx=0x7f66ac00dc60 comp rx=0 tx=0).stop 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 shutdown_connections 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f669c0a3cc0 0x7f669c0a6180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b010f5d0 0x7f66b019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b0102530 0x7f66b019b560 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 --2- 172.21.15.32:0/1029881802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b0102e50 0x7f66b019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 >> 172.21.15.32:0/1029881802 conn(0x7f66b0077a60 msgr2=0x7f66b0110a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 shutdown_connections 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:10.244+0000 7f66b74b8700 1 -- 172.21.15.32:0/1029881802 wait complete. 2024-04-16T05:34:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-16T05:34:10.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:10 smithi082 ceph-mon[36195]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:34:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:10 smithi032 ceph-mon[32351]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:34:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:10 smithi134 ceph-mon[36153]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:34:10.979 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-04-16T05:34:08.938565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657300,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":595,"num_read_kb":421,"num_write":352,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361292,"kb_used_data":12140,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580532,"statfs":{"total":1151940427776,"available":1151570464768,"internally_reserved":0,"allocated":12431360,"data_stored":7957706,"data_compressed":360915,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":38,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003653"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948428+0000","last_change":"2024-04-16T05:32:34.759518+0000","last_active":"2024-04-16T05:34:02.948428+0000","last_peered":"2024-04-16T05:34:02.948428+0000","last_clean":"2024-04-16T05:34:02.948428+0000","last_became_active":"2024-04-16T05:32:34.759155+0000","last_became_peered":"2024-04-16T05:32:34.759155+0000","last_unstale":"2024-04-16T05:34:02.948428+0000","last_undegraded":"2024-04-16T05:34:02.948428+0000","last_fullsized":"2024-04-16T05:34:02.948428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:39.935886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935858+0000","last_change":"2024-04-16T05:32:34.758553+0000","last_active":"2024-04-16T05:34:02.935858+0000","last_peered":"2024-04-16T05:34:02.935858+0000","last_clean":"2024-04-16T05:34:02.935858+0000","last_became_active":"2024-04-16T05:32:34.758242+0000","last_became_peered":"2024-04-16T05:32:34.758242+0000","last_unstale":"2024-04-16T05:34:02.935858+0000","last_undegraded":"2024-04-16T05:34:02.935858+0000","last_fullsized":"2024-04-16T05:34:02.935858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:09.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.940029+0000","last_change":"2024-04-16T05:32:34.754815+0000","last_active":"2024-04-16T05:34:02.940029+0000","last_peered":"2024-04-16T05:34:02.940029+0000","last_clean":"2024-04-16T05:34:02.940029+0000","last_became_active":"2024-04-16T05:32:34.754488+0000","last_became_peered":"2024-04-16T05:32:34.754488+0000","last_unstale":"2024-04-16T05:34:02.940029+0000","last_undegraded":"2024-04-16T05:34:02.940029+0000","last_fullsized":"2024-04-16T05:34:02.940029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:10:32.379770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935653+0000","last_change":"2024-04-16T05:33:34.766742+0000","last_active":"2024-04-16T05:34:02.935653+0000","last_peered":"2024-04-16T05:34:02.935653+0000","last_clean":"2024-04-16T05:34:02.935653+0000","last_became_active":"2024-04-16T05:33:34.766332+0000","last_became_peered":"2024-04-16T05:33:34.766332+0000","last_unstale":"2024-04-16T05:34:02.935653+0000","last_undegraded":"2024-04-16T05:34:02.935653+0000","last_fullsized":"2024-04-16T05:34:02.935653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:52:14.644496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.929983+0000","last_change":"2024-04-16T05:32:34.755623+0000","last_active":"2024-04-16T05:34:02.929983+0000","last_peered":"2024-04-16T05:34:02.929983+0000","last_clean":"2024-04-16T05:34:02.929983+0000","last_became_active":"2024-04-16T05:32:34.754205+0000","last_became_peered":"2024-04-16T05:32:34.754205+0000","last_unstale":"2024-04-16T05:34:02.929983+0000","last_undegraded":"2024-04-16T05:34:02.929983+0000","last_fullsized":"2024-04-16T05:34:02.929983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:34:07.998734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.943737+0000","last_change":"2024-04-16T05:32:34.753767+0000","last_active":"2024-04-16T05:34:02.943737+0000","last_peered":"2024-04-16T05:34:02.943737+0000","last_clean":"2024-04-16T05:34:02.943737+0000","last_became_active":"2024-04-16T05:32:34.753448+0000","last_became_peered":"2024-04-16T05:32:34.753448+0000","last_unstale":"2024-04-16T05:34:02.943737+0000","last_undegraded":"2024-04-16T05:34:02.943737+0000","last_fullsized":"2024-04-16T05:34:02.943737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:27:42.798615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938967+0000","last_change":"2024-04-16T05:32:34.754049+0000","last_active":"2024-04-16T05:34:02.938967+0000","last_peered":"2024-04-16T05:34:02.938967+0000","last_clean":"2024-04-16T05:34:02.938967+0000","last_became_active":"2024-04-16T05:32:34.753757+0000","last_became_peered":"2024-04-16T05:32:34.753757+0000","last_unstale":"2024-04-16T05:34:02.938967+0000","last_undegraded":"2024-04-16T05:34:02.938967+0000","last_fullsized":"2024-04-16T05:34:02.938967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:42:26.747100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.934758+0000","last_change":"2024-04-16T05:32:34.762950+0000","last_active":"2024-04-16T05:34:02.934758+0000","last_peered":"2024-04-16T05:34:02.934758+0000","last_clean":"2024-04-16T05:34:02.934758+0000","last_became_active":"2024-04-16T05:32:34.762073+0000","last_became_peered":"2024-04-16T05:32:34.762073+0000","last_unstale":"2024-04-16T05:34:02.934758+0000","last_undegraded":"2024-04-16T05:34:02.934758+0000","last_fullsized":"2024-04-16T05:34:02.934758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:57:26.623655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.947968+0000","last_change":"2024-04-16T05:32:34.751027+0000","last_active":"2024-04-16T05:34:02.947968+0000","last_peered":"2024-04-16T05:34:02.947968+0000","last_clean":"2024-04-16T05:34:02.947968+0000","last_became_active":"2024-04-16T05:32:34.750697+0000","last_became_peered":"2024-04-16T05:32:34.750697+0000","last_unstale":"2024-04-16T05:34:02.947968+0000","last_undegraded":"2024-04-16T05:34:02.947968+0000","last_fullsized":"2024-04-16T05:34:02.947968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:39:33.834972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.943773+0000","last_change":"2024-04-16T05:32:34.752293+0000","last_active":"2024-04-16T05:34:02.943773+0000","last_peered":"2024-04-16T05:34:02.943773+0000","last_clean":"2024-04-16T05:34:02.943773+0000","last_became_active":"2024-04-16T05:32:34.752008+0000","last_became_peered":"2024-04-16T05:32:34.752008+0000","last_unstale":"2024-04-16T05:34:02.943773+0000","last_undegraded":"2024-04-16T05:34:02.943773+0000","last_fullsized":"2024-04-16T05:34:02.943773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:02:17.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.941247+0000","last_change":"2024-04-16T05:32:34.760961+0000","last_active":"2024-04-16T05:34:02.941247+0000","last_peered":"2024-04-16T05:34:02.941247+0000","last_clean":"2024-04-16T05:34:02.941247+0000","last_became_active":"2024-04-16T05:32:34.760640+0000","last_became_peered":"2024-04-16T05:32:34.760640+0000","last_unstale":"2024-04-16T05:34:02.941247+0000","last_undegraded":"2024-04-16T05:34:02.941247+0000","last_fullsized":"2024-04-16T05:34:02.941247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:47:28.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938913+0000","last_change":"2024-04-16T05:32:34.748646+0000","last_active":"2024-04-16T05:34:02.938913+0000","last_peered":"2024-04-16T05:34:02.938913+0000","last_clean":"2024-04-16T05:34:02.938913+0000","last_became_active":"2024-04-16T05:32:34.748090+0000","last_became_peered":"2024-04-16T05:32:34.748090+0000","last_unstale":"2024-04-16T05:34:02.938913+0000","last_undegraded":"2024-04-16T05:34:02.938913+0000","last_fullsized":"2024-04-16T05:34:02.938913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:00:19.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.933168+0000","last_change":"2024-04-16T05:32:34.747025+0000","last_active":"2024-04-16T05:34:02.933168+0000","last_peered":"2024-04-16T05:34:02.933168+0000","last_clean":"2024-04-16T05:34:02.933168+0000","last_became_active":"2024-04-16T05:32:34.746711+0000","last_became_peered":"2024-04-16T05:32:34.746711+0000","last_unstale":"2024-04-16T05:34:02.933168+0000","last_undegraded":"2024-04-16T05:34:02.933168+0000","last_fullsized":"2024-04-16T05:34:02.933168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:40:30.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.928188+0000","last_change":"2024-04-16T05:32:34.756756+0000","last_active":"2024-04-16T05:34:02.928188+0000","last_peered":"2024-04-16T05:34:02.928188+0000","last_clean":"2024-04-16T05:34:02.928188+0000","last_became_active":"2024-04-16T05:32:34.756442+0000","last_became_peered":"2024-04-16T05:32:34.756442+0000","last_unstale":"2024-04-16T05:34:02.928188+0000","last_undegraded":"2024-04-16T05:34:02.928188+0000","last_fullsized":"2024-04-16T05:34:02.928188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:33.864944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.940021+0000","last_change":"2024-04-16T05:32:34.765443+0000","last_active":"2024-04-16T05:34:02.940021+0000","last_peered":"2024-04-16T05:34:02.940021+0000","last_clean":"2024-04-16T05:34:02.940021+0000","last_became_active":"2024-04-16T05:32:34.765133+0000","last_became_peered":"2024-04-16T05:32:34.765133+0000","last_unstale":"2024-04-16T05:34:02.940021+0000","last_undegraded":"2024-04-16T05:34:02.940021+0000","last_fullsized":"2024-04-16T05:34:02.940021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:31.452184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.941676+0000","last_change":"2024-04-16T05:32:34.757577+0000","last_active":"2024-04-16T05:34:02.941676+0000","last_peered":"2024-04-16T05:34:02.941676+0000","last_clean":"2024-04-16T05:34:02.941676+0000","last_became_active":"2024-04-16T05:32:34.753451+0000","last_became_peered":"2024-04-16T05:32:34.753451+0000","last_unstale":"2024-04-16T05:34:02.941676+0000","last_undegraded":"2024-04-16T05:34:02.941676+0000","last_fullsized":"2024-04-16T05:34:02.941676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:56.617824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948568+0000","last_change":"2024-04-16T05:32:34.759776+0000","last_active":"2024-04-16T05:34:02.948568+0000","last_peered":"2024-04-16T05:34:02.948568+0000","last_clean":"2024-04-16T05:34:02.948568+0000","last_became_active":"2024-04-16T05:32:34.759371+0000","last_became_peered":"2024-04-16T05:32:34.759371+0000","last_unstale":"2024-04-16T05:34:02.948568+0000","last_undegraded":"2024-04-16T05:34:02.948568+0000","last_fullsized":"2024-04-16T05:34:02.948568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:36.458515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.929006+0000","last_change":"2024-04-16T05:32:34.747803+0000","last_active":"2024-04-16T05:34:02.929006+0000","last_peered":"2024-04-16T05:34:02.929006+0000","last_clean":"2024-04-16T05:34:02.929006+0000","last_became_active":"2024-04-16T05:32:34.747472+0000","last_became_peered":"2024-04-16T05:32:34.747472+0000","last_unstale":"2024-04-16T05:34:02.929006+0000","last_undegraded":"2024-04-16T05:34:02.929006+0000","last_fullsized":"2024-04-16T05:34:02.929006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:41.118574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.937007+0000","last_change":"2024-04-16T05:32:34.754151+0000","last_active":"2024-04-16T05:34:02.937007+0000","last_peered":"2024-04-16T05:34:02.937007+0000","last_clean":"2024-04-16T05:34:02.937007+0000","last_became_active":"2024-04-16T05:32:34.753814+0000","last_became_peered":"2024-04-16T05:32:34.753814+0000","last_unstale":"2024-04-16T05:34:02.937007+0000","last_undegraded":"2024-04-16T05:34:02.937007+0000","last_fullsized":"2024-04-16T05:34:02.937007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:13:00.207978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762389+0000","last_change":"2024-04-16T05:32:38.938563+0000","last_active":"2024-04-16T05:33:34.762389+0000","last_peered":"2024-04-16T05:33:34.762389+0000","last_clean":"2024-04-16T05:33:34.762389+0000","last_became_active":"2024-04-16T05:32:38.938097+0000","last_became_peered":"2024-04-16T05:32:38.938097+0000","last_unstale":"2024-04-16T05:33:34.762389+0000","last_undegraded":"2024-04-16T05:33:34.762389+0000","last_fullsized":"2024-04-16T05:33:34.762389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:47:52.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761063+0000","last_change":"2024-04-16T05:32:33.735694+0000","last_active":"2024-04-16T05:33:34.761063+0000","last_peered":"2024-04-16T05:33:34.761063+0000","last_clean":"2024-04-16T05:33:34.761063+0000","last_became_active":"2024-04-16T05:32:33.735414+0000","last_became_peered":"2024-04-16T05:32:33.735414+0000","last_unstale":"2024-04-16T05:33:34.761063+0000","last_undegraded":"2024-04-16T05:33:34.761063+0000","last_fullsized":"2024-04-16T05:33:34.761063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:19:06.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.933441+0000","last_change":"2024-04-16T05:32:34.762315+0000","last_active":"2024-04-16T05:34:02.933441+0000","last_peered":"2024-04-16T05:34:02.933441+0000","last_clean":"2024-04-16T05:34:02.933441+0000","last_became_active":"2024-04-16T05:32:34.761969+0000","last_became_peered":"2024-04-16T05:32:34.761969+0000","last_unstale":"2024-04-16T05:34:02.933441+0000","last_undegraded":"2024-04-16T05:34:02.933441+0000","last_fullsized":"2024-04-16T05:34:02.933441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:00:45.166966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763444+0000","last_change":"2024-04-16T05:32:38.940681+0000","last_active":"2024-04-16T05:33:34.763444+0000","last_peered":"2024-04-16T05:33:34.763444+0000","last_clean":"2024-04-16T05:33:34.763444+0000","last_became_active":"2024-04-16T05:32:38.940414+0000","last_became_peered":"2024-04-16T05:32:38.940414+0000","last_unstale":"2024-04-16T05:33:34.763444+0000","last_undegraded":"2024-04-16T05:33:34.763444+0000","last_fullsized":"2024-04-16T05:33:34.763444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:46:50.061607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.771469+0000","last_change":"2024-04-16T05:32:33.748333+0000","last_active":"2024-04-16T05:33:34.771469+0000","last_peered":"2024-04-16T05:33:34.771469+0000","last_clean":"2024-04-16T05:33:34.771469+0000","last_became_active":"2024-04-16T05:32:33.748022+0000","last_became_peered":"2024-04-16T05:32:33.748022+0000","last_unstale":"2024-04-16T05:33:34.771469+0000","last_undegraded":"2024-04-16T05:33:34.771469+0000","last_fullsized":"2024-04-16T05:33:34.771469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:50:27.771245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.939375+0000","last_change":"2024-04-16T05:32:34.764832+0000","last_active":"2024-04-16T05:34:02.939375+0000","last_peered":"2024-04-16T05:34:02.939375+0000","last_clean":"2024-04-16T05:34:02.939375+0000","last_became_active":"2024-04-16T05:32:34.763180+0000","last_became_peered":"2024-04-16T05:32:34.763180+0000","last_unstale":"2024-04-16T05:34:02.939375+0000","last_undegraded":"2024-04-16T05:34:02.939375+0000","last_fullsized":"2024-04-16T05:34:02.939375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:37:27.804376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763690+0000","last_change":"2024-04-16T05:32:38.937226+0000","last_active":"2024-04-16T05:33:34.763690+0000","last_peered":"2024-04-16T05:33:34.763690+0000","last_clean":"2024-04-16T05:33:34.763690+0000","last_became_active":"2024-04-16T05:32:38.936770+0000","last_became_peered":"2024-04-16T05:32:38.936770+0000","last_unstale":"2024-04-16T05:33:34.763690+0000","last_undegraded":"2024-04-16T05:33:34.763690+0000","last_fullsized":"2024-04-16T05:33:34.763690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:05:28.059175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.776201+0000","last_change":"2024-04-16T05:33:34.775689+0000","last_active":"2024-04-16T05:33:34.776201+0000","last_peered":"2024-04-16T05:33:34.776201+0000","last_clean":"2024-04-16T05:33:34.776201+0000","last_became_active":"2024-04-16T05:33:34.770520+0000","last_became_peered":"2024-04-16T05:33:34.770520+0000","last_unstale":"2024-04-16T05:33:34.776201+0000","last_undegraded":"2024-04-16T05:33:34.776201+0000","last_fullsized":"2024-04-16T05:33:34.776201+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:40:57.616795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948895+0000","last_change":"2024-04-16T05:32:34.758891+0000","last_active":"2024-04-16T05:34:02.948895+0000","last_peered":"2024-04-16T05:34:02.948895+0000","last_clean":"2024-04-16T05:34:02.948895+0000","last_became_active":"2024-04-16T05:32:34.758559+0000","last_became_peered":"2024-04-16T05:32:34.758559+0000","last_unstale":"2024-04-16T05:34:02.948895+0000","last_undegraded":"2024-04-16T05:34:02.948895+0000","last_fullsized":"2024-04-16T05:34:02.948895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:31.213992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763130+0000","last_change":"2024-04-16T05:32:38.937163+0000","last_active":"2024-04-16T05:33:34.763130+0000","last_peered":"2024-04-16T05:33:34.763130+0000","last_clean":"2024-04-16T05:33:34.763130+0000","last_became_active":"2024-04-16T05:32:38.936745+0000","last_became_peered":"2024-04-16T05:32:38.936745+0000","last_unstale":"2024-04-16T05:33:34.763130+0000","last_undegraded":"2024-04-16T05:33:34.763130+0000","last_fullsized":"2024-04-16T05:33:34.763130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:54:37.915041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763843+0000","last_change":"2024-04-16T05:32:33.731842+0000","last_active":"2024-04-16T05:33:34.763843+0000","last_peered":"2024-04-16T05:33:34.763843+0000","last_clean":"2024-04-16T05:33:34.763843+0000","last_became_active":"2024-04-16T05:32:33.731417+0000","last_became_peered":"2024-04-16T05:32:33.731417+0000","last_unstale":"2024-04-16T05:33:34.763843+0000","last_undegraded":"2024-04-16T05:33:34.763843+0000","last_fullsized":"2024-04-16T05:33:34.763843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:04:03.141533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.933420+0000","last_change":"2024-04-16T05:32:34.755062+0000","last_active":"2024-04-16T05:34:02.933420+0000","last_peered":"2024-04-16T05:34:02.933420+0000","last_clean":"2024-04-16T05:34:02.933420+0000","last_became_active":"2024-04-16T05:32:34.754759+0000","last_became_peered":"2024-04-16T05:32:34.754759+0000","last_unstale":"2024-04-16T05:34:02.933420+0000","last_undegraded":"2024-04-16T05:34:02.933420+0000","last_fullsized":"2024-04-16T05:34:02.933420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:53:59.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762985+0000","last_change":"2024-04-16T05:32:38.935646+0000","last_active":"2024-04-16T05:33:34.762985+0000","last_peered":"2024-04-16T05:33:34.762985+0000","last_clean":"2024-04-16T05:33:34.762985+0000","last_became_active":"2024-04-16T05:32:38.935206+0000","last_became_peered":"2024-04-16T05:32:38.935206+0000","last_unstale":"2024-04-16T05:33:34.762985+0000","last_undegraded":"2024-04-16T05:33:34.762985+0000","last_fullsized":"2024-04-16T05:33:34.762985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:09.772136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:05.798298+0000","last_change":"2024-04-16T05:32:33.739343+0000","last_active":"2024-04-16T05:34:05.798298+0000","last_peered":"2024-04-16T05:34:05.798298+0000","last_clean":"2024-04-16T05:34:05.798298+0000","last_became_active":"2024-04-16T05:32:33.738982+0000","last_became_peered":"2024-04-16T05:32:33.738982+0000","last_unstale":"2024-04-16T05:34:05.798298+0000","last_undegraded":"2024-04-16T05:34:05.798298+0000","last_fullsized":"2024-04-16T05:34:05.798298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:36.002804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.924212+0000","last_change":"2024-04-16T05:32:34.749316+0000","last_active":"2024-04-16T05:34:02.924212+0000","last_peered":"2024-04-16T05:34:02.924212+0000","last_clean":"2024-04-16T05:34:02.924212+0000","last_became_active":"2024-04-16T05:32:34.748976+0000","last_became_peered":"2024-04-16T05:32:34.748976+0000","last_unstale":"2024-04-16T05:34:02.924212+0000","last_undegraded":"2024-04-16T05:34:02.924212+0000","last_fullsized":"2024-04-16T05:34:02.924212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:16:29.569402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762740+0000","last_change":"2024-04-16T05:32:38.938633+0000","last_active":"2024-04-16T05:33:34.762740+0000","last_peered":"2024-04-16T05:33:34.762740+0000","last_clean":"2024-04-16T05:33:34.762740+0000","last_became_active":"2024-04-16T05:32:38.938215+0000","last_became_peered":"2024-04-16T05:32:38.938215+0000","last_unstale":"2024-04-16T05:33:34.762740+0000","last_undegraded":"2024-04-16T05:33:34.762740+0000","last_fullsized":"2024-04-16T05:33:34.762740+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:05.078109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.768181+0000","last_change":"2024-04-16T05:32:33.744262+0000","last_active":"2024-04-16T05:33:34.768181+0000","last_peered":"2024-04-16T05:33:34.768181+0000","last_clean":"2024-04-16T05:33:34.768181+0000","last_became_active":"2024-04-16T05:32:33.743866+0000","last_became_peered":"2024-04-16T05:32:33.743866+0000","last_unstale":"2024-04-16T05:33:34.768181+0000","last_undegraded":"2024-04-16T05:33:34.768181+0000","last_fullsized":"2024-04-16T05:33:34.768181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:22:03.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.934939+0000","last_change":"2024-04-16T05:32:34.750201+0000","last_active":"2024-04-16T05:34:02.934939+0000","last_peered":"2024-04-16T05:34:02.934939+0000","last_clean":"2024-04-16T05:34:02.934939+0000","last_became_active":"2024-04-16T05:32:34.749900+0000","last_became_peered":"2024-04-16T05:32:34.749900+0000","last_unstale":"2024-04-16T05:34:02.934939+0000","last_undegraded":"2024-04-16T05:34:02.934939+0000","last_fullsized":"2024-04-16T05:34:02.934939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:42.708584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.767259+0000","last_change":"2024-04-16T05:32:38.931277+0000","last_active":"2024-04-16T05:33:34.767259+0000","last_peered":"2024-04-16T05:33:34.767259+0000","last_clean":"2024-04-16T05:33:34.767259+0000","last_became_active":"2024-04-16T05:32:38.930888+0000","last_became_peered":"2024-04-16T05:32:38.930888+0000","last_unstale":"2024-04-16T05:33:34.767259+0000","last_undegraded":"2024-04-16T05:33:34.767259+0000","last_fullsized":"2024-04-16T05:33:34.767259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:27:31.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763925+0000","last_change":"2024-04-16T05:32:33.747740+0000","last_active":"2024-04-16T05:33:34.763925+0000","last_peered":"2024-04-16T05:33:34.763925+0000","last_clean":"2024-04-16T05:33:34.763925+0000","last_became_active":"2024-04-16T05:32:33.747313+0000","last_became_peered":"2024-04-16T05:32:33.747313+0000","last_unstale":"2024-04-16T05:33:34.763925+0000","last_undegraded":"2024-04-16T05:33:34.763925+0000","last_fullsized":"2024-04-16T05:33:34.763925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:05:03.937790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935630+0000","last_change":"2024-04-16T05:32:34.760075+0000","last_active":"2024-04-16T05:34:02.935630+0000","last_peered":"2024-04-16T05:34:02.935630+0000","last_clean":"2024-04-16T05:34:02.935630+0000","last_became_active":"2024-04-16T05:32:34.759569+0000","last_became_peered":"2024-04-16T05:32:34.759569+0000","last_unstale":"2024-04-16T05:34:02.935630+0000","last_undegraded":"2024-04-16T05:34:02.935630+0000","last_fullsized":"2024-04-16T05:34:02.935630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:54.222877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764355+0000","last_change":"2024-04-16T05:32:38.928568+0000","last_active":"2024-04-16T05:33:34.764355+0000","last_peered":"2024-04-16T05:33:34.764355+0000","last_clean":"2024-04-16T05:33:34.764355+0000","last_became_active":"2024-04-16T05:32:38.928109+0000","last_became_peered":"2024-04-16T05:32:38.928109+0000","last_unstale":"2024-04-16T05:33:34.764355+0000","last_undegraded":"2024-04-16T05:33:34.764355+0000","last_fullsized":"2024-04-16T05:33:34.764355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:24:14.293528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.843483+0000","last_change":"2024-04-16T05:32:33.737803+0000","last_active":"2024-04-16T05:33:34.843483+0000","last_peered":"2024-04-16T05:33:34.843483+0000","last_clean":"2024-04-16T05:33:34.843483+0000","last_became_active":"2024-04-16T05:32:33.737448+0000","last_became_peered":"2024-04-16T05:32:33.737448+0000","last_unstale":"2024-04-16T05:33:34.843483+0000","last_undegraded":"2024-04-16T05:33:34.843483+0000","last_fullsized":"2024-04-16T05:33:34.843483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:44:54.302323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.934113+0000","last_change":"2024-04-16T05:32:34.754070+0000","last_active":"2024-04-16T05:34:02.934113+0000","last_peered":"2024-04-16T05:34:02.934113+0000","last_clean":"2024-04-16T05:34:02.934113+0000","last_became_active":"2024-04-16T05:32:34.753717+0000","last_became_peered":"2024-04-16T05:32:34.753717+0000","last_unstale":"2024-04-16T05:34:02.934113+0000","last_undegraded":"2024-04-16T05:34:02.934113+0000","last_fullsized":"2024-04-16T05:34:02.934113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:58:56.738245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769558+0000","last_change":"2024-04-16T05:32:38.931594+0000","last_active":"2024-04-16T05:33:34.769558+0000","last_peered":"2024-04-16T05:33:34.769558+0000","last_clean":"2024-04-16T05:33:34.769558+0000","last_became_active":"2024-04-16T05:32:38.931188+0000","last_became_peered":"2024-04-16T05:32:38.931188+0000","last_unstale":"2024-04-16T05:33:34.769558+0000","last_undegraded":"2024-04-16T05:33:34.769558+0000","last_fullsized":"2024-04-16T05:33:34.769558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:55.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:04.796234+0000","last_change":"2024-04-16T05:32:33.739752+0000","last_active":"2024-04-16T05:34:04.796234+0000","last_peered":"2024-04-16T05:34:04.796234+0000","last_clean":"2024-04-16T05:34:04.796234+0000","last_became_active":"2024-04-16T05:32:33.739444+0000","last_became_peered":"2024-04-16T05:32:33.739444+0000","last_unstale":"2024-04-16T05:34:04.796234+0000","last_undegraded":"2024-04-16T05:34:04.796234+0000","last_fullsized":"2024-04-16T05:34:04.796234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:04:18.091618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948687+0000","last_change":"2024-04-16T05:32:34.759197+0000","last_active":"2024-04-16T05:34:02.948687+0000","last_peered":"2024-04-16T05:34:02.948687+0000","last_clean":"2024-04-16T05:34:02.948687+0000","last_became_active":"2024-04-16T05:32:34.758783+0000","last_became_peered":"2024-04-16T05:32:34.758783+0000","last_unstale":"2024-04-16T05:34:02.948687+0000","last_undegraded":"2024-04-16T05:34:02.948687+0000","last_fullsized":"2024-04-16T05:34:02.948687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:23:37.020254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.760284+0000","last_change":"2024-04-16T05:32:38.936198+0000","last_active":"2024-04-16T05:33:34.760284+0000","last_peered":"2024-04-16T05:33:34.760284+0000","last_clean":"2024-04-16T05:33:34.760284+0000","last_became_active":"2024-04-16T05:32:38.935698+0000","last_became_peered":"2024-04-16T05:32:38.935698+0000","last_unstale":"2024-04-16T05:33:34.760284+0000","last_undegraded":"2024-04-16T05:33:34.760284+0000","last_fullsized":"2024-04-16T05:33:34.760284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:17:35.602408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769134+0000","last_change":"2024-04-16T05:32:33.737653+0000","last_active":"2024-04-16T05:33:34.769134+0000","last_peered":"2024-04-16T05:33:34.769134+0000","last_clean":"2024-04-16T05:33:34.769134+0000","last_became_active":"2024-04-16T05:32:33.737342+0000","last_became_peered":"2024-04-16T05:32:33.737342+0000","last_unstale":"2024-04-16T05:33:34.769134+0000","last_undegraded":"2024-04-16T05:33:34.769134+0000","last_fullsized":"2024-04-16T05:33:34.769134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:17:35.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938193+0000","last_change":"2024-04-16T05:32:34.765722+0000","last_active":"2024-04-16T05:34:02.938193+0000","last_peered":"2024-04-16T05:34:02.938193+0000","last_clean":"2024-04-16T05:34:02.938193+0000","last_became_active":"2024-04-16T05:32:34.765413+0000","last_became_peered":"2024-04-16T05:32:34.765413+0000","last_unstale":"2024-04-16T05:34:02.938193+0000","last_undegraded":"2024-04-16T05:34:02.938193+0000","last_fullsized":"2024-04-16T05:34:02.938193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:03.884041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764121+0000","last_change":"2024-04-16T05:32:38.936186+0000","last_active":"2024-04-16T05:33:34.764121+0000","last_peered":"2024-04-16T05:33:34.764121+0000","last_clean":"2024-04-16T05:33:34.764121+0000","last_became_active":"2024-04-16T05:32:38.935738+0000","last_became_peered":"2024-04-16T05:32:38.935738+0000","last_unstale":"2024-04-16T05:33:34.764121+0000","last_undegraded":"2024-04-16T05:33:34.764121+0000","last_fullsized":"2024-04-16T05:33:34.764121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:10.519095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.844163+0000","last_change":"2024-04-16T05:32:33.742991+0000","last_active":"2024-04-16T05:33:34.844163+0000","last_peered":"2024-04-16T05:33:34.844163+0000","last_clean":"2024-04-16T05:33:34.844163+0000","last_became_active":"2024-04-16T05:32:33.742659+0000","last_became_peered":"2024-04-16T05:32:33.742659+0000","last_unstale":"2024-04-16T05:33:34.844163+0000","last_undegraded":"2024-04-16T05:33:34.844163+0000","last_fullsized":"2024-04-16T05:33:34.844163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:23.756123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948969+0000","last_change":"2024-04-16T05:32:34.759926+0000","last_active":"2024-04-16T05:34:02.948969+0000","last_peered":"2024-04-16T05:34:02.948969+0000","last_clean":"2024-04-16T05:34:02.948969+0000","last_became_active":"2024-04-16T05:32:34.759626+0000","last_became_peered":"2024-04-16T05:32:34.759626+0000","last_unstale":"2024-04-16T05:34:02.948969+0000","last_undegraded":"2024-04-16T05:34:02.948969+0000","last_fullsized":"2024-04-16T05:34:02.948969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:50.314638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.760612+0000","last_change":"2024-04-16T05:32:38.942135+0000","last_active":"2024-04-16T05:33:34.760612+0000","last_peered":"2024-04-16T05:33:34.760612+0000","last_clean":"2024-04-16T05:33:34.760612+0000","last_became_active":"2024-04-16T05:32:38.941534+0000","last_became_peered":"2024-04-16T05:32:38.941534+0000","last_unstale":"2024-04-16T05:33:34.760612+0000","last_undegraded":"2024-04-16T05:33:34.760612+0000","last_fullsized":"2024-04-16T05:33:34.760612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:20.494516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":211,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:05.797815+0000","last_change":"2024-04-16T05:32:33.734275+0000","last_active":"2024-04-16T05:34:05.797815+0000","last_peered":"2024-04-16T05:34:05.797815+0000","last_clean":"2024-04-16T05:34:05.797815+0000","last_became_active":"2024-04-16T05:32:33.733906+0000","last_became_peered":"2024-04-16T05:32:33.733906+0000","last_unstale":"2024-04-16T05:34:05.797815+0000","last_undegraded":"2024-04-16T05:34:05.797815+0000","last_fullsized":"2024-04-16T05:34:05.797815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:39:23.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":150,"num_read_kb":136,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.930603+0000","last_change":"2024-04-16T05:32:34.750179+0000","last_active":"2024-04-16T05:34:02.930603+0000","last_peered":"2024-04-16T05:34:02.930603+0000","last_clean":"2024-04-16T05:34:02.930603+0000","last_became_active":"2024-04-16T05:32:34.749773+0000","last_became_peered":"2024-04-16T05:32:34.749773+0000","last_unstale":"2024-04-16T05:34:02.930603+0000","last_undegraded":"2024-04-16T05:34:02.930603+0000","last_fullsized":"2024-04-16T05:34:02.930603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:37.621191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.844475+0000","last_change":"2024-04-16T05:32:38.935690+0000","last_active":"2024-04-16T05:33:34.844475+0000","last_peered":"2024-04-16T05:33:34.844475+0000","last_clean":"2024-04-16T05:33:34.844475+0000","last_became_active":"2024-04-16T05:32:38.935282+0000","last_became_peered":"2024-04-16T05:32:38.935282+0000","last_unstale":"2024-04-16T05:33:34.844475+0000","last_undegraded":"2024-04-16T05:33:34.844475+0000","last_fullsized":"2024-04-16T05:33:34.844475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:25:00.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.782574+0000","last_change":"2024-04-16T05:33:34.782195+0000","last_active":"2024-04-16T05:33:34.782574+0000","last_peered":"2024-04-16T05:33:34.782574+0000","last_clean":"2024-04-16T05:33:34.782574+0000","last_became_active":"2024-04-16T05:33:34.771490+0000","last_became_peered":"2024-04-16T05:33:34.771490+0000","last_unstale":"2024-04-16T05:33:34.782574+0000","last_undegraded":"2024-04-16T05:33:34.782574+0000","last_fullsized":"2024-04-16T05:33:34.782574+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:52.513976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.941038+0000","last_change":"2024-04-16T05:32:34.748760+0000","last_active":"2024-04-16T05:34:02.941038+0000","last_peered":"2024-04-16T05:34:02.941038+0000","last_clean":"2024-04-16T05:34:02.941038+0000","last_became_active":"2024-04-16T05:32:34.748432+0000","last_became_peered":"2024-04-16T05:32:34.748432+0000","last_unstale":"2024-04-16T05:34:02.941038+0000","last_undegraded":"2024-04-16T05:34:02.941038+0000","last_fullsized":"2024-04-16T05:34:02.941038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:12:42.859342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.765741+0000","last_change":"2024-04-16T05:32:38.929705+0000","last_active":"2024-04-16T05:33:34.765741+0000","last_peered":"2024-04-16T05:33:34.765741+0000","last_clean":"2024-04-16T05:33:34.765741+0000","last_became_active":"2024-04-16T05:32:38.929282+0000","last_became_peered":"2024-04-16T05:32:38.929282+0000","last_unstale":"2024-04-16T05:33:34.765741+0000","last_undegraded":"2024-04-16T05:33:34.765741+0000","last_fullsized":"2024-04-16T05:33:34.765741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:51:27.763600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.843796+0000","last_change":"2024-04-16T05:32:33.746371+0000","last_active":"2024-04-16T05:33:34.843796+0000","last_peered":"2024-04-16T05:33:34.843796+0000","last_clean":"2024-04-16T05:33:34.843796+0000","last_became_active":"2024-04-16T05:32:33.746077+0000","last_became_peered":"2024-04-16T05:32:33.746077+0000","last_unstale":"2024-04-16T05:33:34.843796+0000","last_undegraded":"2024-04-16T05:33:34.843796+0000","last_fullsized":"2024-04-16T05:33:34.843796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:57:43.193942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.944369+0000","last_change":"2024-04-16T05:32:34.753196+0000","last_active":"2024-04-16T05:34:02.944369+0000","last_peered":"2024-04-16T05:34:02.944369+0000","last_clean":"2024-04-16T05:34:02.944369+0000","last_became_active":"2024-04-16T05:32:34.752881+0000","last_became_peered":"2024-04-16T05:32:34.752881+0000","last_unstale":"2024-04-16T05:34:02.944369+0000","last_undegraded":"2024-04-16T05:34:02.944369+0000","last_fullsized":"2024-04-16T05:34:02.944369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:35:56.715394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.773192+0000","last_change":"2024-04-16T05:33:34.772562+0000","last_active":"2024-04-16T05:33:34.773192+0000","last_peered":"2024-04-16T05:33:34.773192+0000","last_clean":"2024-04-16T05:33:34.773192+0000","last_became_active":"2024-04-16T05:33:34.772052+0000","last_became_peered":"2024-04-16T05:33:34.772052+0000","last_unstale":"2024-04-16T05:33:34.773192+0000","last_undegraded":"2024-04-16T05:33:34.773192+0000","last_fullsized":"2024-04-16T05:33:34.773192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:50:18.425870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:04.797170+0000","last_change":"2024-04-16T05:32:33.735921+0000","last_active":"2024-04-16T05:34:04.797170+0000","last_peered":"2024-04-16T05:34:04.797170+0000","last_clean":"2024-04-16T05:34:04.797170+0000","last_became_active":"2024-04-16T05:32:33.735572+0000","last_became_peered":"2024-04-16T05:32:33.735572+0000","last_unstale":"2024-04-16T05:34:04.797170+0000","last_undegraded":"2024-04-16T05:34:04.797170+0000","last_fullsized":"2024-04-16T05:34:04.797170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:46:56.512827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.932132+0000","last_change":"2024-04-16T05:32:34.762172+0000","last_active":"2024-04-16T05:34:02.932132+0000","last_peered":"2024-04-16T05:34:02.932132+0000","last_clean":"2024-04-16T05:34:02.932132+0000","last_became_active":"2024-04-16T05:32:34.761914+0000","last_became_peered":"2024-04-16T05:32:34.761914+0000","last_unstale":"2024-04-16T05:34:02.932132+0000","last_undegraded":"2024-04-16T05:34:02.932132+0000","last_fullsized":"2024-04-16T05:34:02.932132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:47:52.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.772546+0000","last_change":"2024-04-16T05:33:34.772546+0000","last_active":"2024-04-16T05:33:34.772546+0000","last_peered":"2024-04-16T05:33:34.772546+0000","last_clean":"2024-04-16T05:33:34.772546+0000","last_became_active":"2024-04-16T05:33:34.772100+0000","last_became_peered":"2024-04-16T05:33:34.772100+0000","last_unstale":"2024-04-16T05:33:34.772546+0000","last_undegraded":"2024-04-16T05:33:34.772546+0000","last_fullsized":"2024-04-16T05:33:34.772546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:11:50.157129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763134+0000","last_change":"2024-04-16T05:32:33.743944+0000","last_active":"2024-04-16T05:33:34.763134+0000","last_peered":"2024-04-16T05:33:34.763134+0000","last_clean":"2024-04-16T05:33:34.763134+0000","last_became_active":"2024-04-16T05:32:33.743559+0000","last_became_peered":"2024-04-16T05:32:33.743559+0000","last_unstale":"2024-04-16T05:33:34.763134+0000","last_undegraded":"2024-04-16T05:33:34.763134+0000","last_fullsized":"2024-04-16T05:33:34.763134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:08:48.889356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.931912+0000","last_change":"2024-04-16T05:32:34.756741+0000","last_active":"2024-04-16T05:34:02.931912+0000","last_peered":"2024-04-16T05:34:02.931912+0000","last_clean":"2024-04-16T05:34:02.931912+0000","last_became_active":"2024-04-16T05:32:34.756426+0000","last_became_peered":"2024-04-16T05:32:34.756426+0000","last_unstale":"2024-04-16T05:34:02.931912+0000","last_undegraded":"2024-04-16T05:34:02.931912+0000","last_fullsized":"2024-04-16T05:34:02.931912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:09:30.493857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.843530+0000","last_change":"2024-04-16T05:32:38.934818+0000","last_active":"2024-04-16T05:33:34.843530+0000","last_peered":"2024-04-16T05:33:34.843530+0000","last_clean":"2024-04-16T05:33:34.843530+0000","last_became_active":"2024-04-16T05:32:38.934335+0000","last_became_peered":"2024-04-16T05:32:38.934335+0000","last_unstale":"2024-04-16T05:33:34.843530+0000","last_undegraded":"2024-04-16T05:33:34.843530+0000","last_fullsized":"2024-04-16T05:33:34.843530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:02:41.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764985+0000","last_change":"2024-04-16T05:32:33.731951+0000","last_active":"2024-04-16T05:33:34.764985+0000","last_peered":"2024-04-16T05:33:34.764985+0000","last_clean":"2024-04-16T05:33:34.764985+0000","last_became_active":"2024-04-16T05:32:33.731649+0000","last_became_peered":"2024-04-16T05:32:33.731649+0000","last_unstale":"2024-04-16T05:33:34.764985+0000","last_undegraded":"2024-04-16T05:33:34.764985+0000","last_fullsized":"2024-04-16T05:33:34.764985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:41.258012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.931179+0000","last_change":"2024-04-16T05:32:34.751884+0000","last_active":"2024-04-16T05:34:02.931179+0000","last_peered":"2024-04-16T05:34:02.931179+0000","last_clean":"2024-04-16T05:34:02.931179+0000","last_became_active":"2024-04-16T05:32:34.751559+0000","last_became_peered":"2024-04-16T05:32:34.751559+0000","last_unstale":"2024-04-16T05:34:02.931179+0000","last_undegraded":"2024-04-16T05:34:02.931179+0000","last_fullsized":"2024-04-16T05:34:02.931179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:32:59.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763529+0000","last_change":"2024-04-16T05:32:39.109330+0000","last_active":"2024-04-16T05:33:34.763529+0000","last_peered":"2024-04-16T05:33:34.763529+0000","last_clean":"2024-04-16T05:33:34.763529+0000","last_became_active":"2024-04-16T05:32:39.108718+0000","last_became_peered":"2024-04-16T05:32:39.108718+0000","last_unstale":"2024-04-16T05:33:34.763529+0000","last_undegraded":"2024-04-16T05:33:34.763529+0000","last_fullsized":"2024-04-16T05:33:34.763529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:36:55.418034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761016+0000","last_change":"2024-04-16T05:32:33.741603+0000","last_active":"2024-04-16T05:33:34.761016+0000","last_peered":"2024-04-16T05:33:34.761016+0000","last_clean":"2024-04-16T05:33:34.761016+0000","last_became_active":"2024-04-16T05:32:33.741184+0000","last_became_peered":"2024-04-16T05:32:33.741184+0000","last_unstale":"2024-04-16T05:33:34.761016+0000","last_undegraded":"2024-04-16T05:33:34.761016+0000","last_fullsized":"2024-04-16T05:33:34.761016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:08:38.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.941772+0000","last_change":"2024-04-16T05:32:34.747768+0000","last_active":"2024-04-16T05:34:02.941772+0000","last_peered":"2024-04-16T05:34:02.941772+0000","last_clean":"2024-04-16T05:34:02.941772+0000","last_became_active":"2024-04-16T05:32:34.747440+0000","last_became_peered":"2024-04-16T05:32:34.747440+0000","last_unstale":"2024-04-16T05:34:02.941772+0000","last_undegraded":"2024-04-16T05:34:02.941772+0000","last_fullsized":"2024-04-16T05:34:02.941772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:14:31.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763872+0000","last_change":"2024-04-16T05:32:38.938673+0000","last_active":"2024-04-16T05:33:34.763872+0000","last_peered":"2024-04-16T05:33:34.763872+0000","last_clean":"2024-04-16T05:33:34.763872+0000","last_became_active":"2024-04-16T05:32:38.938252+0000","last_became_peered":"2024-04-16T05:32:38.938252+0000","last_unstale":"2024-04-16T05:33:34.763872+0000","last_undegraded":"2024-04-16T05:33:34.763872+0000","last_fullsized":"2024-04-16T05:33:34.763872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:06.620693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.768545+0000","last_change":"2024-04-16T05:32:33.750812+0000","last_active":"2024-04-16T05:33:34.768545+0000","last_peered":"2024-04-16T05:33:34.768545+0000","last_clean":"2024-04-16T05:33:34.768545+0000","last_became_active":"2024-04-16T05:32:33.750514+0000","last_became_peered":"2024-04-16T05:32:33.750514+0000","last_unstale":"2024-04-16T05:33:34.768545+0000","last_undegraded":"2024-04-16T05:33:34.768545+0000","last_fullsized":"2024-04-16T05:33:34.768545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:53:43.075914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.931534+0000","last_change":"2024-04-16T05:33:34.770449+0000","last_active":"2024-04-16T05:34:02.931534+0000","last_peered":"2024-04-16T05:34:02.931534+0000","last_clean":"2024-04-16T05:34:02.931534+0000","last_became_active":"2024-04-16T05:33:34.770078+0000","last_became_peered":"2024-04-16T05:33:34.770078+0000","last_unstale":"2024-04-16T05:34:02.931534+0000","last_undegraded":"2024-04-16T05:34:02.931534+0000","last_fullsized":"2024-04-16T05:34:02.931534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:30:20.885002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763421+0000","last_change":"2024-04-16T05:32:39.109394+0000","last_active":"2024-04-16T05:33:34.763421+0000","last_peered":"2024-04-16T05:33:34.763421+0000","last_clean":"2024-04-16T05:33:34.763421+0000","last_became_active":"2024-04-16T05:32:39.108924+0000","last_became_peered":"2024-04-16T05:32:39.108924+0000","last_unstale":"2024-04-16T05:33:34.763421+0000","last_undegraded":"2024-04-16T05:33:34.763421+0000","last_fullsized":"2024-04-16T05:33:34.763421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:50:37.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.934717+0000","last_change":"2024-04-16T05:33:34.766071+0000","last_active":"2024-04-16T05:34:02.934717+0000","last_peered":"2024-04-16T05:34:02.934717+0000","last_clean":"2024-04-16T05:34:02.934717+0000","last_became_active":"2024-04-16T05:33:34.765670+0000","last_became_peered":"2024-04-16T05:33:34.765670+0000","last_unstale":"2024-04-16T05:34:02.934717+0000","last_undegraded":"2024-04-16T05:34:02.934717+0000","last_fullsized":"2024-04-16T05:34:02.934717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:54:47.320828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:06.799587+0000","last_change":"2024-04-16T05:32:33.748521+0000","last_active":"2024-04-16T05:34:06.799587+0000","last_peered":"2024-04-16T05:34:06.799587+0000","last_clean":"2024-04-16T05:34:06.799587+0000","last_became_active":"2024-04-16T05:32:33.748222+0000","last_became_peered":"2024-04-16T05:32:33.748222+0000","last_unstale":"2024-04-16T05:34:06.799587+0000","last_undegraded":"2024-04-16T05:34:06.799587+0000","last_fullsized":"2024-04-16T05:34:06.799587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:04.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769001+0000","last_change":"2024-04-16T05:32:38.935995+0000","last_active":"2024-04-16T05:33:34.769001+0000","last_peered":"2024-04-16T05:33:34.769001+0000","last_clean":"2024-04-16T05:33:34.769001+0000","last_became_active":"2024-04-16T05:32:38.935511+0000","last_became_peered":"2024-04-16T05:32:38.935511+0000","last_unstale":"2024-04-16T05:33:34.769001+0000","last_undegraded":"2024-04-16T05:33:34.769001+0000","last_fullsized":"2024-04-16T05:33:34.769001+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:17:08.144537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.932771+0000","last_change":"2024-04-16T05:32:34.760249+0000","last_active":"2024-04-16T05:34:02.932771+0000","last_peered":"2024-04-16T05:34:02.932771+0000","last_clean":"2024-04-16T05:34:02.932771+0000","last_became_active":"2024-04-16T05:32:34.759735+0000","last_became_peered":"2024-04-16T05:32:34.759735+0000","last_unstale":"2024-04-16T05:34:02.932771+0000","last_undegraded":"2024-04-16T05:34:02.932771+0000","last_fullsized":"2024-04-16T05:34:02.932771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:13.349558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763827+0000","last_change":"2024-04-16T05:32:33.731170+0000","last_active":"2024-04-16T05:33:34.763827+0000","last_peered":"2024-04-16T05:33:34.763827+0000","last_clean":"2024-04-16T05:33:34.763827+0000","last_became_active":"2024-04-16T05:32:33.730821+0000","last_became_peered":"2024-04-16T05:32:33.730821+0000","last_unstale":"2024-04-16T05:33:34.763827+0000","last_undegraded":"2024-04-16T05:33:34.763827+0000","last_fullsized":"2024-04-16T05:33:34.763827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:38:15.707590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":363,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.768475+0000","last_change":"2024-04-16T05:31:24.926319+0000","last_active":"2024-04-16T05:33:34.768475+0000","last_peered":"2024-04-16T05:33:34.768475+0000","last_clean":"2024-04-16T05:33:34.768475+0000","last_became_active":"2024-04-16T05:31:24.925936+0000","last_became_peered":"2024-04-16T05:31:24.925936+0000","last_unstale":"2024-04-16T05:33:34.768475+0000","last_undegraded":"2024-04-16T05:33:34.768475+0000","last_fullsized":"2024-04-16T05:33:34.768475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_clean_scrub_stamp":"2024-04-16T05:27:57.876673+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:28:19.212127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.844235+0000","last_change":"2024-04-16T05:32:39.110241+0000","last_active":"2024-04-16T05:33:34.844235+0000","last_peered":"2024-04-16T05:33:34.844235+0000","last_clean":"2024-04-16T05:33:34.844235+0000","last_became_active":"2024-04-16T05:32:39.109792+0000","last_became_peered":"2024-04-16T05:32:39.109792+0000","last_unstale":"2024-04-16T05:33:34.844235+0000","last_undegraded":"2024-04-16T05:33:34.844235+0000","last_fullsized":"2024-04-16T05:33:34.844235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:15:05.925370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935060+0000","last_change":"2024-04-16T05:32:34.751693+0000","last_active":"2024-04-16T05:34:02.935060+0000","last_peered":"2024-04-16T05:34:02.935060+0000","last_clean":"2024-04-16T05:34:02.935060+0000","last_became_active":"2024-04-16T05:32:34.751210+0000","last_became_peered":"2024-04-16T05:32:34.751210+0000","last_unstale":"2024-04-16T05:34:02.935060+0000","last_undegraded":"2024-04-16T05:34:02.935060+0000","last_fullsized":"2024-04-16T05:34:02.935060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:33:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.768542+0000","last_change":"2024-04-16T05:32:33.738310+0000","last_active":"2024-04-16T05:33:34.768542+0000","last_peered":"2024-04-16T05:33:34.768542+0000","last_clean":"2024-04-16T05:33:34.768542+0000","last_became_active":"2024-04-16T05:32:33.738012+0000","last_became_peered":"2024-04-16T05:32:33.738012+0000","last_unstale":"2024-04-16T05:33:34.768542+0000","last_undegraded":"2024-04-16T05:33:34.768542+0000","last_fullsized":"2024-04-16T05:33:34.768542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:22:18.311458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.766912+0000","last_change":"2024-04-16T05:32:38.933465+0000","last_active":"2024-04-16T05:33:34.766912+0000","last_peered":"2024-04-16T05:33:34.766912+0000","last_clean":"2024-04-16T05:33:34.766912+0000","last_became_active":"2024-04-16T05:32:38.932391+0000","last_became_peered":"2024-04-16T05:32:38.932391+0000","last_unstale":"2024-04-16T05:33:34.766912+0000","last_undegraded":"2024-04-16T05:33:34.766912+0000","last_fullsized":"2024-04-16T05:33:34.766912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:39.494556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763957+0000","last_change":"2024-04-16T05:32:33.743241+0000","last_active":"2024-04-16T05:33:34.763957+0000","last_peered":"2024-04-16T05:33:34.763957+0000","last_clean":"2024-04-16T05:33:34.763957+0000","last_became_active":"2024-04-16T05:32:33.742887+0000","last_became_peered":"2024-04-16T05:32:33.742887+0000","last_unstale":"2024-04-16T05:33:34.763957+0000","last_undegraded":"2024-04-16T05:33:34.763957+0000","last_fullsized":"2024-04-16T05:33:34.763957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:24:38.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935151+0000","last_change":"2024-04-16T05:32:34.746228+0000","last_active":"2024-04-16T05:34:02.935151+0000","last_peered":"2024-04-16T05:34:02.935151+0000","last_clean":"2024-04-16T05:34:02.935151+0000","last_became_active":"2024-04-16T05:32:34.745883+0000","last_became_peered":"2024-04-16T05:32:34.745883+0000","last_unstale":"2024-04-16T05:34:02.935151+0000","last_undegraded":"2024-04-16T05:34:02.935151+0000","last_fullsized":"2024-04-16T05:34:02.935151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:33.656358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769577+0000","last_change":"2024-04-16T05:32:39.110156+0000","last_active":"2024-04-16T05:33:34.769577+0000","last_peered":"2024-04-16T05:33:34.769577+0000","last_clean":"2024-04-16T05:33:34.769577+0000","last_became_active":"2024-04-16T05:32:39.108958+0000","last_became_peered":"2024-04-16T05:32:39.108958+0000","last_unstale":"2024-04-16T05:33:34.769577+0000","last_undegraded":"2024-04-16T05:33:34.769577+0000","last_fullsized":"2024-04-16T05:33:34.769577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:27:58.658187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.760846+0000","last_change":"2024-04-16T05:32:33.738179+0000","last_active":"2024-04-16T05:33:34.760846+0000","last_peered":"2024-04-16T05:33:34.760846+0000","last_clean":"2024-04-16T05:33:34.760846+0000","last_became_active":"2024-04-16T05:32:33.737841+0000","last_became_peered":"2024-04-16T05:32:33.737841+0000","last_unstale":"2024-04-16T05:33:34.760846+0000","last_undegraded":"2024-04-16T05:33:34.760846+0000","last_fullsized":"2024-04-16T05:33:34.760846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:13:11.210559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935161+0000","last_change":"2024-04-16T05:32:34.758142+0000","last_active":"2024-04-16T05:34:02.935161+0000","last_peered":"2024-04-16T05:34:02.935161+0000","last_clean":"2024-04-16T05:34:02.935161+0000","last_became_active":"2024-04-16T05:32:34.757825+0000","last_became_peered":"2024-04-16T05:32:34.757825+0000","last_unstale":"2024-04-16T05:34:02.935161+0000","last_undegraded":"2024-04-16T05:34:02.935161+0000","last_fullsized":"2024-04-16T05:34:02.935161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:37.575764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761987+0000","last_change":"2024-04-16T05:32:38.933327+0000","last_active":"2024-04-16T05:33:34.761987+0000","last_peered":"2024-04-16T05:33:34.761987+0000","last_clean":"2024-04-16T05:33:34.761987+0000","last_became_active":"2024-04-16T05:32:38.932808+0000","last_became_peered":"2024-04-16T05:32:38.932808+0000","last_unstale":"2024-04-16T05:33:34.761987+0000","last_undegraded":"2024-04-16T05:33:34.761987+0000","last_fullsized":"2024-04-16T05:33:34.761987+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:03.217103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764455+0000","last_change":"2024-04-16T05:32:33.742405+0000","last_active":"2024-04-16T05:33:34.764455+0000","last_peered":"2024-04-16T05:33:34.764455+0000","last_clean":"2024-04-16T05:33:34.764455+0000","last_became_active":"2024-04-16T05:32:33.742051+0000","last_became_peered":"2024-04-16T05:32:33.742051+0000","last_unstale":"2024-04-16T05:33:34.764455+0000","last_undegraded":"2024-04-16T05:33:34.764455+0000","last_fullsized":"2024-04-16T05:33:34.764455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:35:13.914483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938554+0000","last_change":"2024-04-16T05:33:34.772430+0000","last_active":"2024-04-16T05:34:02.938554+0000","last_peered":"2024-04-16T05:34:02.938554+0000","last_clean":"2024-04-16T05:34:02.938554+0000","last_became_active":"2024-04-16T05:33:34.772067+0000","last_became_peered":"2024-04-16T05:33:34.772067+0000","last_unstale":"2024-04-16T05:34:02.938554+0000","last_undegraded":"2024-04-16T05:34:02.938554+0000","last_fullsized":"2024-04-16T05:34:02.938554+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:24.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.770714+0000","last_change":"2024-04-16T05:33:34.770714+0000","last_active":"2024-04-16T05:33:34.770714+0000","last_peered":"2024-04-16T05:33:34.770714+0000","last_clean":"2024-04-16T05:33:34.770714+0000","last_became_active":"2024-04-16T05:33:34.770249+0000","last_became_peered":"2024-04-16T05:33:34.770249+0000","last_unstale":"2024-04-16T05:33:34.770714+0000","last_undegraded":"2024-04-16T05:33:34.770714+0000","last_fullsized":"2024-04-16T05:33:34.770714+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:46.517492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761304+0000","last_change":"2024-04-16T05:32:33.743031+0000","last_active":"2024-04-16T05:33:34.761304+0000","last_peered":"2024-04-16T05:33:34.761304+0000","last_clean":"2024-04-16T05:33:34.761304+0000","last_became_active":"2024-04-16T05:32:33.742771+0000","last_became_peered":"2024-04-16T05:32:33.742771+0000","last_unstale":"2024-04-16T05:33:34.761304+0000","last_undegraded":"2024-04-16T05:33:34.761304+0000","last_fullsized":"2024-04-16T05:33:34.761304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:06.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.894157+0000","last_change":"2024-04-16T05:32:34.749428+0000","last_active":"2024-04-16T05:34:02.894157+0000","last_peered":"2024-04-16T05:34:02.894157+0000","last_clean":"2024-04-16T05:34:02.894157+0000","last_became_active":"2024-04-16T05:32:34.749087+0000","last_became_peered":"2024-04-16T05:32:34.749087+0000","last_unstale":"2024-04-16T05:34:02.894157+0000","last_undegraded":"2024-04-16T05:34:02.894157+0000","last_fullsized":"2024-04-16T05:34:02.894157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:59:25.124558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761056+0000","last_change":"2024-04-16T05:32:38.942407+0000","last_active":"2024-04-16T05:33:34.761056+0000","last_peered":"2024-04-16T05:33:34.761056+0000","last_clean":"2024-04-16T05:33:34.761056+0000","last_became_active":"2024-04-16T05:32:38.941731+0000","last_became_peered":"2024-04-16T05:32:38.941731+0000","last_unstale":"2024-04-16T05:33:34.761056+0000","last_undegraded":"2024-04-16T05:33:34.761056+0000","last_fullsized":"2024-04-16T05:33:34.761056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:42:27.700203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769319+0000","last_change":"2024-04-16T05:32:33.741487+0000","last_active":"2024-04-16T05:33:34.769319+0000","last_peered":"2024-04-16T05:33:34.769319+0000","last_clean":"2024-04-16T05:33:34.769319+0000","last_became_active":"2024-04-16T05:32:33.741140+0000","last_became_peered":"2024-04-16T05:32:33.741140+0000","last_unstale":"2024-04-16T05:33:34.769319+0000","last_undegraded":"2024-04-16T05:33:34.769319+0000","last_fullsized":"2024-04-16T05:33:34.769319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:21:36.086164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.939265+0000","last_change":"2024-04-16T05:32:34.756608+0000","last_active":"2024-04-16T05:34:02.939265+0000","last_peered":"2024-04-16T05:34:02.939265+0000","last_clean":"2024-04-16T05:34:02.939265+0000","last_became_active":"2024-04-16T05:32:34.756296+0000","last_became_peered":"2024-04-16T05:32:34.756296+0000","last_unstale":"2024-04-16T05:34:02.939265+0000","last_undegraded":"2024-04-16T05:34:02.939265+0000","last_fullsized":"2024-04-16T05:34:02.939265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:28.397333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764227+0000","last_change":"2024-04-16T05:32:38.929876+0000","last_active":"2024-04-16T05:33:34.764227+0000","last_peered":"2024-04-16T05:33:34.764227+0000","last_clean":"2024-04-16T05:33:34.764227+0000","last_became_active":"2024-04-16T05:32:38.928626+0000","last_became_peered":"2024-04-16T05:32:38.928626+0000","last_unstale":"2024-04-16T05:33:34.764227+0000","last_undegraded":"2024-04-16T05:33:34.764227+0000","last_fullsized":"2024-04-16T05:33:34.764227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:08:09.697413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762274+0000","last_change":"2024-04-16T05:32:33.736047+0000","last_active":"2024-04-16T05:33:34.762274+0000","last_peered":"2024-04-16T05:33:34.762274+0000","last_clean":"2024-04-16T05:33:34.762274+0000","last_became_active":"2024-04-16T05:32:33.735683+0000","last_became_peered":"2024-04-16T05:32:33.735683+0000","last_unstale":"2024-04-16T05:33:34.762274+0000","last_undegraded":"2024-04-16T05:33:34.762274+0000","last_fullsized":"2024-04-16T05:33:34.762274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:58:32.969924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938725+0000","last_change":"2024-04-16T05:32:34.763558+0000","last_active":"2024-04-16T05:34:02.938725+0000","last_peered":"2024-04-16T05:34:02.938725+0000","last_clean":"2024-04-16T05:34:02.938725+0000","last_became_active":"2024-04-16T05:32:34.763190+0000","last_became_peered":"2024-04-16T05:32:34.763190+0000","last_unstale":"2024-04-16T05:34:02.938725+0000","last_undegraded":"2024-04-16T05:34:02.938725+0000","last_fullsized":"2024-04-16T05:34:02.938725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:38:31.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764426+0000","last_change":"2024-04-16T05:32:39.109426+0000","last_active":"2024-04-16T05:33:34.764426+0000","last_peered":"2024-04-16T05:33:34.764426+0000","last_clean":"2024-04-16T05:33:34.764426+0000","last_became_active":"2024-04-16T05:32:39.108970+0000","last_became_peered":"2024-04-16T05:32:39.108970+0000","last_unstale":"2024-04-16T05:33:34.764426+0000","last_undegraded":"2024-04-16T05:33:34.764426+0000","last_fullsized":"2024-04-16T05:33:34.764426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:54.824654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763251+0000","last_change":"2024-04-16T05:32:33.734983+0000","last_active":"2024-04-16T05:33:34.763251+0000","last_peered":"2024-04-16T05:33:34.763251+0000","last_clean":"2024-04-16T05:33:34.763251+0000","last_became_active":"2024-04-16T05:32:33.734609+0000","last_became_peered":"2024-04-16T05:32:33.734609+0000","last_unstale":"2024-04-16T05:33:34.763251+0000","last_undegraded":"2024-04-16T05:33:34.763251+0000","last_fullsized":"2024-04-16T05:33:34.763251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:35:59.436499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.944339+0000","last_change":"2024-04-16T05:32:34.759811+0000","last_active":"2024-04-16T05:34:02.944339+0000","last_peered":"2024-04-16T05:34:02.944339+0000","last_clean":"2024-04-16T05:34:02.944339+0000","last_became_active":"2024-04-16T05:32:34.759499+0000","last_became_peered":"2024-04-16T05:32:34.759499+0000","last_unstale":"2024-04-16T05:34:02.944339+0000","last_undegraded":"2024-04-16T05:34:02.944339+0000","last_fullsized":"2024-04-16T05:34:02.944339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:46:28.554191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761397+0000","last_change":"2024-04-16T05:32:38.929021+0000","last_active":"2024-04-16T05:33:34.761397+0000","last_peered":"2024-04-16T05:33:34.761397+0000","last_clean":"2024-04-16T05:33:34.761397+0000","last_became_active":"2024-04-16T05:32:38.928574+0000","last_became_peered":"2024-04-16T05:32:38.928574+0000","last_unstale":"2024-04-16T05:33:34.761397+0000","last_undegraded":"2024-04-16T05:33:34.761397+0000","last_fullsized":"2024-04-16T05:33:34.761397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:11:35.693364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762928+0000","last_change":"2024-04-16T05:32:33.747352+0000","last_active":"2024-04-16T05:33:34.762928+0000","last_peered":"2024-04-16T05:33:34.762928+0000","last_clean":"2024-04-16T05:33:34.762928+0000","last_became_active":"2024-04-16T05:32:33.746989+0000","last_became_peered":"2024-04-16T05:32:33.746989+0000","last_unstale":"2024-04-16T05:33:34.762928+0000","last_undegraded":"2024-04-16T05:33:34.762928+0000","last_fullsized":"2024-04-16T05:33:34.762928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:38.422274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.944464+0000","last_change":"2024-04-16T05:32:34.760636+0000","last_active":"2024-04-16T05:34:02.944464+0000","last_peered":"2024-04-16T05:34:02.944464+0000","last_clean":"2024-04-16T05:34:02.944464+0000","last_became_active":"2024-04-16T05:32:34.760335+0000","last_became_peered":"2024-04-16T05:32:34.760335+0000","last_unstale":"2024-04-16T05:34:02.944464+0000","last_undegraded":"2024-04-16T05:34:02.944464+0000","last_fullsized":"2024-04-16T05:34:02.944464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:04:34.322799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769486+0000","last_change":"2024-04-16T05:33:34.769486+0000","last_active":"2024-04-16T05:33:34.769486+0000","last_peered":"2024-04-16T05:33:34.769486+0000","last_clean":"2024-04-16T05:33:34.769486+0000","last_became_active":"2024-04-16T05:33:34.769025+0000","last_became_peered":"2024-04-16T05:33:34.769025+0000","last_unstale":"2024-04-16T05:33:34.769486+0000","last_undegraded":"2024-04-16T05:33:34.769486+0000","last_fullsized":"2024-04-16T05:33:34.769486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:10.382817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.844326+0000","last_change":"2024-04-16T05:32:33.746346+0000","last_active":"2024-04-16T05:33:34.844326+0000","last_peered":"2024-04-16T05:33:34.844326+0000","last_clean":"2024-04-16T05:33:34.844326+0000","last_became_active":"2024-04-16T05:32:33.746030+0000","last_became_peered":"2024-04-16T05:32:33.746030+0000","last_unstale":"2024-04-16T05:33:34.844326+0000","last_undegraded":"2024-04-16T05:33:34.844326+0000","last_fullsized":"2024-04-16T05:33:34.844326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:54:35.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.932765+0000","last_change":"2024-04-16T05:32:34.752283+0000","last_active":"2024-04-16T05:34:02.932765+0000","last_peered":"2024-04-16T05:34:02.932765+0000","last_clean":"2024-04-16T05:34:02.932765+0000","last_became_active":"2024-04-16T05:32:34.751904+0000","last_became_peered":"2024-04-16T05:32:34.751904+0000","last_unstale":"2024-04-16T05:34:02.932765+0000","last_undegraded":"2024-04-16T05:34:02.932765+0000","last_fullsized":"2024-04-16T05:34:02.932765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:15:52.688683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.843287+0000","last_change":"2024-04-16T05:32:38.934194+0000","last_active":"2024-04-16T05:33:34.843287+0000","last_peered":"2024-04-16T05:33:34.843287+0000","last_clean":"2024-04-16T05:33:34.843287+0000","last_became_active":"2024-04-16T05:32:38.933636+0000","last_became_peered":"2024-04-16T05:32:38.933636+0000","last_unstale":"2024-04-16T05:33:34.843287+0000","last_undegraded":"2024-04-16T05:33:34.843287+0000","last_fullsized":"2024-04-16T05:33:34.843287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:14:40.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763544+0000","last_change":"2024-04-16T05:32:33.744267+0000","last_active":"2024-04-16T05:33:34.763544+0000","last_peered":"2024-04-16T05:33:34.763544+0000","last_clean":"2024-04-16T05:33:34.763544+0000","last_became_active":"2024-04-16T05:32:33.743931+0000","last_became_peered":"2024-04-16T05:32:33.743931+0000","last_unstale":"2024-04-16T05:33:34.763544+0000","last_undegraded":"2024-04-16T05:33:34.763544+0000","last_fullsized":"2024-04-16T05:33:34.763544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:30.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.937487+0000","last_change":"2024-04-16T05:32:34.756833+0000","last_active":"2024-04-16T05:34:02.937487+0000","last_peered":"2024-04-16T05:34:02.937487+0000","last_clean":"2024-04-16T05:34:02.937487+0000","last_became_active":"2024-04-16T05:32:34.756436+0000","last_became_peered":"2024-04-16T05:32:34.756436+0000","last_unstale":"2024-04-16T05:34:02.937487+0000","last_undegraded":"2024-04-16T05:34:02.937487+0000","last_fullsized":"2024-04-16T05:34:02.937487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:20.732946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935401+0000","last_change":"2024-04-16T05:32:34.752744+0000","last_active":"2024-04-16T05:34:02.935401+0000","last_peered":"2024-04-16T05:34:02.935401+0000","last_clean":"2024-04-16T05:34:02.935401+0000","last_became_active":"2024-04-16T05:32:34.752442+0000","last_became_peered":"2024-04-16T05:32:34.752442+0000","last_unstale":"2024-04-16T05:34:02.935401+0000","last_undegraded":"2024-04-16T05:34:02.935401+0000","last_fullsized":"2024-04-16T05:34:02.935401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:01:23.644692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.937616+0000","last_change":"2024-04-16T05:32:34.755118+0000","last_active":"2024-04-16T05:34:02.937616+0000","last_peered":"2024-04-16T05:34:02.937616+0000","last_clean":"2024-04-16T05:34:02.937616+0000","last_became_active":"2024-04-16T05:32:34.754824+0000","last_became_peered":"2024-04-16T05:32:34.754824+0000","last_unstale":"2024-04-16T05:34:02.937616+0000","last_undegraded":"2024-04-16T05:34:02.937616+0000","last_fullsized":"2024-04-16T05:34:02.937616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:58:37.860305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.939902+0000","last_change":"2024-04-16T05:32:34.756144+0000","last_active":"2024-04-16T05:34:02.939902+0000","last_peered":"2024-04-16T05:34:02.939902+0000","last_clean":"2024-04-16T05:34:02.939902+0000","last_became_active":"2024-04-16T05:32:34.755841+0000","last_became_peered":"2024-04-16T05:32:34.755841+0000","last_unstale":"2024-04-16T05:34:02.939902+0000","last_undegraded":"2024-04-16T05:34:02.939902+0000","last_fullsized":"2024-04-16T05:34:02.939902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:23:00.482444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.937769+0000","last_change":"2024-04-16T05:32:34.766204+0000","last_active":"2024-04-16T05:34:02.937769+0000","last_peered":"2024-04-16T05:34:02.937769+0000","last_clean":"2024-04-16T05:34:02.937769+0000","last_became_active":"2024-04-16T05:32:34.765709+0000","last_became_peered":"2024-04-16T05:32:34.765709+0000","last_unstale":"2024-04-16T05:34:02.937769+0000","last_undegraded":"2024-04-16T05:34:02.937769+0000","last_fullsized":"2024-04-16T05:34:02.937769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:23:53.926477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.940717+0000","last_change":"2024-04-16T05:32:34.758513+0000","last_active":"2024-04-16T05:34:02.940717+0000","last_peered":"2024-04-16T05:34:02.940717+0000","last_clean":"2024-04-16T05:34:02.940717+0000","last_became_active":"2024-04-16T05:32:34.758145+0000","last_became_peered":"2024-04-16T05:32:34.758145+0000","last_unstale":"2024-04-16T05:34:02.940717+0000","last_undegraded":"2024-04-16T05:34:02.940717+0000","last_fullsized":"2024-04-16T05:34:02.940717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:09:17.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.930796+0000","last_change":"2024-04-16T05:32:34.750651+0000","last_active":"2024-04-16T05:34:02.930796+0000","last_peered":"2024-04-16T05:34:02.930796+0000","last_clean":"2024-04-16T05:34:02.930796+0000","last_became_active":"2024-04-16T05:32:34.750348+0000","last_became_peered":"2024-04-16T05:32:34.750348+0000","last_unstale":"2024-04-16T05:34:02.930796+0000","last_undegraded":"2024-04-16T05:34:02.930796+0000","last_fullsized":"2024-04-16T05:34:02.930796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:29.282449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935433+0000","last_change":"2024-04-16T05:32:34.750079+0000","last_active":"2024-04-16T05:34:02.935433+0000","last_peered":"2024-04-16T05:34:02.935433+0000","last_clean":"2024-04-16T05:34:02.935433+0000","last_became_active":"2024-04-16T05:32:34.749775+0000","last_became_peered":"2024-04-16T05:32:34.749775+0000","last_unstale":"2024-04-16T05:34:02.935433+0000","last_undegraded":"2024-04-16T05:34:02.935433+0000","last_fullsized":"2024-04-16T05:34:02.935433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:02.297258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.930824+0000","last_change":"2024-04-16T05:32:34.752528+0000","last_active":"2024-04-16T05:34:02.930824+0000","last_peered":"2024-04-16T05:34:02.930824+0000","last_clean":"2024-04-16T05:34:02.930824+0000","last_became_active":"2024-04-16T05:32:34.752150+0000","last_became_peered":"2024-04-16T05:32:34.752150+0000","last_unstale":"2024-04-16T05:34:02.930824+0000","last_undegraded":"2024-04-16T05:34:02.930824+0000","last_fullsized":"2024-04-16T05:34:02.930824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:41:16.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.930666+0000","last_change":"2024-04-16T05:32:34.758486+0000","last_active":"2024-04-16T05:34:02.930666+0000","last_peered":"2024-04-16T05:34:02.930666+0000","last_clean":"2024-04-16T05:34:02.930666+0000","last_became_active":"2024-04-16T05:32:34.754840+0000","last_became_peered":"2024-04-16T05:32:34.754840+0000","last_unstale":"2024-04-16T05:34:02.930666+0000","last_undegraded":"2024-04-16T05:34:02.930666+0000","last_fullsized":"2024-04-16T05:34:02.930666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:35.644337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935014+0000","last_change":"2024-04-16T05:32:34.758933+0000","last_active":"2024-04-16T05:34:02.935014+0000","last_peered":"2024-04-16T05:34:02.935014+0000","last_clean":"2024-04-16T05:34:02.935014+0000","last_became_active":"2024-04-16T05:32:34.758587+0000","last_became_peered":"2024-04-16T05:32:34.758587+0000","last_unstale":"2024-04-16T05:34:02.935014+0000","last_undegraded":"2024-04-16T05:34:02.935014+0000","last_fullsized":"2024-04-16T05:34:02.935014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:19:20.674248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.946578+0000","last_change":"2024-04-16T05:32:34.750846+0000","last_active":"2024-04-16T05:34:02.946578+0000","last_peered":"2024-04-16T05:34:02.946578+0000","last_clean":"2024-04-16T05:34:02.946578+0000","last_became_active":"2024-04-16T05:32:34.750529+0000","last_became_peered":"2024-04-16T05:32:34.750529+0000","last_unstale":"2024-04-16T05:34:02.946578+0000","last_undegraded":"2024-04-16T05:34:02.946578+0000","last_fullsized":"2024-04-16T05:34:02.946578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:32.288459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.943668+0000","last_change":"2024-04-16T05:32:34.752566+0000","last_active":"2024-04-16T05:34:02.943668+0000","last_peered":"2024-04-16T05:34:02.943668+0000","last_clean":"2024-04-16T05:34:02.943668+0000","last_became_active":"2024-04-16T05:32:34.752236+0000","last_became_peered":"2024-04-16T05:32:34.752236+0000","last_unstale":"2024-04-16T05:34:02.943668+0000","last_undegraded":"2024-04-16T05:34:02.943668+0000","last_fullsized":"2024-04-16T05:34:02.943668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:43:36.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.936323+0000","last_change":"2024-04-16T05:32:34.750972+0000","last_active":"2024-04-16T05:34:02.936323+0000","last_peered":"2024-04-16T05:34:02.936323+0000","last_clean":"2024-04-16T05:34:02.936323+0000","last_became_active":"2024-04-16T05:32:34.750671+0000","last_became_peered":"2024-04-16T05:32:34.750671+0000","last_unstale":"2024-04-16T05:34:02.936323+0000","last_undegraded":"2024-04-16T05:34:02.936323+0000","last_fullsized":"2024-04-16T05:34:02.936323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:19:45.331816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283986,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66698,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":459,"num_read_kb":302,"num_write":169,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":200094,"data_compressed":4893,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":500857,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1150647,"data_compressed":58385,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808873,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":505155,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070511,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":500979,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1097561,"data_compressed":56754,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495673,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":496235,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":558169,"data_compressed":22271,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":497623,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":560017,"data_compressed":22271,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":496429,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":501417,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1092617,"data_compressed":56754,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":62024,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63872,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:34:10.984 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph pg dump --format=json 2024-04-16T05:34:11.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:11 smithi032 ceph-mon[32351]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:11.237 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:11.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:11 smithi082 ceph-mon[36195]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:11 smithi134 ceph-mon[36153]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:12.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:12 smithi082 ceph-mon[36195]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-16T05:34:12.458 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:12 smithi032 ceph-mon[32351]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-16T05:34:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:12 smithi134 ceph-mon[36153]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-16T05:34:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 -- 172.21.15.32:0/208111274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 msgr2=0x7f52b0103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/208111274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 0x7f52b0103000 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5298009990 tx=0x7f5298009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 -- 172.21.15.32:0/208111274 shutdown_connections 2024-04-16T05:34:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/208111274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52b0103540 0x7f52b010fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/208111274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b0108c20 0x7f52b0109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/208111274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 0x7f52b0103000 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 -- 172.21.15.32:0/208111274 >> 172.21.15.32:0/208111274 conn(0x7f52b0079ce0 msgr2=0x7f52b007a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 -- 172.21.15.32:0/208111274 shutdown_connections 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.036+0000 7f52b5cc1700 1 -- 172.21.15.32:0/208111274 wait complete. 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52b5cc1700 1 Processor -- start 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52b5cc1700 1 -- start start 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52b5cc1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 0x7f52b0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52b5cc1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52b0103540 0x7f52b01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52b5cc1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b0108c20 0x7f52b019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52b5cc1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112ee0 con 0x7f52b0102b80 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52b5cc1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112d60 con 0x7f52b0103540 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52b5cc1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0113060 con 0x7f52b0108c20 2024-04-16T05:34:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 0x7f52b0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 0x7f52b0199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37570/0 (socket says 172.21.15.32:37570) 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52af7fe700 1 -- 172.21.15.32:0/3574946809 learned_addr learned my addr 172.21.15.32:0/3574946809 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52affff700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b0108c20 0x7f52b019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.037+0000 7f52aeffd700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52b0103540 0x7f52b01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52af7fe700 1 -- 172.21.15.32:0/3574946809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b0108c20 msgr2=0x7f52b019dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52af7fe700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b0108c20 0x7f52b019dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52af7fe700 1 -- 172.21.15.32:0/3574946809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52b0103540 msgr2=0x7f52b01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52af7fe700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52b0103540 0x7f52b01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52af7fe700 1 -- 172.21.15.32:0/3574946809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b019e3a0 con 0x7f52b0102b80 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52af7fe700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 0x7f52b0199390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f52a000ada0 tx=0x7f52a000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52acff9700 1 -- 172.21.15.32:0/3574946809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0017020 con 0x7f52b0102b80 2024-04-16T05:34:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52acff9700 1 -- 172.21.15.32:0/3574946809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a0004bb0 con 0x7f52b0102b80 2024-04-16T05:34:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b019e690 con 0x7f52b0102b80 2024-04-16T05:34:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52acff9700 1 -- 172.21.15.32:0/3574946809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a00054a0 con 0x7f52b0102b80 2024-04-16T05:34:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.038+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b01a5f10 con 0x7f52b0102b80 2024-04-16T05:34:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.039+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b010b0e0 con 0x7f52b0102b80 2024-04-16T05:34:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.040+0000 7f52acff9700 1 -- 172.21.15.32:0/3574946809 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f52a0005660 con 0x7f52b0102b80 2024-04-16T05:34:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.041+0000 7f52acff9700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f529c0a3cc0 0x7f529c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.041+0000 7f52aeffd700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f529c0a3cc0 0x7f529c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.041+0000 7f52acff9700 1 -- 172.21.15.32:0/3574946809 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f52a00cfdb0 con 0x7f52b0102b80 2024-04-16T05:34:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.042+0000 7f52aeffd700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f529c0a3cc0 0x7f529c0a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5298006010 tx=0x7f529801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:13.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.046+0000 7f52acff9700 1 -- 172.21.15.32:0/3574946809 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52a0021020 con 0x7f52b0102b80 2024-04-16T05:34:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.248+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52b019a6f0 con 0x7f529c0a3cc0 2024-04-16T05:34:13.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.257+0000 7f52acff9700 1 -- 172.21.15.32:0/3574946809 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340784 (secure 0 0 0) 0x7f52b019a6f0 con 0x7f529c0a3cc0 2024-04-16T05:34:13.260 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f529c0a3cc0 msgr2=0x7f529c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f529c0a3cc0 0x7f529c0a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5298006010 tx=0x7f529801a040 comp rx=0 tx=0).stop 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 msgr2=0x7f52b0199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 0x7f52b0199390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f52a000ada0 tx=0x7f52a000c330 comp rx=0 tx=0).stop 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 shutdown_connections 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f529c0a3cc0 0x7f529c0a6180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b0108c20 0x7f52b019dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b0102b80 0x7f52b0199390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 --2- 172.21.15.32:0/3574946809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52b0103540 0x7f52b01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 >> 172.21.15.32:0/3574946809 conn(0x7f52b0079ce0 msgr2=0x7f52b01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 shutdown_connections 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:13.260+0000 7f52b5cc1700 1 -- 172.21.15.32:0/3574946809 wait complete. 2024-04-16T05:34:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-16T05:34:13.965 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-04-16T05:34:12.939925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657300,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":620,"num_read_kb":431,"num_write":352,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361292,"kb_used_data":12140,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580532,"statfs":{"total":1151940427776,"available":1151570464768,"internally_reserved":0,"allocated":12431360,"data_stored":7957706,"data_compressed":360915,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":43,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003766"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948428+0000","last_change":"2024-04-16T05:32:34.759518+0000","last_active":"2024-04-16T05:34:02.948428+0000","last_peered":"2024-04-16T05:34:02.948428+0000","last_clean":"2024-04-16T05:34:02.948428+0000","last_became_active":"2024-04-16T05:32:34.759155+0000","last_became_peered":"2024-04-16T05:32:34.759155+0000","last_unstale":"2024-04-16T05:34:02.948428+0000","last_undegraded":"2024-04-16T05:34:02.948428+0000","last_fullsized":"2024-04-16T05:34:02.948428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:39.935886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935858+0000","last_change":"2024-04-16T05:32:34.758553+0000","last_active":"2024-04-16T05:34:02.935858+0000","last_peered":"2024-04-16T05:34:02.935858+0000","last_clean":"2024-04-16T05:34:02.935858+0000","last_became_active":"2024-04-16T05:32:34.758242+0000","last_became_peered":"2024-04-16T05:32:34.758242+0000","last_unstale":"2024-04-16T05:34:02.935858+0000","last_undegraded":"2024-04-16T05:34:02.935858+0000","last_fullsized":"2024-04-16T05:34:02.935858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:09.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.940029+0000","last_change":"2024-04-16T05:32:34.754815+0000","last_active":"2024-04-16T05:34:02.940029+0000","last_peered":"2024-04-16T05:34:02.940029+0000","last_clean":"2024-04-16T05:34:02.940029+0000","last_became_active":"2024-04-16T05:32:34.754488+0000","last_became_peered":"2024-04-16T05:32:34.754488+0000","last_unstale":"2024-04-16T05:34:02.940029+0000","last_undegraded":"2024-04-16T05:34:02.940029+0000","last_fullsized":"2024-04-16T05:34:02.940029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:10:32.379770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935653+0000","last_change":"2024-04-16T05:33:34.766742+0000","last_active":"2024-04-16T05:34:02.935653+0000","last_peered":"2024-04-16T05:34:02.935653+0000","last_clean":"2024-04-16T05:34:02.935653+0000","last_became_active":"2024-04-16T05:33:34.766332+0000","last_became_peered":"2024-04-16T05:33:34.766332+0000","last_unstale":"2024-04-16T05:34:02.935653+0000","last_undegraded":"2024-04-16T05:34:02.935653+0000","last_fullsized":"2024-04-16T05:34:02.935653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:52:14.644496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.929983+0000","last_change":"2024-04-16T05:32:34.755623+0000","last_active":"2024-04-16T05:34:02.929983+0000","last_peered":"2024-04-16T05:34:02.929983+0000","last_clean":"2024-04-16T05:34:02.929983+0000","last_became_active":"2024-04-16T05:32:34.754205+0000","last_became_peered":"2024-04-16T05:32:34.754205+0000","last_unstale":"2024-04-16T05:34:02.929983+0000","last_undegraded":"2024-04-16T05:34:02.929983+0000","last_fullsized":"2024-04-16T05:34:02.929983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:34:07.998734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.943737+0000","last_change":"2024-04-16T05:32:34.753767+0000","last_active":"2024-04-16T05:34:02.943737+0000","last_peered":"2024-04-16T05:34:02.943737+0000","last_clean":"2024-04-16T05:34:02.943737+0000","last_became_active":"2024-04-16T05:32:34.753448+0000","last_became_peered":"2024-04-16T05:32:34.753448+0000","last_unstale":"2024-04-16T05:34:02.943737+0000","last_undegraded":"2024-04-16T05:34:02.943737+0000","last_fullsized":"2024-04-16T05:34:02.943737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:27:42.798615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938967+0000","last_change":"2024-04-16T05:32:34.754049+0000","last_active":"2024-04-16T05:34:02.938967+0000","last_peered":"2024-04-16T05:34:02.938967+0000","last_clean":"2024-04-16T05:34:02.938967+0000","last_became_active":"2024-04-16T05:32:34.753757+0000","last_became_peered":"2024-04-16T05:32:34.753757+0000","last_unstale":"2024-04-16T05:34:02.938967+0000","last_undegraded":"2024-04-16T05:34:02.938967+0000","last_fullsized":"2024-04-16T05:34:02.938967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:42:26.747100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.934758+0000","last_change":"2024-04-16T05:32:34.762950+0000","last_active":"2024-04-16T05:34:02.934758+0000","last_peered":"2024-04-16T05:34:02.934758+0000","last_clean":"2024-04-16T05:34:02.934758+0000","last_became_active":"2024-04-16T05:32:34.762073+0000","last_became_peered":"2024-04-16T05:32:34.762073+0000","last_unstale":"2024-04-16T05:34:02.934758+0000","last_undegraded":"2024-04-16T05:34:02.934758+0000","last_fullsized":"2024-04-16T05:34:02.934758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:57:26.623655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.947968+0000","last_change":"2024-04-16T05:32:34.751027+0000","last_active":"2024-04-16T05:34:02.947968+0000","last_peered":"2024-04-16T05:34:02.947968+0000","last_clean":"2024-04-16T05:34:02.947968+0000","last_became_active":"2024-04-16T05:32:34.750697+0000","last_became_peered":"2024-04-16T05:32:34.750697+0000","last_unstale":"2024-04-16T05:34:02.947968+0000","last_undegraded":"2024-04-16T05:34:02.947968+0000","last_fullsized":"2024-04-16T05:34:02.947968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:39:33.834972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.943773+0000","last_change":"2024-04-16T05:32:34.752293+0000","last_active":"2024-04-16T05:34:02.943773+0000","last_peered":"2024-04-16T05:34:02.943773+0000","last_clean":"2024-04-16T05:34:02.943773+0000","last_became_active":"2024-04-16T05:32:34.752008+0000","last_became_peered":"2024-04-16T05:32:34.752008+0000","last_unstale":"2024-04-16T05:34:02.943773+0000","last_undegraded":"2024-04-16T05:34:02.943773+0000","last_fullsized":"2024-04-16T05:34:02.943773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:02:17.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.941247+0000","last_change":"2024-04-16T05:32:34.760961+0000","last_active":"2024-04-16T05:34:02.941247+0000","last_peered":"2024-04-16T05:34:02.941247+0000","last_clean":"2024-04-16T05:34:02.941247+0000","last_became_active":"2024-04-16T05:32:34.760640+0000","last_became_peered":"2024-04-16T05:32:34.760640+0000","last_unstale":"2024-04-16T05:34:02.941247+0000","last_undegraded":"2024-04-16T05:34:02.941247+0000","last_fullsized":"2024-04-16T05:34:02.941247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:47:28.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938913+0000","last_change":"2024-04-16T05:32:34.748646+0000","last_active":"2024-04-16T05:34:02.938913+0000","last_peered":"2024-04-16T05:34:02.938913+0000","last_clean":"2024-04-16T05:34:02.938913+0000","last_became_active":"2024-04-16T05:32:34.748090+0000","last_became_peered":"2024-04-16T05:32:34.748090+0000","last_unstale":"2024-04-16T05:34:02.938913+0000","last_undegraded":"2024-04-16T05:34:02.938913+0000","last_fullsized":"2024-04-16T05:34:02.938913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:00:19.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.933168+0000","last_change":"2024-04-16T05:32:34.747025+0000","last_active":"2024-04-16T05:34:02.933168+0000","last_peered":"2024-04-16T05:34:02.933168+0000","last_clean":"2024-04-16T05:34:02.933168+0000","last_became_active":"2024-04-16T05:32:34.746711+0000","last_became_peered":"2024-04-16T05:32:34.746711+0000","last_unstale":"2024-04-16T05:34:02.933168+0000","last_undegraded":"2024-04-16T05:34:02.933168+0000","last_fullsized":"2024-04-16T05:34:02.933168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:40:30.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.928188+0000","last_change":"2024-04-16T05:32:34.756756+0000","last_active":"2024-04-16T05:34:02.928188+0000","last_peered":"2024-04-16T05:34:02.928188+0000","last_clean":"2024-04-16T05:34:02.928188+0000","last_became_active":"2024-04-16T05:32:34.756442+0000","last_became_peered":"2024-04-16T05:32:34.756442+0000","last_unstale":"2024-04-16T05:34:02.928188+0000","last_undegraded":"2024-04-16T05:34:02.928188+0000","last_fullsized":"2024-04-16T05:34:02.928188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:33.864944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.940021+0000","last_change":"2024-04-16T05:32:34.765443+0000","last_active":"2024-04-16T05:34:02.940021+0000","last_peered":"2024-04-16T05:34:02.940021+0000","last_clean":"2024-04-16T05:34:02.940021+0000","last_became_active":"2024-04-16T05:32:34.765133+0000","last_became_peered":"2024-04-16T05:32:34.765133+0000","last_unstale":"2024-04-16T05:34:02.940021+0000","last_undegraded":"2024-04-16T05:34:02.940021+0000","last_fullsized":"2024-04-16T05:34:02.940021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:31.452184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.941676+0000","last_change":"2024-04-16T05:32:34.757577+0000","last_active":"2024-04-16T05:34:02.941676+0000","last_peered":"2024-04-16T05:34:02.941676+0000","last_clean":"2024-04-16T05:34:02.941676+0000","last_became_active":"2024-04-16T05:32:34.753451+0000","last_became_peered":"2024-04-16T05:32:34.753451+0000","last_unstale":"2024-04-16T05:34:02.941676+0000","last_undegraded":"2024-04-16T05:34:02.941676+0000","last_fullsized":"2024-04-16T05:34:02.941676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:56.617824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948568+0000","last_change":"2024-04-16T05:32:34.759776+0000","last_active":"2024-04-16T05:34:02.948568+0000","last_peered":"2024-04-16T05:34:02.948568+0000","last_clean":"2024-04-16T05:34:02.948568+0000","last_became_active":"2024-04-16T05:32:34.759371+0000","last_became_peered":"2024-04-16T05:32:34.759371+0000","last_unstale":"2024-04-16T05:34:02.948568+0000","last_undegraded":"2024-04-16T05:34:02.948568+0000","last_fullsized":"2024-04-16T05:34:02.948568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:36.458515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.929006+0000","last_change":"2024-04-16T05:32:34.747803+0000","last_active":"2024-04-16T05:34:02.929006+0000","last_peered":"2024-04-16T05:34:02.929006+0000","last_clean":"2024-04-16T05:34:02.929006+0000","last_became_active":"2024-04-16T05:32:34.747472+0000","last_became_peered":"2024-04-16T05:32:34.747472+0000","last_unstale":"2024-04-16T05:34:02.929006+0000","last_undegraded":"2024-04-16T05:34:02.929006+0000","last_fullsized":"2024-04-16T05:34:02.929006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:41.118574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.937007+0000","last_change":"2024-04-16T05:32:34.754151+0000","last_active":"2024-04-16T05:34:02.937007+0000","last_peered":"2024-04-16T05:34:02.937007+0000","last_clean":"2024-04-16T05:34:02.937007+0000","last_became_active":"2024-04-16T05:32:34.753814+0000","last_became_peered":"2024-04-16T05:32:34.753814+0000","last_unstale":"2024-04-16T05:34:02.937007+0000","last_undegraded":"2024-04-16T05:34:02.937007+0000","last_fullsized":"2024-04-16T05:34:02.937007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:13:00.207978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762389+0000","last_change":"2024-04-16T05:32:38.938563+0000","last_active":"2024-04-16T05:33:34.762389+0000","last_peered":"2024-04-16T05:33:34.762389+0000","last_clean":"2024-04-16T05:33:34.762389+0000","last_became_active":"2024-04-16T05:32:38.938097+0000","last_became_peered":"2024-04-16T05:32:38.938097+0000","last_unstale":"2024-04-16T05:33:34.762389+0000","last_undegraded":"2024-04-16T05:33:34.762389+0000","last_fullsized":"2024-04-16T05:33:34.762389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:47:52.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761063+0000","last_change":"2024-04-16T05:32:33.735694+0000","last_active":"2024-04-16T05:33:34.761063+0000","last_peered":"2024-04-16T05:33:34.761063+0000","last_clean":"2024-04-16T05:33:34.761063+0000","last_became_active":"2024-04-16T05:32:33.735414+0000","last_became_peered":"2024-04-16T05:32:33.735414+0000","last_unstale":"2024-04-16T05:33:34.761063+0000","last_undegraded":"2024-04-16T05:33:34.761063+0000","last_fullsized":"2024-04-16T05:33:34.761063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:19:06.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.933441+0000","last_change":"2024-04-16T05:32:34.762315+0000","last_active":"2024-04-16T05:34:02.933441+0000","last_peered":"2024-04-16T05:34:02.933441+0000","last_clean":"2024-04-16T05:34:02.933441+0000","last_became_active":"2024-04-16T05:32:34.761969+0000","last_became_peered":"2024-04-16T05:32:34.761969+0000","last_unstale":"2024-04-16T05:34:02.933441+0000","last_undegraded":"2024-04-16T05:34:02.933441+0000","last_fullsized":"2024-04-16T05:34:02.933441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:00:45.166966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763444+0000","last_change":"2024-04-16T05:32:38.940681+0000","last_active":"2024-04-16T05:33:34.763444+0000","last_peered":"2024-04-16T05:33:34.763444+0000","last_clean":"2024-04-16T05:33:34.763444+0000","last_became_active":"2024-04-16T05:32:38.940414+0000","last_became_peered":"2024-04-16T05:32:38.940414+0000","last_unstale":"2024-04-16T05:33:34.763444+0000","last_undegraded":"2024-04-16T05:33:34.763444+0000","last_fullsized":"2024-04-16T05:33:34.763444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:46:50.061607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.771469+0000","last_change":"2024-04-16T05:32:33.748333+0000","last_active":"2024-04-16T05:33:34.771469+0000","last_peered":"2024-04-16T05:33:34.771469+0000","last_clean":"2024-04-16T05:33:34.771469+0000","last_became_active":"2024-04-16T05:32:33.748022+0000","last_became_peered":"2024-04-16T05:32:33.748022+0000","last_unstale":"2024-04-16T05:33:34.771469+0000","last_undegraded":"2024-04-16T05:33:34.771469+0000","last_fullsized":"2024-04-16T05:33:34.771469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:50:27.771245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.939375+0000","last_change":"2024-04-16T05:32:34.764832+0000","last_active":"2024-04-16T05:34:02.939375+0000","last_peered":"2024-04-16T05:34:02.939375+0000","last_clean":"2024-04-16T05:34:02.939375+0000","last_became_active":"2024-04-16T05:32:34.763180+0000","last_became_peered":"2024-04-16T05:32:34.763180+0000","last_unstale":"2024-04-16T05:34:02.939375+0000","last_undegraded":"2024-04-16T05:34:02.939375+0000","last_fullsized":"2024-04-16T05:34:02.939375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:37:27.804376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763690+0000","last_change":"2024-04-16T05:32:38.937226+0000","last_active":"2024-04-16T05:33:34.763690+0000","last_peered":"2024-04-16T05:33:34.763690+0000","last_clean":"2024-04-16T05:33:34.763690+0000","last_became_active":"2024-04-16T05:32:38.936770+0000","last_became_peered":"2024-04-16T05:32:38.936770+0000","last_unstale":"2024-04-16T05:33:34.763690+0000","last_undegraded":"2024-04-16T05:33:34.763690+0000","last_fullsized":"2024-04-16T05:33:34.763690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:05:28.059175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.776201+0000","last_change":"2024-04-16T05:33:34.775689+0000","last_active":"2024-04-16T05:33:34.776201+0000","last_peered":"2024-04-16T05:33:34.776201+0000","last_clean":"2024-04-16T05:33:34.776201+0000","last_became_active":"2024-04-16T05:33:34.770520+0000","last_became_peered":"2024-04-16T05:33:34.770520+0000","last_unstale":"2024-04-16T05:33:34.776201+0000","last_undegraded":"2024-04-16T05:33:34.776201+0000","last_fullsized":"2024-04-16T05:33:34.776201+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:40:57.616795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948895+0000","last_change":"2024-04-16T05:32:34.758891+0000","last_active":"2024-04-16T05:34:02.948895+0000","last_peered":"2024-04-16T05:34:02.948895+0000","last_clean":"2024-04-16T05:34:02.948895+0000","last_became_active":"2024-04-16T05:32:34.758559+0000","last_became_peered":"2024-04-16T05:32:34.758559+0000","last_unstale":"2024-04-16T05:34:02.948895+0000","last_undegraded":"2024-04-16T05:34:02.948895+0000","last_fullsized":"2024-04-16T05:34:02.948895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:31.213992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763130+0000","last_change":"2024-04-16T05:32:38.937163+0000","last_active":"2024-04-16T05:33:34.763130+0000","last_peered":"2024-04-16T05:33:34.763130+0000","last_clean":"2024-04-16T05:33:34.763130+0000","last_became_active":"2024-04-16T05:32:38.936745+0000","last_became_peered":"2024-04-16T05:32:38.936745+0000","last_unstale":"2024-04-16T05:33:34.763130+0000","last_undegraded":"2024-04-16T05:33:34.763130+0000","last_fullsized":"2024-04-16T05:33:34.763130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:54:37.915041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763843+0000","last_change":"2024-04-16T05:32:33.731842+0000","last_active":"2024-04-16T05:33:34.763843+0000","last_peered":"2024-04-16T05:33:34.763843+0000","last_clean":"2024-04-16T05:33:34.763843+0000","last_became_active":"2024-04-16T05:32:33.731417+0000","last_became_peered":"2024-04-16T05:32:33.731417+0000","last_unstale":"2024-04-16T05:33:34.763843+0000","last_undegraded":"2024-04-16T05:33:34.763843+0000","last_fullsized":"2024-04-16T05:33:34.763843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:04:03.141533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.933420+0000","last_change":"2024-04-16T05:32:34.755062+0000","last_active":"2024-04-16T05:34:02.933420+0000","last_peered":"2024-04-16T05:34:02.933420+0000","last_clean":"2024-04-16T05:34:02.933420+0000","last_became_active":"2024-04-16T05:32:34.754759+0000","last_became_peered":"2024-04-16T05:32:34.754759+0000","last_unstale":"2024-04-16T05:34:02.933420+0000","last_undegraded":"2024-04-16T05:34:02.933420+0000","last_fullsized":"2024-04-16T05:34:02.933420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:53:59.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762985+0000","last_change":"2024-04-16T05:32:38.935646+0000","last_active":"2024-04-16T05:33:34.762985+0000","last_peered":"2024-04-16T05:33:34.762985+0000","last_clean":"2024-04-16T05:33:34.762985+0000","last_became_active":"2024-04-16T05:32:38.935206+0000","last_became_peered":"2024-04-16T05:32:38.935206+0000","last_unstale":"2024-04-16T05:33:34.762985+0000","last_undegraded":"2024-04-16T05:33:34.762985+0000","last_fullsized":"2024-04-16T05:33:34.762985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:09.772136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:10.806378+0000","last_change":"2024-04-16T05:32:33.739343+0000","last_active":"2024-04-16T05:34:10.806378+0000","last_peered":"2024-04-16T05:34:10.806378+0000","last_clean":"2024-04-16T05:34:10.806378+0000","last_became_active":"2024-04-16T05:32:33.738982+0000","last_became_peered":"2024-04-16T05:32:33.738982+0000","last_unstale":"2024-04-16T05:34:10.806378+0000","last_undegraded":"2024-04-16T05:34:10.806378+0000","last_fullsized":"2024-04-16T05:34:10.806378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:36.002804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.924212+0000","last_change":"2024-04-16T05:32:34.749316+0000","last_active":"2024-04-16T05:34:02.924212+0000","last_peered":"2024-04-16T05:34:02.924212+0000","last_clean":"2024-04-16T05:34:02.924212+0000","last_became_active":"2024-04-16T05:32:34.748976+0000","last_became_peered":"2024-04-16T05:32:34.748976+0000","last_unstale":"2024-04-16T05:34:02.924212+0000","last_undegraded":"2024-04-16T05:34:02.924212+0000","last_fullsized":"2024-04-16T05:34:02.924212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:16:29.569402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762740+0000","last_change":"2024-04-16T05:32:38.938633+0000","last_active":"2024-04-16T05:33:34.762740+0000","last_peered":"2024-04-16T05:33:34.762740+0000","last_clean":"2024-04-16T05:33:34.762740+0000","last_became_active":"2024-04-16T05:32:38.938215+0000","last_became_peered":"2024-04-16T05:32:38.938215+0000","last_unstale":"2024-04-16T05:33:34.762740+0000","last_undegraded":"2024-04-16T05:33:34.762740+0000","last_fullsized":"2024-04-16T05:33:34.762740+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:05.078109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.768181+0000","last_change":"2024-04-16T05:32:33.744262+0000","last_active":"2024-04-16T05:33:34.768181+0000","last_peered":"2024-04-16T05:33:34.768181+0000","last_clean":"2024-04-16T05:33:34.768181+0000","last_became_active":"2024-04-16T05:32:33.743866+0000","last_became_peered":"2024-04-16T05:32:33.743866+0000","last_unstale":"2024-04-16T05:33:34.768181+0000","last_undegraded":"2024-04-16T05:33:34.768181+0000","last_fullsized":"2024-04-16T05:33:34.768181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:22:03.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.934939+0000","last_change":"2024-04-16T05:32:34.750201+0000","last_active":"2024-04-16T05:34:02.934939+0000","last_peered":"2024-04-16T05:34:02.934939+0000","last_clean":"2024-04-16T05:34:02.934939+0000","last_became_active":"2024-04-16T05:32:34.749900+0000","last_became_peered":"2024-04-16T05:32:34.749900+0000","last_unstale":"2024-04-16T05:34:02.934939+0000","last_undegraded":"2024-04-16T05:34:02.934939+0000","last_fullsized":"2024-04-16T05:34:02.934939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:42.708584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.767259+0000","last_change":"2024-04-16T05:32:38.931277+0000","last_active":"2024-04-16T05:33:34.767259+0000","last_peered":"2024-04-16T05:33:34.767259+0000","last_clean":"2024-04-16T05:33:34.767259+0000","last_became_active":"2024-04-16T05:32:38.930888+0000","last_became_peered":"2024-04-16T05:32:38.930888+0000","last_unstale":"2024-04-16T05:33:34.767259+0000","last_undegraded":"2024-04-16T05:33:34.767259+0000","last_fullsized":"2024-04-16T05:33:34.767259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:27:31.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763925+0000","last_change":"2024-04-16T05:32:33.747740+0000","last_active":"2024-04-16T05:33:34.763925+0000","last_peered":"2024-04-16T05:33:34.763925+0000","last_clean":"2024-04-16T05:33:34.763925+0000","last_became_active":"2024-04-16T05:32:33.747313+0000","last_became_peered":"2024-04-16T05:32:33.747313+0000","last_unstale":"2024-04-16T05:33:34.763925+0000","last_undegraded":"2024-04-16T05:33:34.763925+0000","last_fullsized":"2024-04-16T05:33:34.763925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:05:03.937790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935630+0000","last_change":"2024-04-16T05:32:34.760075+0000","last_active":"2024-04-16T05:34:02.935630+0000","last_peered":"2024-04-16T05:34:02.935630+0000","last_clean":"2024-04-16T05:34:02.935630+0000","last_became_active":"2024-04-16T05:32:34.759569+0000","last_became_peered":"2024-04-16T05:32:34.759569+0000","last_unstale":"2024-04-16T05:34:02.935630+0000","last_undegraded":"2024-04-16T05:34:02.935630+0000","last_fullsized":"2024-04-16T05:34:02.935630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:54.222877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764355+0000","last_change":"2024-04-16T05:32:38.928568+0000","last_active":"2024-04-16T05:33:34.764355+0000","last_peered":"2024-04-16T05:33:34.764355+0000","last_clean":"2024-04-16T05:33:34.764355+0000","last_became_active":"2024-04-16T05:32:38.928109+0000","last_became_peered":"2024-04-16T05:32:38.928109+0000","last_unstale":"2024-04-16T05:33:34.764355+0000","last_undegraded":"2024-04-16T05:33:34.764355+0000","last_fullsized":"2024-04-16T05:33:34.764355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:24:14.293528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.843483+0000","last_change":"2024-04-16T05:32:33.737803+0000","last_active":"2024-04-16T05:33:34.843483+0000","last_peered":"2024-04-16T05:33:34.843483+0000","last_clean":"2024-04-16T05:33:34.843483+0000","last_became_active":"2024-04-16T05:32:33.737448+0000","last_became_peered":"2024-04-16T05:32:33.737448+0000","last_unstale":"2024-04-16T05:33:34.843483+0000","last_undegraded":"2024-04-16T05:33:34.843483+0000","last_fullsized":"2024-04-16T05:33:34.843483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:44:54.302323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.934113+0000","last_change":"2024-04-16T05:32:34.754070+0000","last_active":"2024-04-16T05:34:02.934113+0000","last_peered":"2024-04-16T05:34:02.934113+0000","last_clean":"2024-04-16T05:34:02.934113+0000","last_became_active":"2024-04-16T05:32:34.753717+0000","last_became_peered":"2024-04-16T05:32:34.753717+0000","last_unstale":"2024-04-16T05:34:02.934113+0000","last_undegraded":"2024-04-16T05:34:02.934113+0000","last_fullsized":"2024-04-16T05:34:02.934113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:58:56.738245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769558+0000","last_change":"2024-04-16T05:32:38.931594+0000","last_active":"2024-04-16T05:33:34.769558+0000","last_peered":"2024-04-16T05:33:34.769558+0000","last_clean":"2024-04-16T05:33:34.769558+0000","last_became_active":"2024-04-16T05:32:38.931188+0000","last_became_peered":"2024-04-16T05:32:38.931188+0000","last_unstale":"2024-04-16T05:33:34.769558+0000","last_undegraded":"2024-04-16T05:33:34.769558+0000","last_fullsized":"2024-04-16T05:33:34.769558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:55.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:09.804046+0000","last_change":"2024-04-16T05:32:33.739752+0000","last_active":"2024-04-16T05:34:09.804046+0000","last_peered":"2024-04-16T05:34:09.804046+0000","last_clean":"2024-04-16T05:34:09.804046+0000","last_became_active":"2024-04-16T05:32:33.739444+0000","last_became_peered":"2024-04-16T05:32:33.739444+0000","last_unstale":"2024-04-16T05:34:09.804046+0000","last_undegraded":"2024-04-16T05:34:09.804046+0000","last_fullsized":"2024-04-16T05:34:09.804046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:04:18.091618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948687+0000","last_change":"2024-04-16T05:32:34.759197+0000","last_active":"2024-04-16T05:34:02.948687+0000","last_peered":"2024-04-16T05:34:02.948687+0000","last_clean":"2024-04-16T05:34:02.948687+0000","last_became_active":"2024-04-16T05:32:34.758783+0000","last_became_peered":"2024-04-16T05:32:34.758783+0000","last_unstale":"2024-04-16T05:34:02.948687+0000","last_undegraded":"2024-04-16T05:34:02.948687+0000","last_fullsized":"2024-04-16T05:34:02.948687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:23:37.020254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.760284+0000","last_change":"2024-04-16T05:32:38.936198+0000","last_active":"2024-04-16T05:33:34.760284+0000","last_peered":"2024-04-16T05:33:34.760284+0000","last_clean":"2024-04-16T05:33:34.760284+0000","last_became_active":"2024-04-16T05:32:38.935698+0000","last_became_peered":"2024-04-16T05:32:38.935698+0000","last_unstale":"2024-04-16T05:33:34.760284+0000","last_undegraded":"2024-04-16T05:33:34.760284+0000","last_fullsized":"2024-04-16T05:33:34.760284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:17:35.602408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769134+0000","last_change":"2024-04-16T05:32:33.737653+0000","last_active":"2024-04-16T05:33:34.769134+0000","last_peered":"2024-04-16T05:33:34.769134+0000","last_clean":"2024-04-16T05:33:34.769134+0000","last_became_active":"2024-04-16T05:32:33.737342+0000","last_became_peered":"2024-04-16T05:32:33.737342+0000","last_unstale":"2024-04-16T05:33:34.769134+0000","last_undegraded":"2024-04-16T05:33:34.769134+0000","last_fullsized":"2024-04-16T05:33:34.769134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:17:35.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938193+0000","last_change":"2024-04-16T05:32:34.765722+0000","last_active":"2024-04-16T05:34:02.938193+0000","last_peered":"2024-04-16T05:34:02.938193+0000","last_clean":"2024-04-16T05:34:02.938193+0000","last_became_active":"2024-04-16T05:32:34.765413+0000","last_became_peered":"2024-04-16T05:32:34.765413+0000","last_unstale":"2024-04-16T05:34:02.938193+0000","last_undegraded":"2024-04-16T05:34:02.938193+0000","last_fullsized":"2024-04-16T05:34:02.938193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:03.884041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764121+0000","last_change":"2024-04-16T05:32:38.936186+0000","last_active":"2024-04-16T05:33:34.764121+0000","last_peered":"2024-04-16T05:33:34.764121+0000","last_clean":"2024-04-16T05:33:34.764121+0000","last_became_active":"2024-04-16T05:32:38.935738+0000","last_became_peered":"2024-04-16T05:32:38.935738+0000","last_unstale":"2024-04-16T05:33:34.764121+0000","last_undegraded":"2024-04-16T05:33:34.764121+0000","last_fullsized":"2024-04-16T05:33:34.764121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:10.519095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.844163+0000","last_change":"2024-04-16T05:32:33.742991+0000","last_active":"2024-04-16T05:33:34.844163+0000","last_peered":"2024-04-16T05:33:34.844163+0000","last_clean":"2024-04-16T05:33:34.844163+0000","last_became_active":"2024-04-16T05:32:33.742659+0000","last_became_peered":"2024-04-16T05:32:33.742659+0000","last_unstale":"2024-04-16T05:33:34.844163+0000","last_undegraded":"2024-04-16T05:33:34.844163+0000","last_fullsized":"2024-04-16T05:33:34.844163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:23.756123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.948969+0000","last_change":"2024-04-16T05:32:34.759926+0000","last_active":"2024-04-16T05:34:02.948969+0000","last_peered":"2024-04-16T05:34:02.948969+0000","last_clean":"2024-04-16T05:34:02.948969+0000","last_became_active":"2024-04-16T05:32:34.759626+0000","last_became_peered":"2024-04-16T05:32:34.759626+0000","last_unstale":"2024-04-16T05:34:02.948969+0000","last_undegraded":"2024-04-16T05:34:02.948969+0000","last_fullsized":"2024-04-16T05:34:02.948969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:50.314638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.760612+0000","last_change":"2024-04-16T05:32:38.942135+0000","last_active":"2024-04-16T05:33:34.760612+0000","last_peered":"2024-04-16T05:33:34.760612+0000","last_clean":"2024-04-16T05:33:34.760612+0000","last_became_active":"2024-04-16T05:32:38.941534+0000","last_became_peered":"2024-04-16T05:32:38.941534+0000","last_unstale":"2024-04-16T05:33:34.760612+0000","last_undegraded":"2024-04-16T05:33:34.760612+0000","last_fullsized":"2024-04-16T05:33:34.760612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:20.494516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:10.805462+0000","last_change":"2024-04-16T05:32:33.734275+0000","last_active":"2024-04-16T05:34:10.805462+0000","last_peered":"2024-04-16T05:34:10.805462+0000","last_clean":"2024-04-16T05:34:10.805462+0000","last_became_active":"2024-04-16T05:32:33.733906+0000","last_became_peered":"2024-04-16T05:32:33.733906+0000","last_unstale":"2024-04-16T05:34:10.805462+0000","last_undegraded":"2024-04-16T05:34:10.805462+0000","last_fullsized":"2024-04-16T05:34:10.805462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:39:23.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":160,"num_read_kb":141,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.930603+0000","last_change":"2024-04-16T05:32:34.750179+0000","last_active":"2024-04-16T05:34:02.930603+0000","last_peered":"2024-04-16T05:34:02.930603+0000","last_clean":"2024-04-16T05:34:02.930603+0000","last_became_active":"2024-04-16T05:32:34.749773+0000","last_became_peered":"2024-04-16T05:32:34.749773+0000","last_unstale":"2024-04-16T05:34:02.930603+0000","last_undegraded":"2024-04-16T05:34:02.930603+0000","last_fullsized":"2024-04-16T05:34:02.930603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:37.621191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.844475+0000","last_change":"2024-04-16T05:32:38.935690+0000","last_active":"2024-04-16T05:33:34.844475+0000","last_peered":"2024-04-16T05:33:34.844475+0000","last_clean":"2024-04-16T05:33:34.844475+0000","last_became_active":"2024-04-16T05:32:38.935282+0000","last_became_peered":"2024-04-16T05:32:38.935282+0000","last_unstale":"2024-04-16T05:33:34.844475+0000","last_undegraded":"2024-04-16T05:33:34.844475+0000","last_fullsized":"2024-04-16T05:33:34.844475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:25:00.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.782574+0000","last_change":"2024-04-16T05:33:34.782195+0000","last_active":"2024-04-16T05:33:34.782574+0000","last_peered":"2024-04-16T05:33:34.782574+0000","last_clean":"2024-04-16T05:33:34.782574+0000","last_became_active":"2024-04-16T05:33:34.771490+0000","last_became_peered":"2024-04-16T05:33:34.771490+0000","last_unstale":"2024-04-16T05:33:34.782574+0000","last_undegraded":"2024-04-16T05:33:34.782574+0000","last_fullsized":"2024-04-16T05:33:34.782574+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:52.513976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.941038+0000","last_change":"2024-04-16T05:32:34.748760+0000","last_active":"2024-04-16T05:34:02.941038+0000","last_peered":"2024-04-16T05:34:02.941038+0000","last_clean":"2024-04-16T05:34:02.941038+0000","last_became_active":"2024-04-16T05:32:34.748432+0000","last_became_peered":"2024-04-16T05:32:34.748432+0000","last_unstale":"2024-04-16T05:34:02.941038+0000","last_undegraded":"2024-04-16T05:34:02.941038+0000","last_fullsized":"2024-04-16T05:34:02.941038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:12:42.859342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.765741+0000","last_change":"2024-04-16T05:32:38.929705+0000","last_active":"2024-04-16T05:33:34.765741+0000","last_peered":"2024-04-16T05:33:34.765741+0000","last_clean":"2024-04-16T05:33:34.765741+0000","last_became_active":"2024-04-16T05:32:38.929282+0000","last_became_peered":"2024-04-16T05:32:38.929282+0000","last_unstale":"2024-04-16T05:33:34.765741+0000","last_undegraded":"2024-04-16T05:33:34.765741+0000","last_fullsized":"2024-04-16T05:33:34.765741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:51:27.763600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.843796+0000","last_change":"2024-04-16T05:32:33.746371+0000","last_active":"2024-04-16T05:33:34.843796+0000","last_peered":"2024-04-16T05:33:34.843796+0000","last_clean":"2024-04-16T05:33:34.843796+0000","last_became_active":"2024-04-16T05:32:33.746077+0000","last_became_peered":"2024-04-16T05:32:33.746077+0000","last_unstale":"2024-04-16T05:33:34.843796+0000","last_undegraded":"2024-04-16T05:33:34.843796+0000","last_fullsized":"2024-04-16T05:33:34.843796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:57:43.193942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.944369+0000","last_change":"2024-04-16T05:32:34.753196+0000","last_active":"2024-04-16T05:34:02.944369+0000","last_peered":"2024-04-16T05:34:02.944369+0000","last_clean":"2024-04-16T05:34:02.944369+0000","last_became_active":"2024-04-16T05:32:34.752881+0000","last_became_peered":"2024-04-16T05:32:34.752881+0000","last_unstale":"2024-04-16T05:34:02.944369+0000","last_undegraded":"2024-04-16T05:34:02.944369+0000","last_fullsized":"2024-04-16T05:34:02.944369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:35:56.715394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.773192+0000","last_change":"2024-04-16T05:33:34.772562+0000","last_active":"2024-04-16T05:33:34.773192+0000","last_peered":"2024-04-16T05:33:34.773192+0000","last_clean":"2024-04-16T05:33:34.773192+0000","last_became_active":"2024-04-16T05:33:34.772052+0000","last_became_peered":"2024-04-16T05:33:34.772052+0000","last_unstale":"2024-04-16T05:33:34.773192+0000","last_undegraded":"2024-04-16T05:33:34.773192+0000","last_fullsized":"2024-04-16T05:33:34.773192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:50:18.425870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:09.804649+0000","last_change":"2024-04-16T05:32:33.735921+0000","last_active":"2024-04-16T05:34:09.804649+0000","last_peered":"2024-04-16T05:34:09.804649+0000","last_clean":"2024-04-16T05:34:09.804649+0000","last_became_active":"2024-04-16T05:32:33.735572+0000","last_became_peered":"2024-04-16T05:32:33.735572+0000","last_unstale":"2024-04-16T05:34:09.804649+0000","last_undegraded":"2024-04-16T05:34:09.804649+0000","last_fullsized":"2024-04-16T05:34:09.804649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:46:56.512827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.932132+0000","last_change":"2024-04-16T05:32:34.762172+0000","last_active":"2024-04-16T05:34:02.932132+0000","last_peered":"2024-04-16T05:34:02.932132+0000","last_clean":"2024-04-16T05:34:02.932132+0000","last_became_active":"2024-04-16T05:32:34.761914+0000","last_became_peered":"2024-04-16T05:32:34.761914+0000","last_unstale":"2024-04-16T05:34:02.932132+0000","last_undegraded":"2024-04-16T05:34:02.932132+0000","last_fullsized":"2024-04-16T05:34:02.932132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:47:52.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.772546+0000","last_change":"2024-04-16T05:33:34.772546+0000","last_active":"2024-04-16T05:33:34.772546+0000","last_peered":"2024-04-16T05:33:34.772546+0000","last_clean":"2024-04-16T05:33:34.772546+0000","last_became_active":"2024-04-16T05:33:34.772100+0000","last_became_peered":"2024-04-16T05:33:34.772100+0000","last_unstale":"2024-04-16T05:33:34.772546+0000","last_undegraded":"2024-04-16T05:33:34.772546+0000","last_fullsized":"2024-04-16T05:33:34.772546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:11:50.157129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763134+0000","last_change":"2024-04-16T05:32:33.743944+0000","last_active":"2024-04-16T05:33:34.763134+0000","last_peered":"2024-04-16T05:33:34.763134+0000","last_clean":"2024-04-16T05:33:34.763134+0000","last_became_active":"2024-04-16T05:32:33.743559+0000","last_became_peered":"2024-04-16T05:32:33.743559+0000","last_unstale":"2024-04-16T05:33:34.763134+0000","last_undegraded":"2024-04-16T05:33:34.763134+0000","last_fullsized":"2024-04-16T05:33:34.763134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:08:48.889356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.931912+0000","last_change":"2024-04-16T05:32:34.756741+0000","last_active":"2024-04-16T05:34:02.931912+0000","last_peered":"2024-04-16T05:34:02.931912+0000","last_clean":"2024-04-16T05:34:02.931912+0000","last_became_active":"2024-04-16T05:32:34.756426+0000","last_became_peered":"2024-04-16T05:32:34.756426+0000","last_unstale":"2024-04-16T05:34:02.931912+0000","last_undegraded":"2024-04-16T05:34:02.931912+0000","last_fullsized":"2024-04-16T05:34:02.931912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:09:30.493857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.843530+0000","last_change":"2024-04-16T05:32:38.934818+0000","last_active":"2024-04-16T05:33:34.843530+0000","last_peered":"2024-04-16T05:33:34.843530+0000","last_clean":"2024-04-16T05:33:34.843530+0000","last_became_active":"2024-04-16T05:32:38.934335+0000","last_became_peered":"2024-04-16T05:32:38.934335+0000","last_unstale":"2024-04-16T05:33:34.843530+0000","last_undegraded":"2024-04-16T05:33:34.843530+0000","last_fullsized":"2024-04-16T05:33:34.843530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:02:41.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764985+0000","last_change":"2024-04-16T05:32:33.731951+0000","last_active":"2024-04-16T05:33:34.764985+0000","last_peered":"2024-04-16T05:33:34.764985+0000","last_clean":"2024-04-16T05:33:34.764985+0000","last_became_active":"2024-04-16T05:32:33.731649+0000","last_became_peered":"2024-04-16T05:32:33.731649+0000","last_unstale":"2024-04-16T05:33:34.764985+0000","last_undegraded":"2024-04-16T05:33:34.764985+0000","last_fullsized":"2024-04-16T05:33:34.764985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:41.258012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.931179+0000","last_change":"2024-04-16T05:32:34.751884+0000","last_active":"2024-04-16T05:34:02.931179+0000","last_peered":"2024-04-16T05:34:02.931179+0000","last_clean":"2024-04-16T05:34:02.931179+0000","last_became_active":"2024-04-16T05:32:34.751559+0000","last_became_peered":"2024-04-16T05:32:34.751559+0000","last_unstale":"2024-04-16T05:34:02.931179+0000","last_undegraded":"2024-04-16T05:34:02.931179+0000","last_fullsized":"2024-04-16T05:34:02.931179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:32:59.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763529+0000","last_change":"2024-04-16T05:32:39.109330+0000","last_active":"2024-04-16T05:33:34.763529+0000","last_peered":"2024-04-16T05:33:34.763529+0000","last_clean":"2024-04-16T05:33:34.763529+0000","last_became_active":"2024-04-16T05:32:39.108718+0000","last_became_peered":"2024-04-16T05:32:39.108718+0000","last_unstale":"2024-04-16T05:33:34.763529+0000","last_undegraded":"2024-04-16T05:33:34.763529+0000","last_fullsized":"2024-04-16T05:33:34.763529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:36:55.418034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761016+0000","last_change":"2024-04-16T05:32:33.741603+0000","last_active":"2024-04-16T05:33:34.761016+0000","last_peered":"2024-04-16T05:33:34.761016+0000","last_clean":"2024-04-16T05:33:34.761016+0000","last_became_active":"2024-04-16T05:32:33.741184+0000","last_became_peered":"2024-04-16T05:32:33.741184+0000","last_unstale":"2024-04-16T05:33:34.761016+0000","last_undegraded":"2024-04-16T05:33:34.761016+0000","last_fullsized":"2024-04-16T05:33:34.761016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:08:38.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.941772+0000","last_change":"2024-04-16T05:32:34.747768+0000","last_active":"2024-04-16T05:34:02.941772+0000","last_peered":"2024-04-16T05:34:02.941772+0000","last_clean":"2024-04-16T05:34:02.941772+0000","last_became_active":"2024-04-16T05:32:34.747440+0000","last_became_peered":"2024-04-16T05:32:34.747440+0000","last_unstale":"2024-04-16T05:34:02.941772+0000","last_undegraded":"2024-04-16T05:34:02.941772+0000","last_fullsized":"2024-04-16T05:34:02.941772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:14:31.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763872+0000","last_change":"2024-04-16T05:32:38.938673+0000","last_active":"2024-04-16T05:33:34.763872+0000","last_peered":"2024-04-16T05:33:34.763872+0000","last_clean":"2024-04-16T05:33:34.763872+0000","last_became_active":"2024-04-16T05:32:38.938252+0000","last_became_peered":"2024-04-16T05:32:38.938252+0000","last_unstale":"2024-04-16T05:33:34.763872+0000","last_undegraded":"2024-04-16T05:33:34.763872+0000","last_fullsized":"2024-04-16T05:33:34.763872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:06.620693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.768545+0000","last_change":"2024-04-16T05:32:33.750812+0000","last_active":"2024-04-16T05:33:34.768545+0000","last_peered":"2024-04-16T05:33:34.768545+0000","last_clean":"2024-04-16T05:33:34.768545+0000","last_became_active":"2024-04-16T05:32:33.750514+0000","last_became_peered":"2024-04-16T05:32:33.750514+0000","last_unstale":"2024-04-16T05:33:34.768545+0000","last_undegraded":"2024-04-16T05:33:34.768545+0000","last_fullsized":"2024-04-16T05:33:34.768545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:53:43.075914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.931534+0000","last_change":"2024-04-16T05:33:34.770449+0000","last_active":"2024-04-16T05:34:02.931534+0000","last_peered":"2024-04-16T05:34:02.931534+0000","last_clean":"2024-04-16T05:34:02.931534+0000","last_became_active":"2024-04-16T05:33:34.770078+0000","last_became_peered":"2024-04-16T05:33:34.770078+0000","last_unstale":"2024-04-16T05:34:02.931534+0000","last_undegraded":"2024-04-16T05:34:02.931534+0000","last_fullsized":"2024-04-16T05:34:02.931534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:30:20.885002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763421+0000","last_change":"2024-04-16T05:32:39.109394+0000","last_active":"2024-04-16T05:33:34.763421+0000","last_peered":"2024-04-16T05:33:34.763421+0000","last_clean":"2024-04-16T05:33:34.763421+0000","last_became_active":"2024-04-16T05:32:39.108924+0000","last_became_peered":"2024-04-16T05:32:39.108924+0000","last_unstale":"2024-04-16T05:33:34.763421+0000","last_undegraded":"2024-04-16T05:33:34.763421+0000","last_fullsized":"2024-04-16T05:33:34.763421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:50:37.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.934717+0000","last_change":"2024-04-16T05:33:34.766071+0000","last_active":"2024-04-16T05:34:02.934717+0000","last_peered":"2024-04-16T05:34:02.934717+0000","last_clean":"2024-04-16T05:34:02.934717+0000","last_became_active":"2024-04-16T05:33:34.765670+0000","last_became_peered":"2024-04-16T05:33:34.765670+0000","last_unstale":"2024-04-16T05:34:02.934717+0000","last_undegraded":"2024-04-16T05:34:02.934717+0000","last_fullsized":"2024-04-16T05:34:02.934717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:54:47.320828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:06.799587+0000","last_change":"2024-04-16T05:32:33.748521+0000","last_active":"2024-04-16T05:34:06.799587+0000","last_peered":"2024-04-16T05:34:06.799587+0000","last_clean":"2024-04-16T05:34:06.799587+0000","last_became_active":"2024-04-16T05:32:33.748222+0000","last_became_peered":"2024-04-16T05:32:33.748222+0000","last_unstale":"2024-04-16T05:34:06.799587+0000","last_undegraded":"2024-04-16T05:34:06.799587+0000","last_fullsized":"2024-04-16T05:34:06.799587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:04.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769001+0000","last_change":"2024-04-16T05:32:38.935995+0000","last_active":"2024-04-16T05:33:34.769001+0000","last_peered":"2024-04-16T05:33:34.769001+0000","last_clean":"2024-04-16T05:33:34.769001+0000","last_became_active":"2024-04-16T05:32:38.935511+0000","last_became_peered":"2024-04-16T05:32:38.935511+0000","last_unstale":"2024-04-16T05:33:34.769001+0000","last_undegraded":"2024-04-16T05:33:34.769001+0000","last_fullsized":"2024-04-16T05:33:34.769001+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:17:08.144537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.932771+0000","last_change":"2024-04-16T05:32:34.760249+0000","last_active":"2024-04-16T05:34:02.932771+0000","last_peered":"2024-04-16T05:34:02.932771+0000","last_clean":"2024-04-16T05:34:02.932771+0000","last_became_active":"2024-04-16T05:32:34.759735+0000","last_became_peered":"2024-04-16T05:32:34.759735+0000","last_unstale":"2024-04-16T05:34:02.932771+0000","last_undegraded":"2024-04-16T05:34:02.932771+0000","last_fullsized":"2024-04-16T05:34:02.932771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:13.349558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763827+0000","last_change":"2024-04-16T05:32:33.731170+0000","last_active":"2024-04-16T05:33:34.763827+0000","last_peered":"2024-04-16T05:33:34.763827+0000","last_clean":"2024-04-16T05:33:34.763827+0000","last_became_active":"2024-04-16T05:32:33.730821+0000","last_became_peered":"2024-04-16T05:32:33.730821+0000","last_unstale":"2024-04-16T05:33:34.763827+0000","last_undegraded":"2024-04-16T05:33:34.763827+0000","last_fullsized":"2024-04-16T05:33:34.763827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:38:15.707590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":363,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.768475+0000","last_change":"2024-04-16T05:31:24.926319+0000","last_active":"2024-04-16T05:33:34.768475+0000","last_peered":"2024-04-16T05:33:34.768475+0000","last_clean":"2024-04-16T05:33:34.768475+0000","last_became_active":"2024-04-16T05:31:24.925936+0000","last_became_peered":"2024-04-16T05:31:24.925936+0000","last_unstale":"2024-04-16T05:33:34.768475+0000","last_undegraded":"2024-04-16T05:33:34.768475+0000","last_fullsized":"2024-04-16T05:33:34.768475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_clean_scrub_stamp":"2024-04-16T05:27:57.876673+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:28:19.212127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.844235+0000","last_change":"2024-04-16T05:32:39.110241+0000","last_active":"2024-04-16T05:33:34.844235+0000","last_peered":"2024-04-16T05:33:34.844235+0000","last_clean":"2024-04-16T05:33:34.844235+0000","last_became_active":"2024-04-16T05:32:39.109792+0000","last_became_peered":"2024-04-16T05:32:39.109792+0000","last_unstale":"2024-04-16T05:33:34.844235+0000","last_undegraded":"2024-04-16T05:33:34.844235+0000","last_fullsized":"2024-04-16T05:33:34.844235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:15:05.925370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935060+0000","last_change":"2024-04-16T05:32:34.751693+0000","last_active":"2024-04-16T05:34:02.935060+0000","last_peered":"2024-04-16T05:34:02.935060+0000","last_clean":"2024-04-16T05:34:02.935060+0000","last_became_active":"2024-04-16T05:32:34.751210+0000","last_became_peered":"2024-04-16T05:32:34.751210+0000","last_unstale":"2024-04-16T05:34:02.935060+0000","last_undegraded":"2024-04-16T05:34:02.935060+0000","last_fullsized":"2024-04-16T05:34:02.935060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:33:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.768542+0000","last_change":"2024-04-16T05:32:33.738310+0000","last_active":"2024-04-16T05:33:34.768542+0000","last_peered":"2024-04-16T05:33:34.768542+0000","last_clean":"2024-04-16T05:33:34.768542+0000","last_became_active":"2024-04-16T05:32:33.738012+0000","last_became_peered":"2024-04-16T05:32:33.738012+0000","last_unstale":"2024-04-16T05:33:34.768542+0000","last_undegraded":"2024-04-16T05:33:34.768542+0000","last_fullsized":"2024-04-16T05:33:34.768542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:22:18.311458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.766912+0000","last_change":"2024-04-16T05:32:38.933465+0000","last_active":"2024-04-16T05:33:34.766912+0000","last_peered":"2024-04-16T05:33:34.766912+0000","last_clean":"2024-04-16T05:33:34.766912+0000","last_became_active":"2024-04-16T05:32:38.932391+0000","last_became_peered":"2024-04-16T05:32:38.932391+0000","last_unstale":"2024-04-16T05:33:34.766912+0000","last_undegraded":"2024-04-16T05:33:34.766912+0000","last_fullsized":"2024-04-16T05:33:34.766912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:39.494556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763957+0000","last_change":"2024-04-16T05:32:33.743241+0000","last_active":"2024-04-16T05:33:34.763957+0000","last_peered":"2024-04-16T05:33:34.763957+0000","last_clean":"2024-04-16T05:33:34.763957+0000","last_became_active":"2024-04-16T05:32:33.742887+0000","last_became_peered":"2024-04-16T05:32:33.742887+0000","last_unstale":"2024-04-16T05:33:34.763957+0000","last_undegraded":"2024-04-16T05:33:34.763957+0000","last_fullsized":"2024-04-16T05:33:34.763957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:24:38.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935151+0000","last_change":"2024-04-16T05:32:34.746228+0000","last_active":"2024-04-16T05:34:02.935151+0000","last_peered":"2024-04-16T05:34:02.935151+0000","last_clean":"2024-04-16T05:34:02.935151+0000","last_became_active":"2024-04-16T05:32:34.745883+0000","last_became_peered":"2024-04-16T05:32:34.745883+0000","last_unstale":"2024-04-16T05:34:02.935151+0000","last_undegraded":"2024-04-16T05:34:02.935151+0000","last_fullsized":"2024-04-16T05:34:02.935151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:33.656358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769577+0000","last_change":"2024-04-16T05:32:39.110156+0000","last_active":"2024-04-16T05:33:34.769577+0000","last_peered":"2024-04-16T05:33:34.769577+0000","last_clean":"2024-04-16T05:33:34.769577+0000","last_became_active":"2024-04-16T05:32:39.108958+0000","last_became_peered":"2024-04-16T05:32:39.108958+0000","last_unstale":"2024-04-16T05:33:34.769577+0000","last_undegraded":"2024-04-16T05:33:34.769577+0000","last_fullsized":"2024-04-16T05:33:34.769577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:27:58.658187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.760846+0000","last_change":"2024-04-16T05:32:33.738179+0000","last_active":"2024-04-16T05:33:34.760846+0000","last_peered":"2024-04-16T05:33:34.760846+0000","last_clean":"2024-04-16T05:33:34.760846+0000","last_became_active":"2024-04-16T05:32:33.737841+0000","last_became_peered":"2024-04-16T05:32:33.737841+0000","last_unstale":"2024-04-16T05:33:34.760846+0000","last_undegraded":"2024-04-16T05:33:34.760846+0000","last_fullsized":"2024-04-16T05:33:34.760846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:13:11.210559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935161+0000","last_change":"2024-04-16T05:32:34.758142+0000","last_active":"2024-04-16T05:34:02.935161+0000","last_peered":"2024-04-16T05:34:02.935161+0000","last_clean":"2024-04-16T05:34:02.935161+0000","last_became_active":"2024-04-16T05:32:34.757825+0000","last_became_peered":"2024-04-16T05:32:34.757825+0000","last_unstale":"2024-04-16T05:34:02.935161+0000","last_undegraded":"2024-04-16T05:34:02.935161+0000","last_fullsized":"2024-04-16T05:34:02.935161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:37.575764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761987+0000","last_change":"2024-04-16T05:32:38.933327+0000","last_active":"2024-04-16T05:33:34.761987+0000","last_peered":"2024-04-16T05:33:34.761987+0000","last_clean":"2024-04-16T05:33:34.761987+0000","last_became_active":"2024-04-16T05:32:38.932808+0000","last_became_peered":"2024-04-16T05:32:38.932808+0000","last_unstale":"2024-04-16T05:33:34.761987+0000","last_undegraded":"2024-04-16T05:33:34.761987+0000","last_fullsized":"2024-04-16T05:33:34.761987+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:03.217103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764455+0000","last_change":"2024-04-16T05:32:33.742405+0000","last_active":"2024-04-16T05:33:34.764455+0000","last_peered":"2024-04-16T05:33:34.764455+0000","last_clean":"2024-04-16T05:33:34.764455+0000","last_became_active":"2024-04-16T05:32:33.742051+0000","last_became_peered":"2024-04-16T05:32:33.742051+0000","last_unstale":"2024-04-16T05:33:34.764455+0000","last_undegraded":"2024-04-16T05:33:34.764455+0000","last_fullsized":"2024-04-16T05:33:34.764455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:35:13.914483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938554+0000","last_change":"2024-04-16T05:33:34.772430+0000","last_active":"2024-04-16T05:34:02.938554+0000","last_peered":"2024-04-16T05:34:02.938554+0000","last_clean":"2024-04-16T05:34:02.938554+0000","last_became_active":"2024-04-16T05:33:34.772067+0000","last_became_peered":"2024-04-16T05:33:34.772067+0000","last_unstale":"2024-04-16T05:34:02.938554+0000","last_undegraded":"2024-04-16T05:34:02.938554+0000","last_fullsized":"2024-04-16T05:34:02.938554+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:24.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.770714+0000","last_change":"2024-04-16T05:33:34.770714+0000","last_active":"2024-04-16T05:33:34.770714+0000","last_peered":"2024-04-16T05:33:34.770714+0000","last_clean":"2024-04-16T05:33:34.770714+0000","last_became_active":"2024-04-16T05:33:34.770249+0000","last_became_peered":"2024-04-16T05:33:34.770249+0000","last_unstale":"2024-04-16T05:33:34.770714+0000","last_undegraded":"2024-04-16T05:33:34.770714+0000","last_fullsized":"2024-04-16T05:33:34.770714+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:46.517492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761304+0000","last_change":"2024-04-16T05:32:33.743031+0000","last_active":"2024-04-16T05:33:34.761304+0000","last_peered":"2024-04-16T05:33:34.761304+0000","last_clean":"2024-04-16T05:33:34.761304+0000","last_became_active":"2024-04-16T05:32:33.742771+0000","last_became_peered":"2024-04-16T05:32:33.742771+0000","last_unstale":"2024-04-16T05:33:34.761304+0000","last_undegraded":"2024-04-16T05:33:34.761304+0000","last_fullsized":"2024-04-16T05:33:34.761304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:06.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.894157+0000","last_change":"2024-04-16T05:32:34.749428+0000","last_active":"2024-04-16T05:34:02.894157+0000","last_peered":"2024-04-16T05:34:02.894157+0000","last_clean":"2024-04-16T05:34:02.894157+0000","last_became_active":"2024-04-16T05:32:34.749087+0000","last_became_peered":"2024-04-16T05:32:34.749087+0000","last_unstale":"2024-04-16T05:34:02.894157+0000","last_undegraded":"2024-04-16T05:34:02.894157+0000","last_fullsized":"2024-04-16T05:34:02.894157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:59:25.124558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761056+0000","last_change":"2024-04-16T05:32:38.942407+0000","last_active":"2024-04-16T05:33:34.761056+0000","last_peered":"2024-04-16T05:33:34.761056+0000","last_clean":"2024-04-16T05:33:34.761056+0000","last_became_active":"2024-04-16T05:32:38.941731+0000","last_became_peered":"2024-04-16T05:32:38.941731+0000","last_unstale":"2024-04-16T05:33:34.761056+0000","last_undegraded":"2024-04-16T05:33:34.761056+0000","last_fullsized":"2024-04-16T05:33:34.761056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:42:27.700203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769319+0000","last_change":"2024-04-16T05:32:33.741487+0000","last_active":"2024-04-16T05:33:34.769319+0000","last_peered":"2024-04-16T05:33:34.769319+0000","last_clean":"2024-04-16T05:33:34.769319+0000","last_became_active":"2024-04-16T05:32:33.741140+0000","last_became_peered":"2024-04-16T05:32:33.741140+0000","last_unstale":"2024-04-16T05:33:34.769319+0000","last_undegraded":"2024-04-16T05:33:34.769319+0000","last_fullsized":"2024-04-16T05:33:34.769319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:21:36.086164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.939265+0000","last_change":"2024-04-16T05:32:34.756608+0000","last_active":"2024-04-16T05:34:02.939265+0000","last_peered":"2024-04-16T05:34:02.939265+0000","last_clean":"2024-04-16T05:34:02.939265+0000","last_became_active":"2024-04-16T05:32:34.756296+0000","last_became_peered":"2024-04-16T05:32:34.756296+0000","last_unstale":"2024-04-16T05:34:02.939265+0000","last_undegraded":"2024-04-16T05:34:02.939265+0000","last_fullsized":"2024-04-16T05:34:02.939265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:28.397333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764227+0000","last_change":"2024-04-16T05:32:38.929876+0000","last_active":"2024-04-16T05:33:34.764227+0000","last_peered":"2024-04-16T05:33:34.764227+0000","last_clean":"2024-04-16T05:33:34.764227+0000","last_became_active":"2024-04-16T05:32:38.928626+0000","last_became_peered":"2024-04-16T05:32:38.928626+0000","last_unstale":"2024-04-16T05:33:34.764227+0000","last_undegraded":"2024-04-16T05:33:34.764227+0000","last_fullsized":"2024-04-16T05:33:34.764227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:08:09.697413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762274+0000","last_change":"2024-04-16T05:32:33.736047+0000","last_active":"2024-04-16T05:33:34.762274+0000","last_peered":"2024-04-16T05:33:34.762274+0000","last_clean":"2024-04-16T05:33:34.762274+0000","last_became_active":"2024-04-16T05:32:33.735683+0000","last_became_peered":"2024-04-16T05:32:33.735683+0000","last_unstale":"2024-04-16T05:33:34.762274+0000","last_undegraded":"2024-04-16T05:33:34.762274+0000","last_fullsized":"2024-04-16T05:33:34.762274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:58:32.969924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.938725+0000","last_change":"2024-04-16T05:32:34.763558+0000","last_active":"2024-04-16T05:34:02.938725+0000","last_peered":"2024-04-16T05:34:02.938725+0000","last_clean":"2024-04-16T05:34:02.938725+0000","last_became_active":"2024-04-16T05:32:34.763190+0000","last_became_peered":"2024-04-16T05:32:34.763190+0000","last_unstale":"2024-04-16T05:34:02.938725+0000","last_undegraded":"2024-04-16T05:34:02.938725+0000","last_fullsized":"2024-04-16T05:34:02.938725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:38:31.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.764426+0000","last_change":"2024-04-16T05:32:39.109426+0000","last_active":"2024-04-16T05:33:34.764426+0000","last_peered":"2024-04-16T05:33:34.764426+0000","last_clean":"2024-04-16T05:33:34.764426+0000","last_became_active":"2024-04-16T05:32:39.108970+0000","last_became_peered":"2024-04-16T05:32:39.108970+0000","last_unstale":"2024-04-16T05:33:34.764426+0000","last_undegraded":"2024-04-16T05:33:34.764426+0000","last_fullsized":"2024-04-16T05:33:34.764426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:54.824654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763251+0000","last_change":"2024-04-16T05:32:33.734983+0000","last_active":"2024-04-16T05:33:34.763251+0000","last_peered":"2024-04-16T05:33:34.763251+0000","last_clean":"2024-04-16T05:33:34.763251+0000","last_became_active":"2024-04-16T05:32:33.734609+0000","last_became_peered":"2024-04-16T05:32:33.734609+0000","last_unstale":"2024-04-16T05:33:34.763251+0000","last_undegraded":"2024-04-16T05:33:34.763251+0000","last_fullsized":"2024-04-16T05:33:34.763251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:35:59.436499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.944339+0000","last_change":"2024-04-16T05:32:34.759811+0000","last_active":"2024-04-16T05:34:02.944339+0000","last_peered":"2024-04-16T05:34:02.944339+0000","last_clean":"2024-04-16T05:34:02.944339+0000","last_became_active":"2024-04-16T05:32:34.759499+0000","last_became_peered":"2024-04-16T05:32:34.759499+0000","last_unstale":"2024-04-16T05:34:02.944339+0000","last_undegraded":"2024-04-16T05:34:02.944339+0000","last_fullsized":"2024-04-16T05:34:02.944339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:46:28.554191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.761397+0000","last_change":"2024-04-16T05:32:38.929021+0000","last_active":"2024-04-16T05:33:34.761397+0000","last_peered":"2024-04-16T05:33:34.761397+0000","last_clean":"2024-04-16T05:33:34.761397+0000","last_became_active":"2024-04-16T05:32:38.928574+0000","last_became_peered":"2024-04-16T05:32:38.928574+0000","last_unstale":"2024-04-16T05:33:34.761397+0000","last_undegraded":"2024-04-16T05:33:34.761397+0000","last_fullsized":"2024-04-16T05:33:34.761397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:11:35.693364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.762928+0000","last_change":"2024-04-16T05:32:33.747352+0000","last_active":"2024-04-16T05:33:34.762928+0000","last_peered":"2024-04-16T05:33:34.762928+0000","last_clean":"2024-04-16T05:33:34.762928+0000","last_became_active":"2024-04-16T05:32:33.746989+0000","last_became_peered":"2024-04-16T05:32:33.746989+0000","last_unstale":"2024-04-16T05:33:34.762928+0000","last_undegraded":"2024-04-16T05:33:34.762928+0000","last_fullsized":"2024-04-16T05:33:34.762928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:38.422274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.944464+0000","last_change":"2024-04-16T05:32:34.760636+0000","last_active":"2024-04-16T05:34:02.944464+0000","last_peered":"2024-04-16T05:34:02.944464+0000","last_clean":"2024-04-16T05:34:02.944464+0000","last_became_active":"2024-04-16T05:32:34.760335+0000","last_became_peered":"2024-04-16T05:32:34.760335+0000","last_unstale":"2024-04-16T05:34:02.944464+0000","last_undegraded":"2024-04-16T05:34:02.944464+0000","last_fullsized":"2024-04-16T05:34:02.944464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:04:34.322799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.769486+0000","last_change":"2024-04-16T05:33:34.769486+0000","last_active":"2024-04-16T05:33:34.769486+0000","last_peered":"2024-04-16T05:33:34.769486+0000","last_clean":"2024-04-16T05:33:34.769486+0000","last_became_active":"2024-04-16T05:33:34.769025+0000","last_became_peered":"2024-04-16T05:33:34.769025+0000","last_unstale":"2024-04-16T05:33:34.769486+0000","last_undegraded":"2024-04-16T05:33:34.769486+0000","last_fullsized":"2024-04-16T05:33:34.769486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:10.382817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.844326+0000","last_change":"2024-04-16T05:32:33.746346+0000","last_active":"2024-04-16T05:33:34.844326+0000","last_peered":"2024-04-16T05:33:34.844326+0000","last_clean":"2024-04-16T05:33:34.844326+0000","last_became_active":"2024-04-16T05:32:33.746030+0000","last_became_peered":"2024-04-16T05:32:33.746030+0000","last_unstale":"2024-04-16T05:33:34.844326+0000","last_undegraded":"2024-04-16T05:33:34.844326+0000","last_fullsized":"2024-04-16T05:33:34.844326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:54:35.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.932765+0000","last_change":"2024-04-16T05:32:34.752283+0000","last_active":"2024-04-16T05:34:02.932765+0000","last_peered":"2024-04-16T05:34:02.932765+0000","last_clean":"2024-04-16T05:34:02.932765+0000","last_became_active":"2024-04-16T05:32:34.751904+0000","last_became_peered":"2024-04-16T05:32:34.751904+0000","last_unstale":"2024-04-16T05:34:02.932765+0000","last_undegraded":"2024-04-16T05:34:02.932765+0000","last_fullsized":"2024-04-16T05:34:02.932765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:15:52.688683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.843287+0000","last_change":"2024-04-16T05:32:38.934194+0000","last_active":"2024-04-16T05:33:34.843287+0000","last_peered":"2024-04-16T05:33:34.843287+0000","last_clean":"2024-04-16T05:33:34.843287+0000","last_became_active":"2024-04-16T05:32:38.933636+0000","last_became_peered":"2024-04-16T05:32:38.933636+0000","last_unstale":"2024-04-16T05:33:34.843287+0000","last_undegraded":"2024-04-16T05:33:34.843287+0000","last_fullsized":"2024-04-16T05:33:34.843287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:14:40.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:33:34.763544+0000","last_change":"2024-04-16T05:32:33.744267+0000","last_active":"2024-04-16T05:33:34.763544+0000","last_peered":"2024-04-16T05:33:34.763544+0000","last_clean":"2024-04-16T05:33:34.763544+0000","last_became_active":"2024-04-16T05:32:33.743931+0000","last_became_peered":"2024-04-16T05:32:33.743931+0000","last_unstale":"2024-04-16T05:33:34.763544+0000","last_undegraded":"2024-04-16T05:33:34.763544+0000","last_fullsized":"2024-04-16T05:33:34.763544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:30.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.937487+0000","last_change":"2024-04-16T05:32:34.756833+0000","last_active":"2024-04-16T05:34:02.937487+0000","last_peered":"2024-04-16T05:34:02.937487+0000","last_clean":"2024-04-16T05:34:02.937487+0000","last_became_active":"2024-04-16T05:32:34.756436+0000","last_became_peered":"2024-04-16T05:32:34.756436+0000","last_unstale":"2024-04-16T05:34:02.937487+0000","last_undegraded":"2024-04-16T05:34:02.937487+0000","last_fullsized":"2024-04-16T05:34:02.937487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:20.732946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935401+0000","last_change":"2024-04-16T05:32:34.752744+0000","last_active":"2024-04-16T05:34:02.935401+0000","last_peered":"2024-04-16T05:34:02.935401+0000","last_clean":"2024-04-16T05:34:02.935401+0000","last_became_active":"2024-04-16T05:32:34.752442+0000","last_became_peered":"2024-04-16T05:32:34.752442+0000","last_unstale":"2024-04-16T05:34:02.935401+0000","last_undegraded":"2024-04-16T05:34:02.935401+0000","last_fullsized":"2024-04-16T05:34:02.935401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:01:23.644692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.937616+0000","last_change":"2024-04-16T05:32:34.755118+0000","last_active":"2024-04-16T05:34:02.937616+0000","last_peered":"2024-04-16T05:34:02.937616+0000","last_clean":"2024-04-16T05:34:02.937616+0000","last_became_active":"2024-04-16T05:32:34.754824+0000","last_became_peered":"2024-04-16T05:32:34.754824+0000","last_unstale":"2024-04-16T05:34:02.937616+0000","last_undegraded":"2024-04-16T05:34:02.937616+0000","last_fullsized":"2024-04-16T05:34:02.937616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:58:37.860305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.939902+0000","last_change":"2024-04-16T05:32:34.756144+0000","last_active":"2024-04-16T05:34:02.939902+0000","last_peered":"2024-04-16T05:34:02.939902+0000","last_clean":"2024-04-16T05:34:02.939902+0000","last_became_active":"2024-04-16T05:32:34.755841+0000","last_became_peered":"2024-04-16T05:32:34.755841+0000","last_unstale":"2024-04-16T05:34:02.939902+0000","last_undegraded":"2024-04-16T05:34:02.939902+0000","last_fullsized":"2024-04-16T05:34:02.939902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:23:00.482444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.937769+0000","last_change":"2024-04-16T05:32:34.766204+0000","last_active":"2024-04-16T05:34:02.937769+0000","last_peered":"2024-04-16T05:34:02.937769+0000","last_clean":"2024-04-16T05:34:02.937769+0000","last_became_active":"2024-04-16T05:32:34.765709+0000","last_became_peered":"2024-04-16T05:32:34.765709+0000","last_unstale":"2024-04-16T05:34:02.937769+0000","last_undegraded":"2024-04-16T05:34:02.937769+0000","last_fullsized":"2024-04-16T05:34:02.937769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:23:53.926477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.940717+0000","last_change":"2024-04-16T05:32:34.758513+0000","last_active":"2024-04-16T05:34:02.940717+0000","last_peered":"2024-04-16T05:34:02.940717+0000","last_clean":"2024-04-16T05:34:02.940717+0000","last_became_active":"2024-04-16T05:32:34.758145+0000","last_became_peered":"2024-04-16T05:32:34.758145+0000","last_unstale":"2024-04-16T05:34:02.940717+0000","last_undegraded":"2024-04-16T05:34:02.940717+0000","last_fullsized":"2024-04-16T05:34:02.940717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:09:17.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.930796+0000","last_change":"2024-04-16T05:32:34.750651+0000","last_active":"2024-04-16T05:34:02.930796+0000","last_peered":"2024-04-16T05:34:02.930796+0000","last_clean":"2024-04-16T05:34:02.930796+0000","last_became_active":"2024-04-16T05:32:34.750348+0000","last_became_peered":"2024-04-16T05:32:34.750348+0000","last_unstale":"2024-04-16T05:34:02.930796+0000","last_undegraded":"2024-04-16T05:34:02.930796+0000","last_fullsized":"2024-04-16T05:34:02.930796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:29.282449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935433+0000","last_change":"2024-04-16T05:32:34.750079+0000","last_active":"2024-04-16T05:34:02.935433+0000","last_peered":"2024-04-16T05:34:02.935433+0000","last_clean":"2024-04-16T05:34:02.935433+0000","last_became_active":"2024-04-16T05:32:34.749775+0000","last_became_peered":"2024-04-16T05:32:34.749775+0000","last_unstale":"2024-04-16T05:34:02.935433+0000","last_undegraded":"2024-04-16T05:34:02.935433+0000","last_fullsized":"2024-04-16T05:34:02.935433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:02.297258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.930824+0000","last_change":"2024-04-16T05:32:34.752528+0000","last_active":"2024-04-16T05:34:02.930824+0000","last_peered":"2024-04-16T05:34:02.930824+0000","last_clean":"2024-04-16T05:34:02.930824+0000","last_became_active":"2024-04-16T05:32:34.752150+0000","last_became_peered":"2024-04-16T05:32:34.752150+0000","last_unstale":"2024-04-16T05:34:02.930824+0000","last_undegraded":"2024-04-16T05:34:02.930824+0000","last_fullsized":"2024-04-16T05:34:02.930824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:41:16.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.930666+0000","last_change":"2024-04-16T05:32:34.758486+0000","last_active":"2024-04-16T05:34:02.930666+0000","last_peered":"2024-04-16T05:34:02.930666+0000","last_clean":"2024-04-16T05:34:02.930666+0000","last_became_active":"2024-04-16T05:32:34.754840+0000","last_became_peered":"2024-04-16T05:32:34.754840+0000","last_unstale":"2024-04-16T05:34:02.930666+0000","last_undegraded":"2024-04-16T05:34:02.930666+0000","last_fullsized":"2024-04-16T05:34:02.930666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:35.644337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.935014+0000","last_change":"2024-04-16T05:32:34.758933+0000","last_active":"2024-04-16T05:34:02.935014+0000","last_peered":"2024-04-16T05:34:02.935014+0000","last_clean":"2024-04-16T05:34:02.935014+0000","last_became_active":"2024-04-16T05:32:34.758587+0000","last_became_peered":"2024-04-16T05:32:34.758587+0000","last_unstale":"2024-04-16T05:34:02.935014+0000","last_undegraded":"2024-04-16T05:34:02.935014+0000","last_fullsized":"2024-04-16T05:34:02.935014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:19:20.674248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.946578+0000","last_change":"2024-04-16T05:32:34.750846+0000","last_active":"2024-04-16T05:34:02.946578+0000","last_peered":"2024-04-16T05:34:02.946578+0000","last_clean":"2024-04-16T05:34:02.946578+0000","last_became_active":"2024-04-16T05:32:34.750529+0000","last_became_peered":"2024-04-16T05:32:34.750529+0000","last_unstale":"2024-04-16T05:34:02.946578+0000","last_undegraded":"2024-04-16T05:34:02.946578+0000","last_fullsized":"2024-04-16T05:34:02.946578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:32.288459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.943668+0000","last_change":"2024-04-16T05:32:34.752566+0000","last_active":"2024-04-16T05:34:02.943668+0000","last_peered":"2024-04-16T05:34:02.943668+0000","last_clean":"2024-04-16T05:34:02.943668+0000","last_became_active":"2024-04-16T05:32:34.752236+0000","last_became_peered":"2024-04-16T05:32:34.752236+0000","last_unstale":"2024-04-16T05:34:02.943668+0000","last_undegraded":"2024-04-16T05:34:02.943668+0000","last_fullsized":"2024-04-16T05:34:02.943668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:43:36.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T05:34:02.936323+0000","last_change":"2024-04-16T05:32:34.750972+0000","last_active":"2024-04-16T05:34:02.936323+0000","last_peered":"2024-04-16T05:34:02.936323+0000","last_clean":"2024-04-16T05:34:02.936323+0000","last_became_active":"2024-04-16T05:32:34.750671+0000","last_became_peered":"2024-04-16T05:32:34.750671+0000","last_unstale":"2024-04-16T05:34:02.936323+0000","last_undegraded":"2024-04-16T05:34:02.936323+0000","last_fullsized":"2024-04-16T05:34:02.936323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:19:45.331816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283986,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66698,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":484,"num_read_kb":312,"num_write":169,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":200094,"data_compressed":4893,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":500857,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1150647,"data_compressed":58385,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":505155,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070511,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":500979,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1097561,"data_compressed":56754,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495673,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":496235,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":558169,"data_compressed":22271,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":497623,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":560017,"data_compressed":22271,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":496429,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":501417,"data_compressed":20640,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1092617,"data_compressed":56754,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34472,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":62024,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63872,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:34:13.970 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-16T05:34:13.970 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-16T05:34:13.970 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-16T05:34:13.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph health --format=json 2024-04-16T05:34:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:14 smithi032 ceph-mon[32351]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:34:14.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:14 smithi032 ceph-mon[32351]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:14.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:14 smithi082 ceph-mon[36195]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:34:14.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:14 smithi082 ceph-mon[36195]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:14.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:14 smithi134 ceph-mon[36153]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:34:14.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:14 smithi134 ceph-mon[36153]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:16.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 -- 172.21.15.32:0/2971692593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48102b40 msgr2=0x7fac48102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 --2- 172.21.15.32:0/2971692593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48102b40 0x7fac48102fc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009990 tx=0x7fac3c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 -- 172.21.15.32:0/2971692593 shutdown_connections 2024-04-16T05:34:16.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 --2- 172.21.15.32:0/2971692593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac48103500 0x7fac4810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 --2- 172.21.15.32:0/2971692593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac48108b90 0x7fac48108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 --2- 172.21.15.32:0/2971692593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48102b40 0x7fac48102fc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 -- 172.21.15.32:0/2971692593 >> 172.21.15.32:0/2971692593 conn(0x7fac48079c50 msgr2=0x7fac4807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 -- 172.21.15.32:0/2971692593 shutdown_connections 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 -- 172.21.15.32:0/2971692593 wait complete. 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.012+0000 7fac4da6f700 1 Processor -- start 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac4da6f700 1 -- start start 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac4da6f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac48102b40 0x7fac48199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac4da6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48103500 0x7fac481997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac4da6f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac48108b90 0x7fac4819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac4da6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fac48112df0 con 0x7fac48103500 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac4da6f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac48112c70 con 0x7fac48102b40 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac4da6f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac48112f70 con 0x7fac48108b90 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48103500 0x7fac481997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48103500 0x7fac481997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37592/0 (socket says 172.21.15.32:37592) 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 -- 172.21.15.32:0/1856024836 learned_addr learned my addr 172.21.15.32:0/1856024836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac477fe700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac48102b40 0x7fac48199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac47fff700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac48108b90 0x7fac4819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 -- 172.21.15.32:0/1856024836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac48108b90 msgr2=0x7fac4819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac48108b90 0x7fac4819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 -- 172.21.15.32:0/1856024836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac48102b40 msgr2=0x7fac48199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac48102b40 0x7fac48199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 -- 172.21.15.32:0/1856024836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4819e270 con 0x7fac48103500 2024-04-16T05:34:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.013+0000 7fac46ffd700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48103500 0x7fac481997a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fac3c000c00 tx=0x7fac3c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.014+0000 7fac44ff9700 1 -- 172.21.15.32:0/1856024836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c027070 con 0x7fac48103500 2024-04-16T05:34:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.014+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4819e500 con 0x7fac48103500 2024-04-16T05:34:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.014+0000 7fac44ff9700 1 -- 172.21.15.32:0/1856024836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac3c003780 con 0x7fac48103500 2024-04-16T05:34:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.014+0000 7fac44ff9700 1 -- 172.21.15.32:0/1856024836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c022dd0 con 0x7fac48103500 2024-04-16T05:34:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.014+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac481a5de0 con 0x7fac48103500 2024-04-16T05:34:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.015+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac480697e0 con 0x7fac48103500 2024-04-16T05:34:16.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.016+0000 7fac44ff9700 1 -- 172.21.15.32:0/1856024836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fac3c02a070 con 0x7fac48103500 2024-04-16T05:34:16.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.016+0000 7fac44ff9700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fac300a3c40 0x7fac300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:16.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.016+0000 7fac477fe700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fac300a3c40 0x7fac300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:16.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.016+0000 7fac44ff9700 1 -- 172.21.15.32:0/1856024836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fac3c0d1450 con 0x7fac48103500 2024-04-16T05:34:16.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.017+0000 7fac477fe700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fac300a3c40 0x7fac300a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac3800abd0 tx=0x7fac3800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.021+0000 7fac44ff9700 1 -- 172.21.15.32:0/1856024836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac3c013070 con 0x7fac48103500 2024-04-16T05:34:16.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.278+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fac4802dfe0 con 0x7fac48103500 2024-04-16T05:34:16.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.279+0000 7fac44ff9700 1 -- 172.21.15.32:0/1856024836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fac3c09dbf0 con 0x7fac48103500 2024-04-16T05:34:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:16.277 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fac300a3c40 msgr2=0x7fac300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fac300a3c40 0x7fac300a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac3800abd0 tx=0x7fac3800a380 comp rx=0 tx=0).stop 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48103500 msgr2=0x7fac481997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48103500 0x7fac481997a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fac3c000c00 tx=0x7fac3c0047b0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 shutdown_connections 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fac300a3c40 0x7fac300a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac48108b90 0x7fac4819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac48102b40 0x7fac48199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 --2- 172.21.15.32:0/1856024836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac48103500 0x7fac481997a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 >> 172.21.15.32:0/1856024836 conn(0x7fac48079c50 msgr2=0x7fac48101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 shutdown_connections 2024-04-16T05:34:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:16.281+0000 7fac4da6f700 1 -- 172.21.15.32:0/1856024836 wait complete. 2024-04-16T05:34:16.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:16 smithi082 ceph-mon[36195]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:34:16.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:16 smithi032 ceph-mon[32351]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:34:16.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:16 smithi134 ceph-mon[36153]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:34:16.948 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-16T05:34:16.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.060+0000 7f8490e0d640 1 -- 172.21.15.32:0/2650911792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 msgr2=0x7f848c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.060+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2650911792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 0x7f848c1013e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f847c009900 tx=0x7f847c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 -- 172.21.15.32:0/2650911792 shutdown_connections 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2650911792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c102b70 0x7f848c109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2650911792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f848c1021b0 0x7f848c102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2650911792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 0x7f848c1013e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 -- 172.21.15.32:0/2650911792 >> 172.21.15.32:0/2650911792 conn(0x7f848c0fc550 msgr2=0x7f848c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 -- 172.21.15.32:0/2650911792 shutdown_connections 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 -- 172.21.15.32:0/2650911792 wait complete. 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 Processor -- start 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 -- start start 2024-04-16T05:34:17.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 0x7f848c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f848c1021b0 0x7f848c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c102b70 0x7f848c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f848c10cbe0 con 0x7f848c100fc0 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f848c10ca60 con 0x7f848c102b70 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.061+0000 7f8490e0d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f848c10cd60 con 0x7f848c1021b0 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 0x7f848c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 0x7f848c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37610/0 (socket says 172.21.15.32:37610) 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 -- 172.21.15.32:0/2563155444 learned_addr learned my addr 172.21.15.32:0/2563155444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848ad9d640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c102b70 0x7f848c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f8489d9b640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f848c1021b0 0x7f848c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 -- 172.21.15.32:0/2563155444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f848c1021b0 msgr2=0x7f848c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f848c1021b0 0x7f848c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 -- 172.21.15.32:0/2563155444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c102b70 msgr2=0x7f848c19f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c102b70 0x7f848c19f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 -- 172.21.15.32:0/2563155444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c19ffa0 con 0x7f848c100fc0 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f8489d9b640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f848c1021b0 0x7f848c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f848a59c640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 0x7f848c198380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f847c000c00 tx=0x7f847c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f847b7fe640 1 -- 172.21.15.32:0/2563155444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c041070 con 0x7f848c100fc0 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848c1a0230 con 0x7f848c100fc0 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f847b7fe640 1 -- 172.21.15.32:0/2563155444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c004450 con 0x7f848c100fc0 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f847b7fe640 1 -- 172.21.15.32:0/2563155444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c034440 con 0x7f848c100fc0 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.062+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848c1a0720 con 0x7f848c100fc0 2024-04-16T05:34:17.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.063+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848c1a0a40 con 0x7f848c100fc0 2024-04-16T05:34:17.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.065+0000 7f847b7fe640 1 -- 172.21.15.32:0/2563155444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f847c044070 con 0x7f848c100fc0 2024-04-16T05:34:17.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.065+0000 7f847b7fe640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f84740a3bb0 0x7f84740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.065+0000 7f8489d9b640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f84740a3bb0 0x7f84740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.065+0000 7f847b7fe640 1 -- 172.21.15.32:0/2563155444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f847c0f24d0 con 0x7f848c100fc0 2024-04-16T05:34:17.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.066+0000 7f8489d9b640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f84740a3bb0 0x7f84740a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f848c199890 tx=0x7f8480005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:17.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.069+0000 7f847b7fe640 1 -- 172.21.15.32:0/2563155444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f847c0bebe0 con 0x7f848c100fc0 2024-04-16T05:34:17.212 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:17 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1856024836' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T05:34:17.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.261+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f848c1a0c10 con 0x7f848c100fc0 2024-04-16T05:34:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.263+0000 7f847b7fe640 1 -- 172.21.15.32:0/2563155444 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24618 (secure 0 0 0) 0x7f847c0be4e0 con 0x7f848c100fc0 2024-04-16T05:34:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:17.262 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:33:34.757328+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:17.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f84740a3bb0 msgr2=0x7f84740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f84740a3bb0 0x7f84740a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f848c199890 tx=0x7f8480005f90 comp rx=0 tx=0).stop 2024-04-16T05:34:17.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 msgr2=0x7f848c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 0x7f848c198380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f847c000c00 tx=0x7f847c02c640 comp rx=0 tx=0).stop 2024-04-16T05:34:17.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 shutdown_connections 2024-04-16T05:34:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f84740a3bb0 0x7f84740a6070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c102b70 0x7f848c19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f848c1021b0 0x7f848c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 --2- 172.21.15.32:0/2563155444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848c100fc0 0x7f848c198380 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 >> 172.21.15.32:0/2563155444 conn(0x7f848c0fc550 msgr2=0x7f848c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 shutdown_connections 2024-04-16T05:34:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.265+0000 7f8490e0d640 1 -- 172.21.15.32:0/2563155444 wait complete. 2024-04-16T05:34:17.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:17.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 -- 172.21.15.32:0/1920031292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68501006d0 msgr2=0x7f6850100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 --2- 172.21.15.32:0/1920031292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68501006d0 0x7f6850100af0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6840009900 tx=0x7f684002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 -- 172.21.15.32:0/1920031292 shutdown_connections 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 --2- 172.21.15.32:0/1920031292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6850102280 0x7f6850108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 --2- 172.21.15.32:0/1920031292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68501018c0 0x7f6850101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 --2- 172.21.15.32:0/1920031292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68501006d0 0x7f6850100af0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 -- 172.21.15.32:0/1920031292 >> 172.21.15.32:0/1920031292 conn(0x7f68500fbc40 msgr2=0x7f68500fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 -- 172.21.15.32:0/1920031292 shutdown_connections 2024-04-16T05:34:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 -- 172.21.15.32:0/1920031292 wait complete. 2024-04-16T05:34:17.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 Processor -- start 2024-04-16T05:34:17.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.387+0000 7f6856790640 1 -- start start 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6856790640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68501006d0 0x7f685019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6856790640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68501018c0 0x7f685019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6856790640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6850102280 0x7f68501a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6856790640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f685010bae0 con 0x7f6850102280 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6856790640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f685010b960 con 0x7f68501006d0 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6856790640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f685010bc60 con 0x7f68501018c0 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6850102280 0x7f68501a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6850102280 0x7f68501a1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37628/0 (socket says 172.21.15.32:37628) 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 -- 172.21.15.32:0/3756786336 learned_addr learned my addr 172.21.15.32:0/3756786336 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f684f7fe640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68501018c0 0x7f685019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f684ffff640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68501006d0 0x7f685019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 -- 172.21.15.32:0/3756786336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68501018c0 msgr2=0x7f685019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68501018c0 0x7f685019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 -- 172.21.15.32:0/3756786336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68501006d0 msgr2=0x7f685019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68501006d0 0x7f685019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 -- 172.21.15.32:0/3756786336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68501a21a0 con 0x7f6850102280 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f684ffff640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68501006d0 0x7f685019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f6854d2d640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6850102280 0x7f68501a1b30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f684800dd70 tx=0x7f684800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.388+0000 7f684d7fa640 1 -- 172.21.15.32:0/3756786336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848004d20 con 0x7f6850102280 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.389+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68501a2490 con 0x7f6850102280 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.389+0000 7f684d7fa640 1 -- 172.21.15.32:0/3756786336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684800be40 con 0x7f6850102280 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.389+0000 7f684d7fa640 1 -- 172.21.15.32:0/3756786336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848005720 con 0x7f6850102280 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.389+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68501a2a30 con 0x7f6850102280 2024-04-16T05:34:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.390+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6850107a10 con 0x7f6850102280 2024-04-16T05:34:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.391+0000 7f684d7fa640 1 -- 172.21.15.32:0/3756786336 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f68480058c0 con 0x7f6850102280 2024-04-16T05:34:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.391+0000 7f684d7fa640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68380a3bb0 0x7f68380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.391+0000 7f684ffff640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68380a3bb0 0x7f68380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.391+0000 7f684d7fa640 1 -- 172.21.15.32:0/3756786336 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f68480d0d30 con 0x7f6850102280 2024-04-16T05:34:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.392+0000 7f684ffff640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68380a3bb0 0x7f68380a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f684002c7b0 tx=0x7f6840005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:17.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.396+0000 7f684d7fa640 1 -- 172.21.15.32:0/3756786336 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f684809d3b0 con 0x7f6850102280 2024-04-16T05:34:17.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.581+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6850062c00 con 0x7f6850102280 2024-04-16T05:34:17.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.582+0000 7f684d7fa640 1 -- 172.21.15.32:0/3756786336 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f684809ccb0 con 0x7f6850102280 2024-04-16T05:34:17.580 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:34:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.583+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68380a3bb0 msgr2=0x7f68380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.583+0000 7f6856790640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68380a3bb0 0x7f68380a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f684002c7b0 tx=0x7f6840005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6850102280 msgr2=0x7f68501a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6850102280 0x7f68501a1b30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f684800dd70 tx=0x7f684800b730 comp rx=0 tx=0).stop 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 shutdown_connections 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f68380a3bb0 0x7f68380a6070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6850102280 0x7f68501a1b30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68501018c0 0x7f685019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 --2- 172.21.15.32:0/3756786336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68501006d0 0x7f685019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 >> 172.21.15.32:0/3756786336 conn(0x7f68500fbc40 msgr2=0x7f6850104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 shutdown_connections 2024-04-16T05:34:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.584+0000 7f6856790640 1 -- 172.21.15.32:0/3756786336 wait complete. 2024-04-16T05:34:17.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:34:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:17 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1856024836' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T05:34:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:17 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1856024836' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 -- 172.21.15.32:0/565743464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc102b40 msgr2=0x7f57dc104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 --2- 172.21.15.32:0/565743464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc102b40 0x7f57dc104f30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009900 tx=0x7f57d802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 -- 172.21.15.32:0/565743464 shutdown_connections 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 --2- 172.21.15.32:0/565743464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc107da0 0x7f57dc10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 --2- 172.21.15.32:0/565743464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57dc105470 0x7f57dc107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 --2- 172.21.15.32:0/565743464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc102b40 0x7f57dc104f30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 -- 172.21.15.32:0/565743464 >> 172.21.15.32:0/565743464 conn(0x7f57dc0fc4f0 msgr2=0x7f57dc0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 -- 172.21.15.32:0/565743464 shutdown_connections 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.703+0000 7f57e33af640 1 -- 172.21.15.32:0/565743464 wait complete. 2024-04-16T05:34:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e33af640 1 Processor -- start 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e33af640 1 -- start start 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e33af640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc102b40 0x7f57dc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e33af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57dc105470 0x7f57dc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e33af640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc107da0 0x7f57dc19f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e33af640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10dcf0 con 0x7f57dc107da0 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e33af640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10db70 con 0x7f57dc102b40 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e33af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10de70 con 0x7f57dc105470 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e194c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc107da0 0x7f57dc19f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e194c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc107da0 0x7f57dc19f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37650/0 (socket says 172.21.15.32:37650) 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e194c640 1 -- 172.21.15.32:0/228099115 learned_addr learned my addr 172.21.15.32:0/228099115 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e094a640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57dc105470 0x7f57dc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e114b640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc102b40 0x7f57dc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e194c640 1 -- 172.21.15.32:0/228099115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57dc105470 msgr2=0x7f57dc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e194c640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57dc105470 0x7f57dc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e194c640 1 -- 172.21.15.32:0/228099115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc102b40 msgr2=0x7f57dc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e194c640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc102b40 0x7f57dc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e194c640 1 -- 172.21.15.32:0/228099115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc19fff0 con 0x7f57dc107da0 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.704+0000 7f57e114b640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc102b40 0x7f57dc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.705+0000 7f57e194c640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc107da0 0x7f57dc19f980 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009fd0 tx=0x7f57d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.705+0000 7f57d27fc640 1 -- 172.21.15.32:0/228099115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4017070 con 0x7f57dc107da0 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.705+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc1a02e0 con 0x7f57dc107da0 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.705+0000 7f57d27fc640 1 -- 172.21.15.32:0/228099115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d4003e50 con 0x7f57dc107da0 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.705+0000 7f57d27fc640 1 -- 172.21.15.32:0/228099115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d40106c0 con 0x7f57dc107da0 2024-04-16T05:34:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.705+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc1a07e0 con 0x7f57dc107da0 2024-04-16T05:34:17.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.706+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc0fdfe0 con 0x7f57dc107da0 2024-04-16T05:34:17.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.707+0000 7f57d27fc640 1 -- 172.21.15.32:0/228099115 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f57d4003970 con 0x7f57dc107da0 2024-04-16T05:34:17.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.708+0000 7f57d27fc640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f57c80a3bb0 0x7f57c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:17.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.708+0000 7f57e114b640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f57c80a3bb0 0x7f57c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:17.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.708+0000 7f57d27fc640 1 -- 172.21.15.32:0/228099115 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f57d40d04f0 con 0x7f57dc107da0 2024-04-16T05:34:17.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.708+0000 7f57e114b640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f57c80a3bb0 0x7f57c80a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57d8006010 tx=0x7f57d802c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:17.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.712+0000 7f57d27fc640 1 -- 172.21.15.32:0/228099115 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57d409cb70 con 0x7f57dc107da0 2024-04-16T05:34:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.900+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f57dc062c00 con 0x7f57dc107da0 2024-04-16T05:34:17.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.900+0000 7f57d27fc640 1 -- 172.21.15.32:0/228099115 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f57d409c470 con 0x7f57dc107da0 2024-04-16T05:34:17.899 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f57c80a3bb0 msgr2=0x7f57c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f57c80a3bb0 0x7f57c80a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57d8006010 tx=0x7f57d802c8a0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc107da0 msgr2=0x7f57dc19f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc107da0 0x7f57dc19f980 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009fd0 tx=0x7f57d400b260 comp rx=0 tx=0).stop 2024-04-16T05:34:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 shutdown_connections 2024-04-16T05:34:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f57c80a3bb0 0x7f57c80a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc107da0 0x7f57dc19f980 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57dc105470 0x7f57dc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 --2- 172.21.15.32:0/228099115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc102b40 0x7f57dc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 >> 172.21.15.32:0/228099115 conn(0x7f57dc0fc4f0 msgr2=0x7f57dc103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 shutdown_connections 2024-04-16T05:34:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:17.902+0000 7f57e33af640 1 -- 172.21.15.32:0/228099115 wait complete. 2024-04-16T05:34:17.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:34:18.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.022+0000 7f0257bd4640 1 -- 172.21.15.32:0/463003562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 msgr2=0x7f0250104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.022+0000 7f0257bd4640 1 --2- 172.21.15.32:0/463003562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 0x7f0250104ed0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0240009ca0 tx=0x7f024002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.022+0000 7f0257bd4640 1 -- 172.21.15.32:0/463003562 shutdown_connections 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.022+0000 7f0257bd4640 1 --2- 172.21.15.32:0/463003562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0250107d40 0x7f025010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.022+0000 7f0257bd4640 1 --2- 172.21.15.32:0/463003562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0250105410 0x7f0250107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.022+0000 7f0257bd4640 1 --2- 172.21.15.32:0/463003562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 0x7f0250104ed0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.022+0000 7f0257bd4640 1 -- 172.21.15.32:0/463003562 >> 172.21.15.32:0/463003562 conn(0x7f02500fc4d0 msgr2=0x7f02500fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.022+0000 7f0257bd4640 1 -- 172.21.15.32:0/463003562 shutdown_connections 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 -- 172.21.15.32:0/463003562 wait complete. 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 Processor -- start 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 -- start start 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 0x7f0250198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0250105410 0x7f02501988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0250107d40 0x7f025019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f025010dcf0 con 0x7f0250102ae0 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f025010db70 con 0x7f0250105410 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0257bd4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f025010de70 con 0x7f0250107d40 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0255970640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 0x7f0250198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0255970640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 0x7f0250198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37682/0 (socket says 172.21.15.32:37682) 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0255970640 1 -- 172.21.15.32:0/3378420290 learned_addr learned my addr 172.21.15.32:0/3378420290 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.023+0000 7f0256171640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0250107d40 0x7f025019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f025516f640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0250105410 0x7f02501988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f0255970640 1 -- 172.21.15.32:0/3378420290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0250107d40 msgr2=0x7f025019f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f0255970640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0250107d40 0x7f025019f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f0255970640 1 -- 172.21.15.32:0/3378420290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0250105410 msgr2=0x7f02501988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f0255970640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0250105410 0x7f02501988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f0255970640 1 -- 172.21.15.32:0/3378420290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025019ff80 con 0x7f0250102ae0 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f025516f640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0250105410 0x7f02501988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f0255970640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 0x7f0250198360 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f02400324c0 tx=0x7f0240032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f024effd640 1 -- 172.21.15.32:0/3378420290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240039040 con 0x7f0250102ae0 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02501a0210 con 0x7f0250102ae0 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f024effd640 1 -- 172.21.15.32:0/3378420290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0240035c20 con 0x7f0250102ae0 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f024effd640 1 -- 172.21.15.32:0/3378420290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240004310 con 0x7f0250102ae0 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.024+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501a0700 con 0x7f0250102ae0 2024-04-16T05:34:18.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.025+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02500fdfc0 con 0x7f0250102ae0 2024-04-16T05:34:18.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.026+0000 7f024effd640 1 -- 172.21.15.32:0/3378420290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0240046030 con 0x7f0250102ae0 2024-04-16T05:34:18.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.027+0000 7f024effd640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.027+0000 7f025516f640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.027+0000 7f024effd640 1 -- 172.21.15.32:0/3378420290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f02400f2230 con 0x7f0250102ae0 2024-04-16T05:34:18.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.027+0000 7f025516f640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f02380a3bb0 0x7f02380a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02440058c0 tx=0x7f0244005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.031+0000 7f024effd640 1 -- 172.21.15.32:0/3378420290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f024004d060 con 0x7f0250102ae0 2024-04-16T05:34:18.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.219+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f025002d840 con 0x7f0250102ae0 2024-04-16T05:34:18.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.220+0000 7f024effd640 1 -- 172.21.15.32:0/3378420290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f02400be940 con 0x7f0250102ae0 2024-04-16T05:34:18.218 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:34:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f02380a3bb0 msgr2=0x7f02380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f02380a3bb0 0x7f02380a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02440058c0 tx=0x7f0244005850 comp rx=0 tx=0).stop 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 msgr2=0x7f0250198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 0x7f0250198360 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f02400324c0 tx=0x7f0240032f40 comp rx=0 tx=0).stop 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 shutdown_connections 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0250107d40 0x7f025019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0250105410 0x7f02501988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 --2- 172.21.15.32:0/3378420290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0250102ae0 0x7f0250198360 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 >> 172.21.15.32:0/3378420290 conn(0x7f02500fc4d0 msgr2=0x7f0250103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 shutdown_connections 2024-04-16T05:34:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.222+0000 7f0257bd4640 1 -- 172.21.15.32:0/3378420290 wait complete. 2024-04-16T05:34:18.231 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:18 smithi032 ceph-mon[32351]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:34:18.231 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:18 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2563155444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:18.231 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:18 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3756786336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:18.231 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:18 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/228099115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:18.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:34:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 -- 172.21.15.32:0/3350509429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102090 msgr2=0x7f5290102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 --2- 172.21.15.32:0/3350509429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102090 0x7f5290102510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5280009900 tx=0x7f528002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 -- 172.21.15.32:0/3350509429 shutdown_connections 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 --2- 172.21.15.32:0/3350509429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5290102a50 0x7f5290109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 --2- 172.21.15.32:0/3350509429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102090 0x7f5290102510 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 --2- 172.21.15.32:0/3350509429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5290100f40 0x7f5290101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 -- 172.21.15.32:0/3350509429 >> 172.21.15.32:0/3350509429 conn(0x7f52900fc4b0 msgr2=0x7f52900fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 -- 172.21.15.32:0/3350509429 shutdown_connections 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.341+0000 7f5296a74640 1 -- 172.21.15.32:0/3350509429 wait complete. 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5296a74640 1 Processor -- start 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5296a74640 1 -- start start 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5296a74640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5290100f40 0x7f5290198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5296a74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5290102090 0x7f5290198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5296a74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102a50 0x7f529019f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5296a74640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f529010ccc0 con 0x7f5290102a50 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5296a74640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f529010cb40 con 0x7f5290100f40 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5296a74640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f529010ce40 con 0x7f5290102090 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102a50 0x7f529019f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102a50 0x7f529019f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37694/0 (socket says 172.21.15.32:37694) 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 -- 172.21.15.32:0/703954911 learned_addr learned my addr 172.21.15.32:0/703954911 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5287fff640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5290102090 0x7f5290198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5294810640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5290100f40 0x7f5290198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 -- 172.21.15.32:0/703954911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5290102090 msgr2=0x7f5290198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5290102090 0x7f5290198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 -- 172.21.15.32:0/703954911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5290100f40 msgr2=0x7f5290198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5290100f40 0x7f5290198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 -- 172.21.15.32:0/703954911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52901a0050 con 0x7f5290102a50 2024-04-16T05:34:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.342+0000 7f5295011640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102a50 0x7f529019f9e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f528800b970 tx=0x7f528800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.343+0000 7f5285ffb640 1 -- 172.21.15.32:0/703954911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288019070 con 0x7f5290102a50 2024-04-16T05:34:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.343+0000 7f5285ffb640 1 -- 172.21.15.32:0/703954911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528800cb40 con 0x7f5290102a50 2024-04-16T05:34:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.343+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52901a0340 con 0x7f5290102a50 2024-04-16T05:34:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.343+0000 7f5285ffb640 1 -- 172.21.15.32:0/703954911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288007650 con 0x7f5290102a50 2024-04-16T05:34:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.343+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901a0890 con 0x7f5290102a50 2024-04-16T05:34:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.344+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52901a0b50 con 0x7f5290102a50 2024-04-16T05:34:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.345+0000 7f5285ffb640 1 -- 172.21.15.32:0/703954911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f528801c070 con 0x7f5290102a50 2024-04-16T05:34:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.346+0000 7f5285ffb640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52780a3c30 0x7f52780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.346+0000 7f5294810640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52780a3c30 0x7f52780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.346+0000 7f5285ffb640 1 -- 172.21.15.32:0/703954911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f52880cfd50 con 0x7f5290102a50 2024-04-16T05:34:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.346+0000 7f5294810640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52780a3c30 0x7f52780a60f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f528c00a810 tx=0x7f528c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:18.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.350+0000 7f5285ffb640 1 -- 172.21.15.32:0/703954911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52880d3030 con 0x7f5290102a50 2024-04-16T05:34:18.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.536+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f52901a0d20 con 0x7f5290102a50 2024-04-16T05:34:18.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.536+0000 7f5285ffb640 1 -- 172.21.15.32:0/703954911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f528809c3d0 con 0x7f5290102a50 2024-04-16T05:34:18.535 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52780a3c30 msgr2=0x7f52780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52780a3c30 0x7f52780a60f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f528c00a810 tx=0x7f528c008040 comp rx=0 tx=0).stop 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102a50 msgr2=0x7f529019f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102a50 0x7f529019f9e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f528800b970 tx=0x7f528800be30 comp rx=0 tx=0).stop 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 shutdown_connections 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f52780a3c30 0x7f52780a60f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5290102a50 0x7f529019f9e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5290102090 0x7f5290198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 --2- 172.21.15.32:0/703954911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5290100f40 0x7f5290198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 >> 172.21.15.32:0/703954911 conn(0x7f52900fc4b0 msgr2=0x7f52900fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:18.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 shutdown_connections 2024-04-16T05:34:18.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.538+0000 7f5296a74640 1 -- 172.21.15.32:0/703954911 wait complete. 2024-04-16T05:34:18.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:18 smithi134 ceph-mon[36153]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:34:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:18 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2563155444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:18 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3756786336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:18.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:18 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/228099115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:18.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:18 smithi082 ceph-mon[36195]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:34:18.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:18 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2563155444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:18.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:18 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3756786336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:18.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:18 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/228099115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/1261907390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b181020e0 msgr2=0x7f0b18102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/1261907390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b181020e0 0x7f0b18102560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009900 tx=0x7f0b0c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/1261907390 shutdown_connections 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/1261907390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b18102aa0 0x7f0b181093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/1261907390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b181020e0 0x7f0b18102560 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/1261907390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b18100f10 0x7f0b18101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/1261907390 >> 172.21.15.32:0/1261907390 conn(0x7f0b180fc4f0 msgr2=0x7f0b180fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/1261907390 shutdown_connections 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/1261907390 wait complete. 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 Processor -- start 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.655+0000 7f0b1e7d6640 1 -- start start 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1e7d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b18100f10 0x7f0b1806b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1e7d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b181020e0 0x7f0b1819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1e7d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18102aa0 0x7f0b1819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1e7d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1810cbe0 con 0x7f0b18102aa0 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1e7d6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1810ca60 con 0x7f0b18100f10 2024-04-16T05:34:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1e7d6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1810cd60 con 0x7f0b181020e0 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18102aa0 0x7f0b1819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18102aa0 0x7f0b1819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53476/0 (socket says 172.21.15.32:53476) 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 -- 172.21.15.32:0/2674987523 learned_addr learned my addr 172.21.15.32:0/2674987523 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b17fff640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b18100f10 0x7f0b1806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b177fe640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b181020e0 0x7f0b1819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 -- 172.21.15.32:0/2674987523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b181020e0 msgr2=0x7f0b1819d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b181020e0 0x7f0b1819d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 -- 172.21.15.32:0/2674987523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b18100f10 msgr2=0x7f0b1806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b18100f10 0x7f0b1806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 -- 172.21.15.32:0/2674987523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1819ff50 con 0x7f0b18102aa0 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1cd73640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18102aa0 0x7f0b1819f690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000cce0 tx=0x7f0b1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b157fa640 1 -- 172.21.15.32:0/2674987523 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000f450 con 0x7f0b18102aa0 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b157fa640 1 -- 172.21.15.32:0/2674987523 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1000fad0 con 0x7f0b18102aa0 2024-04-16T05:34:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b181a0240 con 0x7f0b18102aa0 2024-04-16T05:34:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.656+0000 7f0b157fa640 1 -- 172.21.15.32:0/2674987523 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10018c10 con 0x7f0b18102aa0 2024-04-16T05:34:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.657+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b181a0710 con 0x7f0b18102aa0 2024-04-16T05:34:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.657+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b18069270 con 0x7f0b18102aa0 2024-04-16T05:34:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.659+0000 7f0b157fa640 1 -- 172.21.15.32:0/2674987523 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0b1000f5f0 con 0x7f0b18102aa0 2024-04-16T05:34:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.659+0000 7f0b157fa640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b000a3bb0 0x7f0b000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.659+0000 7f0b157fa640 1 -- 172.21.15.32:0/2674987523 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0b100d0d50 con 0x7f0b18102aa0 2024-04-16T05:34:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.659+0000 7f0b17fff640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b000a3bb0 0x7f0b000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.660+0000 7f0b17fff640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b000a3bb0 0x7f0b000a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0b08005fd0 tx=0x7f0b0800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:18.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.664+0000 7f0b157fa640 1 -- 172.21.15.32:0/2674987523 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b100d4030 con 0x7f0b18102aa0 2024-04-16T05:34:18.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.893+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b18062c00 con 0x7f0b18102aa0 2024-04-16T05:34:18.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.894+0000 7f0b157fa640 1 -- 172.21.15.32:0/2674987523 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11876 (secure 0 0 0) 0x7f0b10014030 con 0x7f0b18102aa0 2024-04-16T05:34:18.892 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:18.892 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:32:55.688634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24461,24494]},"id":1}]} 2024-04-16T05:34:18.893 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b000a3bb0 msgr2=0x7f0b000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b000a3bb0 0x7f0b000a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0b08005fd0 tx=0x7f0b0800b390 comp rx=0 tx=0).stop 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18102aa0 msgr2=0x7f0b1819f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18102aa0 0x7f0b1819f690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000cce0 tx=0x7f0b1000a620 comp rx=0 tx=0).stop 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 shutdown_connections 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b000a3bb0 0x7f0b000a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18102aa0 0x7f0b1819f690 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b181020e0 0x7f0b1819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 --2- 172.21.15.32:0/2674987523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b18100f10 0x7f0b1806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 >> 172.21.15.32:0/2674987523 conn(0x7f0b180fc4f0 msgr2=0x7f0b181054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:18.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 shutdown_connections 2024-04-16T05:34:18.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:18.896+0000 7f0b1e7d6640 1 -- 172.21.15.32:0/2674987523 wait complete. 2024-04-16T05:34:18.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 -- 172.21.15.32:0/3368297194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 msgr2=0x7f62fc10a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 --2- 172.21.15.32:0/3368297194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 0x7f62fc10a180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009ca0 tx=0x7f62f402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 -- 172.21.15.32:0/3368297194 shutdown_connections 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 --2- 172.21.15.32:0/3368297194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 0x7f62fc10a180 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 --2- 172.21.15.32:0/3368297194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc105400 0x7f62fc1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 --2- 172.21.15.32:0/3368297194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc102ad0 0x7f62fc104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 -- 172.21.15.32:0/3368297194 >> 172.21.15.32:0/3368297194 conn(0x7f62fc0fc4e0 msgr2=0x7f62fc0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 -- 172.21.15.32:0/3368297194 shutdown_connections 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.014+0000 7f6301688640 1 -- 172.21.15.32:0/3368297194 wait complete. 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f6301688640 1 Processor -- start 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f6301688640 1 -- start start 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f6301688640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc102ad0 0x7f62fc102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f6301688640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc105400 0x7f62fc100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f6301688640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 0x7f62fc100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f6301688640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc10dce0 con 0x7f62fc107d30 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f6301688640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc10db60 con 0x7f62fc105400 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f6301688640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc10de60 con 0x7f62fc102ad0 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62fb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 0x7f62fc100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62fb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 0x7f62fc100fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53500/0 (socket says 172.21.15.32:53500) 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62fb7fe640 1 -- 172.21.15.32:0/1070681775 learned_addr learned my addr 172.21.15.32:0/1070681775 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62faffd640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc102ad0 0x7f62fc102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62f27fc640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc105400 0x7f62fc100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62fb7fe640 1 -- 172.21.15.32:0/1070681775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc102ad0 msgr2=0x7f62fc102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62fb7fe640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc102ad0 0x7f62fc102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62fb7fe640 1 -- 172.21.15.32:0/1070681775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc105400 msgr2=0x7f62fc100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62fb7fe640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc105400 0x7f62fc100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62fb7fe640 1 -- 172.21.15.32:0/1070681775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc101850 con 0x7f62fc107d30 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.015+0000 7f62f27fc640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc105400 0x7f62fc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.016+0000 7f62fb7fe640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 0x7f62fc100fc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f62f402c700 tx=0x7f62f4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.016+0000 7f62f8ff9640 1 -- 172.21.15.32:0/1070681775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4039040 con 0x7f62fc107d30 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.016+0000 7f62f8ff9640 1 -- 172.21.15.32:0/1070681775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f4044070 con 0x7f62fc107d30 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.016+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62fc1a23f0 con 0x7f62fc107d30 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.016+0000 7f62f8ff9640 1 -- 172.21.15.32:0/1070681775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f40042a0 con 0x7f62fc107d30 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.016+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62fc1a2990 con 0x7f62fc107d30 2024-04-16T05:34:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.017+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62fc0fdfd0 con 0x7f62fc107d30 2024-04-16T05:34:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.018+0000 7f62f8ff9640 1 -- 172.21.15.32:0/1070681775 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f62f4003970 con 0x7f62fc107d30 2024-04-16T05:34:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.019+0000 7f62f8ff9640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f62e80a3c30 0x7f62e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.019+0000 7f62faffd640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f62e80a3c30 0x7f62e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.019+0000 7f62f8ff9640 1 -- 172.21.15.32:0/1070681775 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f62f40f2cb0 con 0x7f62fc107d30 2024-04-16T05:34:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.019+0000 7f62faffd640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f62e80a3c30 0x7f62e80a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005fd0 tx=0x7f62ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.023+0000 7f62f8ff9640 1 -- 172.21.15.32:0/1070681775 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62f40bf3c0 con 0x7f62fc107d30 2024-04-16T05:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.212+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62fc062c00 con 0x7f62fc107d30 2024-04-16T05:34:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.213+0000 7f62f8ff9640 1 -- 172.21.15.32:0/1070681775 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24618 (secure 0 0 0) 0x7f62f40becc0 con 0x7f62fc107d30 2024-04-16T05:34:19.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:19.213 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:33:34.757328+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f62e80a3c30 msgr2=0x7f62e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f62e80a3c30 0x7f62e80a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005fd0 tx=0x7f62ec005850 comp rx=0 tx=0).stop 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 msgr2=0x7f62fc100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 0x7f62fc100fc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f62f402c700 tx=0x7f62f4032f20 comp rx=0 tx=0).stop 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 shutdown_connections 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f62e80a3c30 0x7f62e80a60f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc107d30 0x7f62fc100fc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc105400 0x7f62fc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 --2- 172.21.15.32:0/1070681775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc102ad0 0x7f62fc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 >> 172.21.15.32:0/1070681775 conn(0x7f62fc0fc4e0 msgr2=0x7f62fc103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 shutdown_connections 2024-04-16T05:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.216+0000 7f6301688640 1 -- 172.21.15.32:0/1070681775 wait complete. 2024-04-16T05:34:19.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:19.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 -- 172.21.15.32:0/1757033823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64100f40 msgr2=0x7f4e64101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 --2- 172.21.15.32:0/1757033823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64100f40 0x7f4e64101360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009ca0 tx=0x7f4e5402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 -- 172.21.15.32:0/1757033823 shutdown_connections 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 --2- 172.21.15.32:0/1757033823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e64102a50 0x7f4e64109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 --2- 172.21.15.32:0/1757033823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e64102090 0x7f4e64102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 --2- 172.21.15.32:0/1757033823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64100f40 0x7f4e64101360 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 -- 172.21.15.32:0/1757033823 >> 172.21.15.32:0/1757033823 conn(0x7f4e640fc4b0 msgr2=0x7f4e640fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 -- 172.21.15.32:0/1757033823 shutdown_connections 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 -- 172.21.15.32:0/1757033823 wait complete. 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 Processor -- start 2024-04-16T05:34:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 -- start start 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e64100f40 0x7f4e64198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64102090 0x7f4e641989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e64102a50 0x7f4e6419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.336+0000 7f4e6b174640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410cc60 con 0x7f4e64102090 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e6b174640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410cae0 con 0x7f4e64102a50 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e6b174640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410cde0 con 0x7f4e64100f40 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64102090 0x7f4e641989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64102090 0x7f4e641989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53530/0 (socket says 172.21.15.32:53530) 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 -- 172.21.15.32:0/3981258241 learned_addr learned my addr 172.21.15.32:0/3981258241 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e68f10640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e64100f40 0x7f4e64198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e69711640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e64102a50 0x7f4e6419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 -- 172.21.15.32:0/3981258241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e64100f40 msgr2=0x7f4e64198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e64100f40 0x7f4e64198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 -- 172.21.15.32:0/3981258241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e64102a50 msgr2=0x7f4e6419fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e64102a50 0x7f4e6419fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 -- 172.21.15.32:0/3981258241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e641a0090 con 0x7f4e64102090 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e69711640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e64102a50 0x7f4e6419fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e68f10640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e64100f40 0x7f4e64198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e63fff640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64102090 0x7f4e641989b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00b450 tx=0x7f4e4c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e61ffb640 1 -- 172.21.15.32:0/3981258241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c015020 con 0x7f4e64102090 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e641a0380 con 0x7f4e64102090 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e61ffb640 1 -- 172.21.15.32:0/3981258241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c011d70 con 0x7f4e64102090 2024-04-16T05:34:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.337+0000 7f4e61ffb640 1 -- 172.21.15.32:0/3981258241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00f340 con 0x7f4e64102090 2024-04-16T05:34:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.338+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e641a08d0 con 0x7f4e64102090 2024-04-16T05:34:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.339+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e641a0b90 con 0x7f4e64102090 2024-04-16T05:34:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.340+0000 7f4e61ffb640 1 -- 172.21.15.32:0/3981258241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e4c018020 con 0x7f4e64102090 2024-04-16T05:34:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.340+0000 7f4e61ffb640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e500a3bb0 0x7f4e500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.340+0000 7f4e68f10640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e500a3bb0 0x7f4e500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.340+0000 7f4e61ffb640 1 -- 172.21.15.32:0/3981258241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4e4c0d02f0 con 0x7f4e64102090 2024-04-16T05:34:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.341+0000 7f4e68f10640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e500a3bb0 0x7f4e500a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4e54006010 tx=0x7f4e540325b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.345+0000 7f4e61ffb640 1 -- 172.21.15.32:0/3981258241 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e4c01f020 con 0x7f4e64102090 2024-04-16T05:34:19.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:19 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3378420290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:19.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:19 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/703954911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:19.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:19 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2674987523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.533+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e641a1300 con 0x7f4e64102090 2024-04-16T05:34:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.534+0000 7f4e61ffb640 1 -- 172.21.15.32:0/3981258241 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f4e4c00d040 con 0x7f4e64102090 2024-04-16T05:34:19.532 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:34:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e500a3bb0 msgr2=0x7f4e500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e500a3bb0 0x7f4e500a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4e54006010 tx=0x7f4e540325b0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64102090 msgr2=0x7f4e641989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64102090 0x7f4e641989b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00b450 tx=0x7f4e4c00b910 comp rx=0 tx=0).stop 2024-04-16T05:34:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 shutdown_connections 2024-04-16T05:34:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e500a3bb0 0x7f4e500a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e64102a50 0x7f4e6419fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e64102090 0x7f4e641989b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 --2- 172.21.15.32:0/3981258241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e64100f40 0x7f4e64198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 >> 172.21.15.32:0/3981258241 conn(0x7f4e640fc4b0 msgr2=0x7f4e64069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 shutdown_connections 2024-04-16T05:34:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.536+0000 7f4e6b174640 1 -- 172.21.15.32:0/3981258241 wait complete. 2024-04-16T05:34:19.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:34:19.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:19 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3378420290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:19.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:19 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/703954911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:19.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:19 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2674987523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:19.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:19 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3378420290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:19.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:19 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/703954911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:19.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:19 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2674987523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 -- 172.21.15.32:0/290589753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 msgr2=0x7f125c10a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 --2- 172.21.15.32:0/290589753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 0x7f125c10a1b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1254009900 tx=0x7f125402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 -- 172.21.15.32:0/290589753 shutdown_connections 2024-04-16T05:34:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 --2- 172.21.15.32:0/290589753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 0x7f125c10a1b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 --2- 172.21.15.32:0/290589753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f125c105490 0x7f125c107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 --2- 172.21.15.32:0/290589753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f125c102b60 0x7f125c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 -- 172.21.15.32:0/290589753 >> 172.21.15.32:0/290589753 conn(0x7f125c0fc4d0 msgr2=0x7f125c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 -- 172.21.15.32:0/290589753 shutdown_connections 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.656+0000 7f1264414640 1 -- 172.21.15.32:0/290589753 wait complete. 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f1264414640 1 Processor -- start 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f1264414640 1 -- start start 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f1264414640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f125c102b60 0x7f125c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f1264414640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f125c105490 0x7f125c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f1264414640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 0x7f125c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f1264414640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10dcf0 con 0x7f125c107dc0 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f1264414640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10db70 con 0x7f125c102b60 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f1264414640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10de70 con 0x7f125c105490 2024-04-16T05:34:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f12629b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 0x7f125c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f12629b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 0x7f125c19f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53554/0 (socket says 172.21.15.32:53554) 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f12629b1640 1 -- 172.21.15.32:0/925414018 learned_addr learned my addr 172.21.15.32:0/925414018 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f12619af640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f125c105490 0x7f125c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.657+0000 7f12621b0640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f125c102b60 0x7f125c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f12629b1640 1 -- 172.21.15.32:0/925414018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f125c105490 msgr2=0x7f125c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f12629b1640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f125c105490 0x7f125c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f12629b1640 1 -- 172.21.15.32:0/925414018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f125c102b60 msgr2=0x7f125c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f12629b1640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f125c102b60 0x7f125c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f12629b1640 1 -- 172.21.15.32:0/925414018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c19ff60 con 0x7f125c107dc0 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f12629b1640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 0x7f125c19f8f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f125402ce70 tx=0x7f1254005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f125b7fe640 1 -- 172.21.15.32:0/925414018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254041070 con 0x7f125c107dc0 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f125b7fe640 1 -- 172.21.15.32:0/925414018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12540054e0 con 0x7f125c107dc0 2024-04-16T05:34:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1a01f0 con 0x7f125c107dc0 2024-04-16T05:34:19.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f125b7fe640 1 -- 172.21.15.32:0/925414018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12540344a0 con 0x7f125c107dc0 2024-04-16T05:34:19.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.658+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a06e0 con 0x7f125c107dc0 2024-04-16T05:34:19.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.659+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c0fdfc0 con 0x7f125c107dc0 2024-04-16T05:34:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.660+0000 7f125b7fe640 1 -- 172.21.15.32:0/925414018 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1254044070 con 0x7f125c107dc0 2024-04-16T05:34:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.661+0000 7f125b7fe640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f12440a3bb0 0x7f12440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.661+0000 7f12621b0640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f12440a3bb0 0x7f12440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.661+0000 7f125b7fe640 1 -- 172.21.15.32:0/925414018 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f12540f2820 con 0x7f125c107dc0 2024-04-16T05:34:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.661+0000 7f12621b0640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f12440a3bb0 0x7f12440a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f124c005fd0 tx=0x7f124c009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.664+0000 7f125b7fe640 1 -- 172.21.15.32:0/925414018 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12540beea0 con 0x7f125c107dc0 2024-04-16T05:34:19.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.850+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f125c02d620 con 0x7f125c107dc0 2024-04-16T05:34:19.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.851+0000 7f125b7fe640 1 -- 172.21.15.32:0/925414018 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f12540be7a0 con 0x7f125c107dc0 2024-04-16T05:34:19.850 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f12440a3bb0 msgr2=0x7f12440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f12440a3bb0 0x7f12440a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f124c005fd0 tx=0x7f124c009870 comp rx=0 tx=0).stop 2024-04-16T05:34:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 msgr2=0x7f125c19f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 0x7f125c19f8f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f125402ce70 tx=0x7f1254005e40 comp rx=0 tx=0).stop 2024-04-16T05:34:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 shutdown_connections 2024-04-16T05:34:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f12440a3bb0 0x7f12440a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c107dc0 0x7f125c19f8f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f125c105490 0x7f125c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 --2- 172.21.15.32:0/925414018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f125c102b60 0x7f125c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 >> 172.21.15.32:0/925414018 conn(0x7f125c0fc4d0 msgr2=0x7f125c103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 shutdown_connections 2024-04-16T05:34:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.853+0000 7f1264414640 1 -- 172.21.15.32:0/925414018 wait complete. 2024-04-16T05:34:19.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:34:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/279237484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac105450 msgr2=0x7fd7ac107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/279237484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac105450 0x7fd7ac107840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd798009900 tx=0x7fd79802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/279237484 shutdown_connections 2024-04-16T05:34:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/279237484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac107d80 0x7fd7ac10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/279237484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac105450 0x7fd7ac107840 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/279237484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7ac102b20 0x7fd7ac104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/279237484 >> 172.21.15.32:0/279237484 conn(0x7fd7ac0fc4d0 msgr2=0x7fd7ac0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/279237484 shutdown_connections 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.972+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/279237484 wait complete. 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b2ac8640 1 Processor -- start 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b2ac8640 1 -- start start 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b2ac8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac102b20 0x7fd7ac1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b2ac8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac105450 0x7fd7ac198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b2ac8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7ac107d80 0x7fd7ac19fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b2ac8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac10dd30 con 0x7fd7ac102b20 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b2ac8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac10dbb0 con 0x7fd7ac105450 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b2ac8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac10deb0 con 0x7fd7ac107d80 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b0864640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac102b20 0x7fd7ac1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b0864640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac102b20 0x7fd7ac1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53578/0 (socket says 172.21.15.32:53578) 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b0864640 1 -- 172.21.15.32:0/1193012631 learned_addr learned my addr 172.21.15.32:0/1193012631 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7abfff640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac105450 0x7fd7ac198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b1065640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7ac107d80 0x7fd7ac19fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b0864640 1 -- 172.21.15.32:0/1193012631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7ac107d80 msgr2=0x7fd7ac19fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b0864640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7ac107d80 0x7fd7ac19fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b0864640 1 -- 172.21.15.32:0/1193012631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac105450 msgr2=0x7fd7ac198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b0864640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac105450 0x7fd7ac198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.973+0000 7fd7b0864640 1 -- 172.21.15.32:0/1193012631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac1a00a0 con 0x7fd7ac102b20 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.974+0000 7fd7b0864640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac102b20 0x7fd7ac1984d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000b4a0 tx=0x7fd7a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.974+0000 7fd7a9ffb640 1 -- 172.21.15.32:0/1193012631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0014070 con 0x7fd7ac102b20 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.974+0000 7fd7a9ffb640 1 -- 172.21.15.32:0/1193012631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7a0010af0 con 0x7fd7ac102b20 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.974+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac1a0390 con 0x7fd7ac102b20 2024-04-16T05:34:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.974+0000 7fd7a9ffb640 1 -- 172.21.15.32:0/1193012631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0017d10 con 0x7fd7ac102b20 2024-04-16T05:34:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.974+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac1a08b0 con 0x7fd7ac102b20 2024-04-16T05:34:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.975+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac0fdfc0 con 0x7fd7ac102b20 2024-04-16T05:34:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.976+0000 7fd7a9ffb640 1 -- 172.21.15.32:0/1193012631 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd7a0017070 con 0x7fd7ac102b20 2024-04-16T05:34:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.977+0000 7fd7a9ffb640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.977+0000 7fd7abfff640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.977+0000 7fd7a9ffb640 1 -- 172.21.15.32:0/1193012631 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd7a00cfbf0 con 0x7fd7ac102b20 2024-04-16T05:34:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.977+0000 7fd7abfff640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd79802c7b0 tx=0x7fd798005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:19.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:19.981+0000 7fd7a9ffb640 1 -- 172.21.15.32:0/1193012631 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7a009c210 con 0x7fd7ac102b20 2024-04-16T05:34:20.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.167+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd7ac062c00 con 0x7fd7ac102b20 2024-04-16T05:34:20.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.168+0000 7fd7a9ffb640 1 -- 172.21.15.32:0/1193012631 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fd7a009bb10 con 0x7fd7ac102b20 2024-04-16T05:34:20.166 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd79c0a3bb0 msgr2=0x7fd79c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd79802c7b0 tx=0x7fd798005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac102b20 msgr2=0x7fd7ac1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac102b20 0x7fd7ac1984d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000b4a0 tx=0x7fd7a000b960 comp rx=0 tx=0).stop 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 shutdown_connections 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7ac107d80 0x7fd7ac19fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac105450 0x7fd7ac198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 --2- 172.21.15.32:0/1193012631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7ac102b20 0x7fd7ac1984d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 >> 172.21.15.32:0/1193012631 conn(0x7fd7ac0fc4d0 msgr2=0x7fd7ac103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 shutdown_connections 2024-04-16T05:34:20.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.170+0000 7fd7b2ac8640 1 -- 172.21.15.32:0/1193012631 wait complete. 2024-04-16T05:34:20.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:34:20.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 -- 172.21.15.32:0/2086040471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 msgr2=0x7f857c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 --2- 172.21.15.32:0/2086040471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 0x7f857c10a490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8574009ca0 tx=0x7f857402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 -- 172.21.15.32:0/2086040471 shutdown_connections 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 --2- 172.21.15.32:0/2086040471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 0x7f857c10a490 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 --2- 172.21.15.32:0/2086040471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f857c103240 0x7f857c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 --2- 172.21.15.32:0/2086040471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857c102050 0x7f857c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 -- 172.21.15.32:0/2086040471 >> 172.21.15.32:0/2086040471 conn(0x7f857c0fd5e0 msgr2=0x7f857c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 -- 172.21.15.32:0/2086040471 shutdown_connections 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.289+0000 7f8584060640 1 -- 172.21.15.32:0/2086040471 wait complete. 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f8584060640 1 Processor -- start 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f8584060640 1 -- start start 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f8584060640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857c102050 0x7f857c075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f8584060640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f857c103240 0x7f857c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f8584060640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 0x7f857c074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f8584060640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10dce0 con 0x7f857c103c00 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f8584060640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10db60 con 0x7f857c102050 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f8584060640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10de60 con 0x7f857c103240 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f85825fd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 0x7f857c074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f85825fd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 0x7f857c074930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53600/0 (socket says 172.21.15.32:53600) 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.290+0000 7f85825fd640 1 -- 172.21.15.32:0/3019166547 learned_addr learned my addr 172.21.15.32:0/3019166547 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f85815fb640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f857c103240 0x7f857c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f8581dfc640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857c102050 0x7f857c075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f85825fd640 1 -- 172.21.15.32:0/3019166547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f857c103240 msgr2=0x7f857c0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f85825fd640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f857c103240 0x7f857c0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f85825fd640 1 -- 172.21.15.32:0/3019166547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857c102050 msgr2=0x7f857c075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f85825fd640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857c102050 0x7f857c075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f85825fd640 1 -- 172.21.15.32:0/3019166547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c0751c0 con 0x7f857c103c00 2024-04-16T05:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f8581dfc640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857c102050 0x7f857c075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f85825fd640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 0x7f857c074930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f857402c700 tx=0x7f8574032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f8572ffd640 1 -- 172.21.15.32:0/3019166547 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574039040 con 0x7f857c103c00 2024-04-16T05:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857c1a2420 con 0x7f857c103c00 2024-04-16T05:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f8572ffd640 1 -- 172.21.15.32:0/3019166547 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8574044070 con 0x7f857c103c00 2024-04-16T05:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f8572ffd640 1 -- 172.21.15.32:0/3019166547 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85740042a0 con 0x7f857c103c00 2024-04-16T05:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.291+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857c1a2970 con 0x7f857c103c00 2024-04-16T05:34:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.292+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857c109370 con 0x7f857c103c00 2024-04-16T05:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.293+0000 7f8572ffd640 1 -- 172.21.15.32:0/3019166547 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8574003970 con 0x7f857c103c00 2024-04-16T05:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.294+0000 7f8572ffd640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85680a3bb0 0x7f85680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.294+0000 7f8581dfc640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85680a3bb0 0x7f85680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.294+0000 7f8572ffd640 1 -- 172.21.15.32:0/3019166547 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f85740f2d20 con 0x7f857c103c00 2024-04-16T05:34:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.294+0000 7f8581dfc640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85680a3bb0 0x7f85680a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f857c1030a0 tx=0x7f8578005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.298+0000 7f8572ffd640 1 -- 172.21.15.32:0/3019166547 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85740333a0 con 0x7f857c103c00 2024-04-16T05:34:20.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:20 smithi032 ceph-mon[32351]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:34:20.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:20 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1070681775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:20.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:20 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3981258241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:20.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:20 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/925414018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.487+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f857c062c00 con 0x7f857c103c00 2024-04-16T05:34:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.487+0000 7f8572ffd640 1 -- 172.21.15.32:0/3019166547 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f85740bf430 con 0x7f857c103c00 2024-04-16T05:34:20.486 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85680a3bb0 msgr2=0x7f85680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85680a3bb0 0x7f85680a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f857c1030a0 tx=0x7f8578005e70 comp rx=0 tx=0).stop 2024-04-16T05:34:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 msgr2=0x7f857c074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 0x7f857c074930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f857402c700 tx=0x7f8574032f20 comp rx=0 tx=0).stop 2024-04-16T05:34:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 shutdown_connections 2024-04-16T05:34:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85680a3bb0 0x7f85680a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f857c103c00 0x7f857c074930 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f857c103240 0x7f857c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 --2- 172.21.15.32:0/3019166547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857c102050 0x7f857c075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 >> 172.21.15.32:0/3019166547 conn(0x7f857c0fd5e0 msgr2=0x7f857c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 shutdown_connections 2024-04-16T05:34:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.489+0000 7f8584060640 1 -- 172.21.15.32:0/3019166547 wait complete. 2024-04-16T05:34:20.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 -- 172.21.15.32:0/1828524633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c102aa0 msgr2=0x7f8f1c1093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 --2- 172.21.15.32:0/1828524633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c102aa0 0x7f8f1c1093c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009ca0 tx=0x7f8f1402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 -- 172.21.15.32:0/1828524633 shutdown_connections 2024-04-16T05:34:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 --2- 172.21.15.32:0/1828524633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c102aa0 0x7f8f1c1093c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 --2- 172.21.15.32:0/1828524633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f1c1020e0 0x7f8f1c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 --2- 172.21.15.32:0/1828524633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f1c100f10 0x7f8f1c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 -- 172.21.15.32:0/1828524633 >> 172.21.15.32:0/1828524633 conn(0x7f8f1c0fc4f0 msgr2=0x7f8f1c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 -- 172.21.15.32:0/1828524633 shutdown_connections 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 -- 172.21.15.32:0/1828524633 wait complete. 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.607+0000 7f8f22274640 1 Processor -- start 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f22274640 1 -- start start 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f22274640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f1c100f10 0x7f8f1c06b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f22274640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c1020e0 0x7f8f1c19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f22274640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f1c102aa0 0x7f8f1c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f22274640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10cc30 con 0x7f8f1c1020e0 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f22274640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10cab0 con 0x7f8f1c100f10 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f22274640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10cdb0 con 0x7f8f1c102aa0 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c1020e0 0x7f8f1c19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c1020e0 0x7f8f1c19d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53616/0 (socket says 172.21.15.32:53616) 2024-04-16T05:34:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1940179890 learned_addr learned my addr 172.21.15.32:0/1940179890 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1bfff640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f1c100f10 0x7f8f1c06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f20811640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f1c102aa0 0x7f8f1c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1940179890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f1c102aa0 msgr2=0x7f8f1c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1b7fe640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f1c102aa0 0x7f8f1c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1940179890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f1c100f10 msgr2=0x7f8f1c06b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1b7fe640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f1c100f10 0x7f8f1c06b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.608+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1940179890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c19ffa0 con 0x7f8f1c1020e0 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.609+0000 7f8f1b7fe640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c1020e0 0x7f8f1c19d070 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b4a0 tx=0x7f8f1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.609+0000 7f8f197fa640 1 -- 172.21.15.32:0/1940179890 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10014070 con 0x7f8f1c1020e0 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.609+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c1a0290 con 0x7f8f1c1020e0 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.609+0000 7f8f197fa640 1 -- 172.21.15.32:0/1940179890 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f10010af0 con 0x7f8f1c1020e0 2024-04-16T05:34:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.609+0000 7f8f197fa640 1 -- 172.21.15.32:0/1940179890 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10017d10 con 0x7f8f1c1020e0 2024-04-16T05:34:20.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.609+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c1a0830 con 0x7f8f1c1020e0 2024-04-16T05:34:20.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.610+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1c1083c0 con 0x7f8f1c1020e0 2024-04-16T05:34:20.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.611+0000 7f8f197fa640 1 -- 172.21.15.32:0/1940179890 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8f10017070 con 0x7f8f1c1020e0 2024-04-16T05:34:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.612+0000 7f8f197fa640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f040a3bb0 0x7f8f040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.612+0000 7f8f1bfff640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f040a3bb0 0x7f8f040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.612+0000 7f8f197fa640 1 -- 172.21.15.32:0/1940179890 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8f100cffa0 con 0x7f8f1c1020e0 2024-04-16T05:34:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.612+0000 7f8f1bfff640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f040a3bb0 0x7f8f040a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c005db0 tx=0x7f8f0c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.616+0000 7f8f197fa640 1 -- 172.21.15.32:0/1940179890 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f1009c620 con 0x7f8f1c1020e0 2024-04-16T05:34:20.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:20 smithi134 ceph-mon[36153]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:34:20.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:20 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1070681775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:20.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:20 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3981258241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:20.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:20 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/925414018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:20.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:20 smithi082 ceph-mon[36195]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:34:20.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:20 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1070681775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:20.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:20 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3981258241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:20.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:20 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/925414018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.839+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f1c02d860 con 0x7f8f1c1020e0 2024-04-16T05:34:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.839+0000 7f8f197fa640 1 -- 172.21.15.32:0/1940179890 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11876 (secure 0 0 0) 0x7f8f1009bf20 con 0x7f8f1c1020e0 2024-04-16T05:34:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:20.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:32:55.688634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24461,24494]},"id":1}]} 2024-04-16T05:34:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-16T05:34:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f040a3bb0 msgr2=0x7f8f040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f040a3bb0 0x7f8f040a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c005db0 tx=0x7f8f0c005d40 comp rx=0 tx=0).stop 2024-04-16T05:34:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c1020e0 msgr2=0x7f8f1c19d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c1020e0 0x7f8f1c19d070 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b4a0 tx=0x7f8f1000b960 comp rx=0 tx=0).stop 2024-04-16T05:34:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 shutdown_connections 2024-04-16T05:34:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f8f040a3bb0 0x7f8f040a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f1c102aa0 0x7f8f1c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1c1020e0 0x7f8f1c19d070 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 --2- 172.21.15.32:0/1940179890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f1c100f10 0x7f8f1c06b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.841+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 >> 172.21.15.32:0/1940179890 conn(0x7f8f1c0fc4f0 msgr2=0x7f8f1c105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.842+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 shutdown_connections 2024-04-16T05:34:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.842+0000 7f8f22274640 1 -- 172.21.15.32:0/1940179890 wait complete. 2024-04-16T05:34:20.852 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 -- 172.21.15.32:0/2719308031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 msgr2=0x7f74d41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 --2- 172.21.15.32:0/2719308031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 0x7f74d41093c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009ca0 tx=0x7f74cc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 -- 172.21.15.32:0/2719308031 shutdown_connections 2024-04-16T05:34:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 --2- 172.21.15.32:0/2719308031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 0x7f74d41093c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 --2- 172.21.15.32:0/2719308031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d41020e0 0x7f74d4102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 --2- 172.21.15.32:0/2719308031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d4100f10 0x7f74d4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 -- 172.21.15.32:0/2719308031 >> 172.21.15.32:0/2719308031 conn(0x7f74d40fc4f0 msgr2=0x7f74d40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 -- 172.21.15.32:0/2719308031 shutdown_connections 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.959+0000 7f74dc263640 1 -- 172.21.15.32:0/2719308031 wait complete. 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74dc263640 1 Processor -- start 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74dc263640 1 -- start start 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74dc263640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d4100f10 0x7f74d406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74dc263640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d41020e0 0x7f74d419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74dc263640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 0x7f74d419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74dc263640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f74d410cbb0 con 0x7f74d4102aa0 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74dc263640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f74d410ca30 con 0x7f74d41020e0 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74dc263640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f74d410cd30 con 0x7f74d4100f10 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74da800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 0x7f74d419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74da800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 0x7f74d419f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53644/0 (socket says 172.21.15.32:53644) 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74da800640 1 -- 172.21.15.32:0/3531494851 learned_addr learned my addr 172.21.15.32:0/3531494851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74d9fff640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d4100f10 0x7f74d406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74da800640 1 -- 172.21.15.32:0/3531494851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d4100f10 msgr2=0x7f74d406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74da800640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d4100f10 0x7f74d406b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74da800640 1 -- 172.21.15.32:0/3531494851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d41020e0 msgr2=0x7f74d419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74da800640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d41020e0 0x7f74d419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.960+0000 7f74da800640 1 -- 172.21.15.32:0/3531494851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d419fe70 con 0x7f74d4102aa0 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.961+0000 7f74da800640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 0x7f74d419f5b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f74cc02c700 tx=0x7f74cc032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.961+0000 7f74caffd640 1 -- 172.21.15.32:0/3531494851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc039040 con 0x7f74d4102aa0 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.961+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d41a0100 con 0x7f74d4102aa0 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.961+0000 7f74caffd640 1 -- 172.21.15.32:0/3531494851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74cc044070 con 0x7f74d4102aa0 2024-04-16T05:34:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.961+0000 7f74caffd640 1 -- 172.21.15.32:0/3531494851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0042a0 con 0x7f74d4102aa0 2024-04-16T05:34:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.961+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d41a0640 con 0x7f74d4102aa0 2024-04-16T05:34:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.962+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d4069270 con 0x7f74d4102aa0 2024-04-16T05:34:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.963+0000 7f74caffd640 1 -- 172.21.15.32:0/3531494851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f74cc003970 con 0x7f74d4102aa0 2024-04-16T05:34:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.964+0000 7f74caffd640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c00a3bb0 0x7f74c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.964+0000 7f74d9fff640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c00a3bb0 0x7f74c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.964+0000 7f74caffd640 1 -- 172.21.15.32:0/3531494851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f74cc0f3360 con 0x7f74d4102aa0 2024-04-16T05:34:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.964+0000 7f74d9fff640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c00a3bb0 0x7f74c00a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f74d0005d00 tx=0x7f74d0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:20.968+0000 7f74caffd640 1 -- 172.21.15.32:0/3531494851 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74cc0bfa70 con 0x7f74d4102aa0 2024-04-16T05:34:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.154+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74d4108310 con 0x7f74d4102aa0 2024-04-16T05:34:21.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.156+0000 7f74caffd640 1 -- 172.21.15.32:0/3531494851 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24618 (secure 0 0 0) 0x7f74cc047e40 con 0x7f74d4102aa0 2024-04-16T05:34:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:21.155 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:33:34.757328+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c00a3bb0 msgr2=0x7f74c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c00a3bb0 0x7f74c00a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f74d0005d00 tx=0x7f74d0005c50 comp rx=0 tx=0).stop 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 msgr2=0x7f74d419f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 0x7f74d419f5b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f74cc02c700 tx=0x7f74cc032f20 comp rx=0 tx=0).stop 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 shutdown_connections 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f74c00a3bb0 0x7f74c00a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d4102aa0 0x7f74d419f5b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d41020e0 0x7f74d419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 --2- 172.21.15.32:0/3531494851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d4100f10 0x7f74d406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 >> 172.21.15.32:0/3531494851 conn(0x7f74d40fc4f0 msgr2=0x7f74d41054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 shutdown_connections 2024-04-16T05:34:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.158+0000 7f74dc263640 1 -- 172.21.15.32:0/3531494851 wait complete. 2024-04-16T05:34:21.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 -- 172.21.15.32:0/3239426333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 msgr2=0x7f46d41091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 --2- 172.21.15.32:0/3239426333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 0x7f46d41091c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009ca0 tx=0x7f46cc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 -- 172.21.15.32:0/3239426333 shutdown_connections 2024-04-16T05:34:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 --2- 172.21.15.32:0/3239426333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 0x7f46d41091c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 --2- 172.21.15.32:0/3239426333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46d4101f70 0x7f46d41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 --2- 172.21.15.32:0/3239426333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d4100d80 0x7f46d41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 -- 172.21.15.32:0/3239426333 >> 172.21.15.32:0/3239426333 conn(0x7f46d40fc2d0 msgr2=0x7f46d40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 -- 172.21.15.32:0/3239426333 shutdown_connections 2024-04-16T05:34:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 -- 172.21.15.32:0/3239426333 wait complete. 2024-04-16T05:34:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 Processor -- start 2024-04-16T05:34:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.276+0000 7f46d359e640 1 -- start start 2024-04-16T05:34:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46d4100d80 0x7f46d4069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d4101f70 0x7f46d419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 0x7f46d419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d359e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410c9e0 con 0x7f46d4102930 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d359e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410c860 con 0x7f46d4101f70 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410cb60 con 0x7f46d4100d80 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 0x7f46d419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 0x7f46d419f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53650/0 (socket says 172.21.15.32:53650) 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 -- 172.21.15.32:0/3783611626 learned_addr learned my addr 172.21.15.32:0/3783611626 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d1d9b640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d4101f70 0x7f46d419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d259c640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46d4100d80 0x7f46d4069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 -- 172.21.15.32:0/3783611626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46d4100d80 msgr2=0x7f46d4069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46d4100d80 0x7f46d4069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 -- 172.21.15.32:0/3783611626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d4101f70 msgr2=0x7f46d419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d4101f70 0x7f46d419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 -- 172.21.15.32:0/3783611626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d419fcf0 con 0x7f46d4102930 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d259c640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46d4100d80 0x7f46d4069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d1d9b640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d4101f70 0x7f46d419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d2d9d640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 0x7f46d419f3e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009bf0 tx=0x7f46cc035ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46cb7fe640 1 -- 172.21.15.32:0/3783611626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc039040 con 0x7f46d4102930 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46cb7fe640 1 -- 172.21.15.32:0/3783611626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46cc044070 con 0x7f46d4102930 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d419ff20 con 0x7f46d4102930 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46cb7fe640 1 -- 172.21.15.32:0/3783611626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0043a0 con 0x7f46d4102930 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.277+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41a03e0 con 0x7f46d4102930 2024-04-16T05:34:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.278+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4107fd0 con 0x7f46d4102930 2024-04-16T05:34:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.279+0000 7f46cb7fe640 1 -- 172.21.15.32:0/3783611626 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f46cc003ae0 con 0x7f46d4102930 2024-04-16T05:34:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.280+0000 7f46cb7fe640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46c00a3c30 0x7f46c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.280+0000 7f46d259c640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46c00a3c30 0x7f46c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.280+0000 7f46cb7fe640 1 -- 172.21.15.32:0/3783611626 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f46cc0f2de0 con 0x7f46d4102930 2024-04-16T05:34:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.281+0000 7f46d259c640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46c00a3c30 0x7f46c00a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46d4101dd0 tx=0x7f46bc005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.285+0000 7f46cb7fe640 1 -- 172.21.15.32:0/3783611626 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46cc0bf460 con 0x7f46d4102930 2024-04-16T05:34:21.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:21 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1193012631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:21.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:21 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3019166547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:21.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:21 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1940179890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:21.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:21 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3531494851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:21.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.510+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46d4062ca0 con 0x7f46d4102930 2024-04-16T05:34:21.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.511+0000 7f46cb7fe640 1 -- 172.21.15.32:0/3783611626 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11876 (secure 0 0 0) 0x7f46cc0bed60 con 0x7f46d4102930 2024-04-16T05:34:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:21.510 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:32:55.688634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24461,24494]},"id":1}]} 2024-04-16T05:34:21.510 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46c00a3c30 msgr2=0x7f46c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46c00a3c30 0x7f46c00a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46d4101dd0 tx=0x7f46bc005db0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 msgr2=0x7f46d419f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 0x7f46d419f3e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009bf0 tx=0x7f46cc035ae0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 shutdown_connections 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f46c00a3c30 0x7f46c00a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d4102930 0x7f46d419f3e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d4101f70 0x7f46d419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 --2- 172.21.15.32:0/3783611626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46d4100d80 0x7f46d4069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 >> 172.21.15.32:0/3783611626 conn(0x7f46d40fc2d0 msgr2=0x7f46d41051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 shutdown_connections 2024-04-16T05:34:21.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.513+0000 7f46d359e640 1 -- 172.21.15.32:0/3783611626 wait complete. 2024-04-16T05:34:21.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-16T05:32:34.109354+0000', 'modified': '2024-04-16T05:32:55.688634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24461, 'mds_2': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/1431936644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 1431936644}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1431936644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/1367430466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 1367430466}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 1367430466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/2091760007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 2091760007}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2091760007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.134:6839/3919403886', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3919403886}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3919403886}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.82:6837/1413733447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 1413733447}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 1413733447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6839/2453097092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2453097092}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2453097092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24460, 24461, 24494]} 2024-04-16T05:34:21.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-16T05:34:21.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-16T05:34:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 -- 172.21.15.32:0/4191378493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90103c00 msgr2=0x7fae9010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 --2- 172.21.15.32:0/4191378493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90103c00 0x7fae9010a490 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae80009900 tx=0x7fae8002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 -- 172.21.15.32:0/4191378493 shutdown_connections 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 --2- 172.21.15.32:0/4191378493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90103c00 0x7fae9010a490 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 --2- 172.21.15.32:0/4191378493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90103240 0x7fae901036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 --2- 172.21.15.32:0/4191378493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae90102050 0x7fae90102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 -- 172.21.15.32:0/4191378493 >> 172.21.15.32:0/4191378493 conn(0x7fae900fd5e0 msgr2=0x7fae900ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 -- 172.21.15.32:0/4191378493 shutdown_connections 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 -- 172.21.15.32:0/4191378493 wait complete. 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 Processor -- start 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.632+0000 7fae95352640 1 -- start start 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae95352640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90102050 0x7fae901982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae95352640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae90103240 0x7fae90198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae95352640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90103c00 0x7fae9019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae95352640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fae9010dd30 con 0x7fae90102050 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae95352640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fae9010dbb0 con 0x7fae90103c00 2024-04-16T05:34:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae95352640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae9010deb0 con 0x7fae90103240 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90102050 0x7fae901982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90102050 0x7fae901982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53672/0 (socket says 172.21.15.32:53672) 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 -- 172.21.15.32:0/3339149523 learned_addr learned my addr 172.21.15.32:0/3339149523 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90103c00 0x7fae9019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae90103240 0x7fae90198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 -- 172.21.15.32:0/3339149523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae90103240 msgr2=0x7fae90198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae90103240 0x7fae90198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 -- 172.21.15.32:0/3339149523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90103c00 msgr2=0x7fae9019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90103c00 0x7fae9019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 -- 172.21.15.32:0/3339149523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9019ff10 con 0x7fae90102050 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae90103240 0x7fae90198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae8effd640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90102050 0x7fae901982f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae8800eaa0 tx=0x7fae8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae87fff640 1 -- 172.21.15.32:0/3339149523 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88016070 con 0x7fae90102050 2024-04-16T05:34:21.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae95352640 1 -- 172.21.15.32:0/3339149523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae901a0200 con 0x7fae90102050 2024-04-16T05:34:21.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae87fff640 1 -- 172.21.15.32:0/3339149523 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae88003e50 con 0x7fae90102050 2024-04-16T05:34:21.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae87fff640 1 -- 172.21.15.32:0/3339149523 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae880116d0 con 0x7fae90102050 2024-04-16T05:34:21.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.633+0000 7fae95352640 1 -- 172.21.15.32:0/3339149523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae901a0700 con 0x7fae90102050 2024-04-16T05:34:21.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:21 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1193012631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:21.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:21 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3019166547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:21.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:21 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1940179890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:21.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:21 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3531494851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae87fff640 1 -- 172.21.15.32:0/3339149523 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fae88019070 con 0x7fae90102050 2024-04-16T05:34:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae98018740 1 Processor -- start 2024-04-16T05:34:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae98018740 1 -- start start 2024-04-16T05:34:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae98018740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f0408ec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae98018740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae98018740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae98018740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f0407fb240 con 0x55f0408f1cd0 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae98018740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f0408589f0 con 0x55f0408ec170 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae98018740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f0408139e0 con 0x55f0408ecb60 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae87fff640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fae7c0a3bb0 0x7fae7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae8e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae8e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53688/0 (socket says 172.21.15.32:53688) 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.636+0000 7fae8e7fc640 1 -- 172.21.15.32:0/1778828876 learned_addr learned my addr 172.21.15.32:0/1778828876 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fae7c0a3bb0 0x7fae7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8effd640 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f0408ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae87fff640 1 -- 172.21.15.32:0/3339149523 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae880d0850 con 0x7fae90102050 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8e7fc640 1 -- 172.21.15.32:0/1778828876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 msgr2=0x55f0408f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8e7fc640 1 -- 172.21.15.32:0/1778828876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 msgr2=0x55f0408ec590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f0408ec590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8e7fc640 1 -- 172.21.15.32:0/1778828876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f040806bc0 con 0x55f0408f1cd0 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f0408ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f4160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fae78009c20 tx=0x7fae7800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b20cfd581d7a7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fae7c0a3bb0 0x7fae7c0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fae80036040 tx=0x7fae80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae857fa640 1 -- 172.21.15.32:0/1778828876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78016070 con 0x55f0408f1cd0 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae857fa640 1 -- 172.21.15.32:0/1778828876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78011950 con 0x55f0408f1cd0 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.637+0000 7fae857fa640 1 -- 172.21.15.32:0/1778828876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78010870 con 0x55f0408f1cd0 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 -- 172.21.15.32:0/1778828876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 msgr2=0x55f0408f4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f4160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fae78009c20 tx=0x7fae7800e6c0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 -- 172.21.15.32:0/1778828876 shutdown_connections 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f4160 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 --2- 172.21.15.32:0/1778828876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f0408ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 -- 172.21.15.32:0/1778828876 >> 172.21.15.32:0/1778828876 conn(0x55f0408e32d0 msgr2=0x55f0408e36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 -- 172.21.15.32:0/1778828876 shutdown_connections 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 -- 172.21.15.32:0/1778828876 wait complete. 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 -- start start 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.638+0000 7fae98018740 1 Processor -- start 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae98018740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae98018740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ecb60 0x55f0408f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae98018740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408f1cd0 0x55f0408fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae98018740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f0408520f0 con 0x55f0408ec170 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae98018740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f04084cdd0 con 0x55f0408ecb60 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae98018740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f04085acd0 con 0x55f0408f1cd0 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae98018740 10 client.0 _ll_register_callbacks cb 0x55f040843570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53702/0 (socket says 172.21.15.32:53702) 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 -- 172.21.15.32:0/4077879675 learned_addr learned my addr 172.21.15.32:0/4077879675 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ecb60 0x55f0408f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408f1cd0 0x55f0408fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 -- 172.21.15.32:0/4077879675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408f1cd0 msgr2=0x55f0408fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408f1cd0 0x55f0408fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 -- 172.21.15.32:0/4077879675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ecb60 msgr2=0x55f0408f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ecb60 0x55f0408f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 -- 172.21.15.32:0/4077879675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0405602d0 con 0x55f0408ec170 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae8effd640 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f6380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fae880cdff0 tx=0x7fae880203f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7080e285792d86bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae767fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae767fc640 1 -- 172.21.15.32:0/4077879675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae880a0b80 con 0x55f0408ec170 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae767fc640 1 -- 172.21.15.32:0/4077879675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae88087070 con 0x55f0408ec170 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.639+0000 7fae98018740 1 -- 172.21.15.32:0/4077879675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae78011950 con 0x55f0408ec170 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae767fc640 1 -- 172.21.15.32:0/4077879675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fae880a5440 con 0x55f0408ec170 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae98018740 10 client.15189 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae98018740 1 -- 172.21.15.32:0/4077879675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f04062a6b0 con 0x55f0408ec170 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae767fc640 1 -- 172.21.15.32:0/4077879675 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78011950 con 0x55f0408ec170 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae767fc640 1 -- 172.21.15.32:0/4077879675 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fae8800f040 con 0x55f0408ec170 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae98018740 10 client.15189 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae98018740 10 client.15189 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae98018740 20 client.15189 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae98018740 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f040902ff0 0x55f040904460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae98018740 4 client.15189 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae98018740 1 -- 172.21.15.32:0/4077879675 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0407d44c0 con 0x55f040902ff0 2024-04-16T05:34:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.640+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f040902ff0 0x55f040904460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.641+0000 7fae767fc640 10 client.15189 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.641+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f040902ff0 0x55f040904460 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=242016c55cee0c4c server_cookie=81009e9f9b5bc879 in_seq=0 out_seq=0 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.641+0000 7fae767fc640 1 -- 172.21.15.32:0/4077879675 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0407d44c0 con 0x55f040902ff0 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.641+0000 7fae767fc640 10 client.15189 handle_command_reply: tid=0 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 client.15189 shutdown 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 -- 172.21.15.32:0/4077879675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 msgr2=0x55f0408f6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f6380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fae880cdff0 tx=0x7fae880203f0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 -- 172.21.15.32:0/4077879675 shutdown_connections 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f040902ff0 0x55f040904460 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408f1cd0 0x55f0408fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ecb60 0x55f0408f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae767fc640 0 client.15189 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 --2- 172.21.15.32:0/4077879675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f6380 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 -- 172.21.15.32:0/4077879675 >> 172.21.15.32:0/4077879675 conn(0x55f0408e32d0 msgr2=0x55f0408e2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 -- 172.21.15.32:0/4077879675 shutdown_connections 2024-04-16T05:34:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 1 -- 172.21.15.32:0/4077879675 wait complete. 2024-04-16T05:34:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.642+0000 7fae98018740 20 client.15189 trim_cache size 0 max 16384 2024-04-16T05:34:21.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:21 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1193012631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:21.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:21 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3019166547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:21.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:21 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1940179890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:21.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:21 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3531494851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae98018740 1 Processor -- start 2024-04-16T05:34:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae98018740 1 -- start start 2024-04-16T05:34:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae98018740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f040901f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae98018740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae98018740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae98018740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f0409027a0 con 0x55f0408f1cd0 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae98018740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f040902e70 con 0x55f0408ec170 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae98018740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f040903540 con 0x55f0408ecb60 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae8f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.660+0000 7fae8f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53704/0 (socket says 172.21.15.32:53704) 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8f7fe640 1 -- 172.21.15.32:0/2545067018 learned_addr learned my addr 172.21.15.32:0/2545067018 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8effd640 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f040901f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8f7fe640 1 -- 172.21.15.32:0/2545067018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 msgr2=0x55f0408fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8f7fe640 1 -- 172.21.15.32:0/2545067018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 msgr2=0x55f040901f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f040901f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8f7fe640 1 -- 172.21.15.32:0/2545067018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0408edb50 con 0x55f0408f1cd0 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8effd640 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f040901f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f5f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fae8002c240 tx=0x7fae8002df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d6b09f2071bb3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae84ff9640 1 -- 172.21.15.32:0/2545067018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80055070 con 0x55f0408f1cd0 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae84ff9640 1 -- 172.21.15.32:0/2545067018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae800046c0 con 0x55f0408f1cd0 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.661+0000 7fae84ff9640 1 -- 172.21.15.32:0/2545067018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80003850 con 0x55f0408f1cd0 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- 172.21.15.32:0/2545067018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 msgr2=0x55f0408f5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f5f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fae8002c240 tx=0x7fae8002df90 comp rx=0 tx=0).stop 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- 172.21.15.32:0/2545067018 shutdown_connections 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408f1cd0 0x55f0408f5f90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 --2- 172.21.15.32:0/2545067018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408ec170 0x55f040901f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- 172.21.15.32:0/2545067018 >> 172.21.15.32:0/2545067018 conn(0x55f0408e32d0 msgr2=0x55f0408e36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- 172.21.15.32:0/2545067018 shutdown_connections 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- 172.21.15.32:0/2545067018 wait complete. 2024-04-16T05:34:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- start start 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 Processor -- start 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408f1cd0 0x55f0408fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f0409042e0 con 0x55f0408ec170 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f040903c10 con 0x55f0408f1cd0 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f0409049b0 con 0x55f0408ecb60 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.662+0000 7fae98018740 10 client.0 _ll_register_callbacks cb 0x7fae8809c5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53712/0 (socket says 172.21.15.32:53712) 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 -- 172.21.15.32:0/1029325192 learned_addr learned my addr 172.21.15.32:0/1029325192 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408f1cd0 0x55f0408fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 -- 172.21.15.32:0/1029325192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 msgr2=0x55f0408f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 -- 172.21.15.32:0/1029325192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408f1cd0 msgr2=0x55f0408fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408f1cd0 0x55f0408fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 -- 172.21.15.32:0/1029325192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0408fd3b0 con 0x55f0408ec170 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8f7fe640 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408f1cd0 0x55f0408fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae8effd640 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f9860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fae880991a0 tx=0x7fae8809c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df72811e2f6604 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae767fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae767fc640 1 -- 172.21.15.32:0/1029325192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8809c130 con 0x55f0408ec170 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae767fc640 1 -- 172.21.15.32:0/1029325192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae880bb550 con 0x55f0408ec170 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae98018740 1 -- 172.21.15.32:0/1029325192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae800046c0 con 0x55f0408ec170 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae767fc640 1 -- 172.21.15.32:0/1029325192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fae8809be30 con 0x55f0408ec170 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae98018740 10 client.15195 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.663+0000 7fae98018740 1 -- 172.21.15.32:0/1029325192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f0409697a0 con 0x55f0408ec170 2024-04-16T05:34:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae767fc640 1 -- 172.21.15.32:0/1029325192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae800046c0 con 0x55f0408ec170 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae767fc640 1 -- 172.21.15.32:0/1029325192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fae8809db00 con 0x55f0408ec170 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae98018740 10 client.15195 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae98018740 10 client.15195 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae98018740 20 client.15195 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae98018740 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f04096e300 0x55f04096d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae98018740 4 client.15195 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae98018740 1 -- 172.21.15.32:0/1029325192 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "status"}) -- 0x55f04085b4e0 con 0x55f04096e300 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f04096e300 0x55f04096d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae767fc640 10 client.15195 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.664+0000 7fae8e7fc640 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f04096e300 0x55f04096d010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=609e110b506b3987 server_cookie=ec1a9034d77c92a4 in_seq=0 out_seq=0 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae767fc640 1 -- 172.21.15.32:0/1029325192 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f04085b4e0 con 0x55f04096e300 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae767fc640 10 client.15195 handle_command_reply: tid=0 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 client.15195 shutdown 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 -- 172.21.15.32:0/1029325192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 msgr2=0x55f0408f9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f9860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fae880991a0 tx=0x7fae8809c060 comp rx=0 tx=0).stop 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 -- 172.21.15.32:0/1029325192 shutdown_connections 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f04096e300 0x55f04096d010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0408f1cd0 0x55f0408fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0408ecb60 0x55f0408f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 --2- 172.21.15.32:0/1029325192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0408ec170 0x55f0408f9860 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae767fc640 0 client.15195 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 -- 172.21.15.32:0/1029325192 >> 172.21.15.32:0/1029325192 conn(0x55f0408e32d0 msgr2=0x55f040901f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 -- 172.21.15.32:0/1029325192 shutdown_connections 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 1 -- 172.21.15.32:0/1029325192 wait complete. 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae98018740 20 client.15195 trim_cache size 0 max 16384 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:34:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24460, 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 86.985094738000001, 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 24, 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 91, 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 119.873103771 2024-04-16T05:34:21.671 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae95352640 1 -- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fae7c0a3bb0 msgr2=0x7fae7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae95352640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fae7c0a3bb0 0x7fae7c0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fae80036040 tx=0x7fae80005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae95352640 1 -- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90102050 msgr2=0x7fae901982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae95352640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90102050 0x7fae901982f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae8800eaa0 tx=0x7fae8800ef60 comp rx=0 tx=0).stop 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae95352640 1 -- 172.21.15.32:0/3339149523 shutdown_connections 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.665+0000 7fae95352640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fae7c0a3bb0 0x7fae7c0a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.666+0000 7fae95352640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90103c00 0x7fae9019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.666+0000 7fae95352640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae90103240 0x7fae90198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.666+0000 7fae95352640 1 --2- 172.21.15.32:0/3339149523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90102050 0x7fae901982f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.666+0000 7fae95352640 1 -- 172.21.15.32:0/3339149523 >> 172.21.15.32:0/3339149523 conn(0x7fae900fd5e0 msgr2=0x7fae901064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.666+0000 7fae95352640 1 -- 172.21.15.32:0/3339149523 shutdown_connections 2024-04-16T05:34:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.666+0000 7fae95352640 1 -- 172.21.15.32:0/3339149523 wait complete. 2024-04-16T05:34:21.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 -- 172.21.15.32:0/1831356950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 msgr2=0x7f0f2c10a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 --2- 172.21.15.32:0/1831356950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 0x7f0f2c10a170 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009ca0 tx=0x7f0f2402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 -- 172.21.15.32:0/1831356950 shutdown_connections 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 --2- 172.21.15.32:0/1831356950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 0x7f0f2c10a170 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 --2- 172.21.15.32:0/1831356950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c1053f0 0x7f0f2c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 --2- 172.21.15.32:0/1831356950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c102ac0 0x7f0f2c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 -- 172.21.15.32:0/1831356950 >> 172.21.15.32:0/1831356950 conn(0x7f0f2c0fc4f0 msgr2=0x7f0f2c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 -- 172.21.15.32:0/1831356950 shutdown_connections 2024-04-16T05:34:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.782+0000 7f0f308f2640 1 -- 172.21.15.32:0/1831356950 wait complete. 2024-04-16T05:34:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f308f2640 1 Processor -- start 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f308f2640 1 -- start start 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f308f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102ac0 0x7f0f2c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f308f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c1053f0 0x7f0f2c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f308f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 0x7f0f2c19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f308f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10dd30 con 0x7f0f2c107d20 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f308f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10dbb0 con 0x7f0f2c1053f0 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f308f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10deb0 con 0x7f0f2c102ac0 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 0x7f0f2c19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 0x7f0f2c19f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53732/0 (socket says 172.21.15.32:53732) 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 -- 172.21.15.32:0/3520592736 learned_addr learned my addr 172.21.15.32:0/3520592736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102ac0 0x7f0f2c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f23fff640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c1053f0 0x7f0f2c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 -- 172.21.15.32:0/3520592736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102ac0 msgr2=0x7f0f2c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102ac0 0x7f0f2c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 -- 172.21.15.32:0/3520592736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c1053f0 msgr2=0x7f0f2c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c1053f0 0x7f0f2c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 -- 172.21.15.32:0/3520592736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c19ffb0 con 0x7f0f2c107d20 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.783+0000 7f0f2ad9d640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 0x7f0f2c19f940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0f2402c700 tx=0x7f0f24032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.784+0000 7f0f237fe640 1 -- 172.21.15.32:0/3520592736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24039040 con 0x7f0f2c107d20 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.784+0000 7f0f237fe640 1 -- 172.21.15.32:0/3520592736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f24044070 con 0x7f0f2c107d20 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.784+0000 7f0f237fe640 1 -- 172.21.15.32:0/3520592736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240042a0 con 0x7f0f2c107d20 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.784+0000 7f0f308f2640 1 -- 172.21.15.32:0/3520592736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c1a0240 con 0x7f0f2c107d20 2024-04-16T05:34:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.784+0000 7f0f308f2640 1 -- 172.21.15.32:0/3520592736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c1a0730 con 0x7f0f2c107d20 2024-04-16T05:34:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f237fe640 1 -- 172.21.15.32:0/3520592736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0f24003970 con 0x7f0f2c107d20 2024-04-16T05:34:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f335b8740 1 Processor -- start 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f335b8740 1 -- start start 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229bfd60 0x55a0229c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229c58c0 0x55a0229c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f335b8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a0228d3240 con 0x55a0229bfd60 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f335b8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a0229309f0 con 0x55a0229c58c0 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.786+0000 7f0f335b8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a0228eb9e0 con 0x55a0229c0750 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229bfd60 0x55a0229c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229bfd60 0x55a0229c0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53742/0 (socket says 172.21.15.32:53742) 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 -- 172.21.15.32:0/1224419804 learned_addr learned my addr 172.21.15.32:0/1224419804 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f237fe640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0f180a3bb0 0x7f0f180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229c58c0 0x55a0229c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 -- 172.21.15.32:0/1224419804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 msgr2=0x55a0229c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f237fe640 1 -- 172.21.15.32:0/3520592736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0f240f2d70 con 0x7f0f2c107d20 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 -- 172.21.15.32:0/1224419804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229c58c0 msgr2=0x55a0229c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f2ad9d640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0f180a3bb0 0x7f0f180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229c58c0 0x55a0229c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 -- 172.21.15.32:0/1224419804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0228debc0 con 0x55a0229bfd60 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229bfd60 0x55a0229c0180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400bef0 tx=0x7f0f1400b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90631a3d73156ded server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f20ff9640 1 -- 172.21.15.32:0/1224419804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f140173f0 con 0x55a0229bfd60 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f2ad9d640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0f180a3bb0 0x7f0f180a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0f240c7430 tx=0x7f0f240095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f20ff9640 1 -- 172.21.15.32:0/1224419804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f14010040 con 0x55a0229bfd60 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.787+0000 7f0f20ff9640 1 -- 172.21.15.32:0/1224419804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14016930 con 0x55a0229bfd60 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 -- 172.21.15.32:0/1224419804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229bfd60 msgr2=0x55a0229c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229bfd60 0x55a0229c0180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400bef0 tx=0x7f0f1400b400 comp rx=0 tx=0).stop 2024-04-16T05:34:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 -- 172.21.15.32:0/1224419804 shutdown_connections 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229c58c0 0x55a0229c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1224419804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229bfd60 0x55a0229c0180 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 -- 172.21.15.32:0/1224419804 >> 172.21.15.32:0/1224419804 conn(0x55a0229bb2d0 msgr2=0x55a0229bb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 -- 172.21.15.32:0/1224419804 shutdown_connections 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 -- 172.21.15.32:0/1224419804 wait complete. 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.788+0000 7f0f335b8740 1 -- start start 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f335b8740 1 Processor -- start 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c0750 0x55a0229ca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c58c0 0x55a0229d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f335b8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a02292a0f0 con 0x55a0229c0750 2024-04-16T05:34:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f335b8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a022924dd0 con 0x55a0229bfd60 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f335b8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a022932cd0 con 0x55a0229c58c0 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f335b8740 10 client.0 _ll_register_callbacks cb 0x55a02291b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c0750 0x55a0229ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c0750 0x55a0229ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53750/0 (socket says 172.21.15.32:53750) 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 -- 172.21.15.32:0/4278192854 learned_addr learned my addr 172.21.15.32:0/4278192854 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f23fff640 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 -- 172.21.15.32:0/4278192854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c58c0 msgr2=0x55a0229d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f23fff640 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c58c0 0x55a0229d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c58c0 0x55a0229d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 -- 172.21.15.32:0/4278192854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 msgr2=0x55a0229c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 -- 172.21.15.32:0/4278192854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0226382d0 con 0x55a0229c0750 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c0750 0x55a0229ca490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00bf00 tx=0x7f0f1c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a7dc8856630fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.789+0000 7f0f11ffb640 1 -- 172.21.15.32:0/4278192854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c01f060 con 0x55a0229c0750 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f11ffb640 1 -- 172.21.15.32:0/4278192854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0f1c0108a0 con 0x55a0229c0750 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 1 -- 172.21.15.32:0/4278192854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f14010040 con 0x55a0229c0750 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f11ffb640 1 -- 172.21.15.32:0/4278192854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0f1c026050 con 0x55a0229c0750 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 10 client.15207 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 1 -- 172.21.15.32:0/4278192854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a0227026b0 con 0x55a0229c0750 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f11ffb640 1 -- 172.21.15.32:0/4278192854 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f14010040 con 0x55a0229c0750 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f11ffb640 1 -- 172.21.15.32:0/4278192854 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0f1c019030 con 0x55a0229c0750 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 10 client.15207 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 10 client.15207 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 20 client.15207 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55a0229d6cf0 0x55a0229d8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 4 client.15207 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f335b8740 1 -- 172.21.15.32:0/4278192854 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0228ac4c0 con 0x55a0229d6cf0 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.790+0000 7f0f23fff640 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55a0229d6cf0 0x55a0229d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.791+0000 7f0f23fff640 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55a0229d6cf0 0x55a0229d8160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=817586e110007060 server_cookie=ee2b0ac1def62884 in_seq=0 out_seq=0 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.791+0000 7f0f11ffb640 10 client.15207 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f11ffb640 1 -- 172.21.15.32:0/4278192854 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0228ac4c0 con 0x55a0229d6cf0 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f11ffb640 10 client.15207 handle_command_reply: tid=0 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 client.15207 shutdown 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 -- 172.21.15.32:0/4278192854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c0750 msgr2=0x55a0229ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c0750 0x55a0229ca490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00bf00 tx=0x7f0f1c012820 comp rx=0 tx=0).stop 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 -- 172.21.15.32:0/4278192854 shutdown_connections 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55a0229d6cf0 0x55a0229d8160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c58c0 0x55a0229d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f11ffb640 0 client.15207 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c0750 0x55a0229ca490 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 --2- 172.21.15.32:0/4278192854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 -- 172.21.15.32:0/4278192854 >> 172.21.15.32:0/4278192854 conn(0x55a0229bb2d0 msgr2=0x55a0229ba410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 -- 172.21.15.32:0/4278192854 shutdown_connections 2024-04-16T05:34:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 1 -- 172.21.15.32:0/4278192854 wait complete. 2024-04-16T05:34:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.792+0000 7f0f335b8740 20 client.15207 trim_cache size 0 max 16384 2024-04-16T05:34:21.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f335b8740 1 Processor -- start 2024-04-16T05:34:21.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f335b8740 1 -- start start 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f335b8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a0229d64a0 con 0x55a0229c58c0 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f335b8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a0229d6b70 con 0x55a0229bfd60 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f335b8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a0229d7240 con 0x55a0229c0750 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53752/0 (socket says 172.21.15.32:53752) 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f23fff640 1 -- 172.21.15.32:0/1223897090 learned_addr learned my addr 172.21.15.32:0/1223897090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f23fff640 1 -- 172.21.15.32:0/1223897090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 msgr2=0x55a0229bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1223897090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.810+0000 7f0f23fff640 1 -- 172.21.15.32:0/1223897090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 msgr2=0x55a0229c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1223897090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f23fff640 1 -- 172.21.15.32:0/1223897090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0229bc040 con 0x55a0229c58c0 2024-04-16T05:34:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1223897090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229c9b60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400bef0 tx=0x7f0f1400bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c639bcd6d85f325 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f13fff640 1 -- 172.21.15.32:0/1223897090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400fe60 con 0x55a0229c58c0 2024-04-16T05:34:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f13fff640 1 -- 172.21.15.32:0/1223897090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f1402d030 con 0x55a0229c58c0 2024-04-16T05:34:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f13fff640 1 -- 172.21.15.32:0/1223897090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f140176e0 con 0x55a0229c58c0 2024-04-16T05:34:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f335b8740 1 -- 172.21.15.32:0/1223897090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 msgr2=0x55a0229c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1223897090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229c9b60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400bef0 tx=0x7f0f1400bcc0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.811+0000 7f0f335b8740 1 -- 172.21.15.32:0/1223897090 shutdown_connections 2024-04-16T05:34:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1223897090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229c9b60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1223897090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1223897090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 -- 172.21.15.32:0/1223897090 >> 172.21.15.32:0/1223897090 conn(0x55a0229bb2d0 msgr2=0x55a0229d5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 -- 172.21.15.32:0/1223897090 shutdown_connections 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 -- 172.21.15.32:0/1223897090 wait complete. 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 -- start start 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 Processor -- start 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a0229d7fe0 con 0x55a0229c58c0 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a0229d7910 con 0x55a0229bfd60 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a0229d86b0 con 0x55a0229c0750 2024-04-16T05:34:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f335b8740 10 client.0 _ll_register_callbacks cb 0x7f0f1c00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53764/0 (socket says 172.21.15.32:53764) 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f23fff640 1 -- 172.21.15.32:0/1575772658 learned_addr learned my addr 172.21.15.32:0/1575772658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f23fff640 1 -- 172.21.15.32:0/1575772658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 msgr2=0x55a0229cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f23fff640 1 -- 172.21.15.32:0/1575772658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 msgr2=0x55a0229cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f23fff640 1 -- 172.21.15.32:0/1575772658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0229d05f0 con 0x55a0229c58c0 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.812+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f23fff640 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229cff00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400b870 tx=0x7f0f1400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8f984608713680 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f11ffb640 1 -- 172.21.15.32:0/1575772658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14029c60 con 0x55a0229c58c0 2024-04-16T05:34:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f11ffb640 1 -- 172.21.15.32:0/1575772658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0f14029e00 con 0x55a0229c58c0 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 1 -- 172.21.15.32:0/1575772658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f1402d030 con 0x55a0229c58c0 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f11ffb640 1 -- 172.21.15.32:0/1575772658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0f140059c0 con 0x55a0229c58c0 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 10 client.15213 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 1 -- 172.21.15.32:0/1575772658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a022a41a10 con 0x55a0229c58c0 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f11ffb640 1 -- 172.21.15.32:0/1575772658 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f1402d030 con 0x55a0229c58c0 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f11ffb640 1 -- 172.21.15.32:0/1575772658 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0f14032420 con 0x55a0229c58c0 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 10 client.15213 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 10 client.15213 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 20 client.15213 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55a022a45ff0 0x55a022a44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 4 client.15213 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.813+0000 7f0f335b8740 1 -- 172.21.15.32:0/1575772658 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "status"}) -- 0x55a0229334e0 con 0x55a022a45ff0 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55a022a45ff0 0x55a022a44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f2a59c640 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55a022a45ff0 0x55a022a44d00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=995fcab3da6d1ce6 server_cookie=464cb85714831cdb in_seq=0 out_seq=0 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f11ffb640 10 client.15213 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f11ffb640 1 -- 172.21.15.32:0/1575772658 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a0229334e0 con 0x55a022a45ff0 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f11ffb640 10 client.15213 handle_command_reply: tid=0 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f335b8740 1 client.15213 shutdown 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f335b8740 1 -- 172.21.15.32:0/1575772658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 msgr2=0x55a0229cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229cff00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400b870 tx=0x7f0f1400ff60 comp rx=0 tx=0).stop 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.814+0000 7f0f335b8740 1 -- 172.21.15.32:0/1575772658 shutdown_connections 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55a022a45ff0 0x55a022a44d00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0229c58c0 0x55a0229cff00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0229c0750 0x55a0229cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f11ffb640 0 client.15213 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f335b8740 1 --2- 172.21.15.32:0/1575772658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0229bfd60 0x55a0229cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f335b8740 1 -- 172.21.15.32:0/1575772658 >> 172.21.15.32:0/1575772658 conn(0x55a0229bb2d0 msgr2=0x55a0229bb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f335b8740 1 -- 172.21.15.32:0/1575772658 shutdown_connections 2024-04-16T05:34:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f335b8740 1 -- 172.21.15.32:0/1575772658 wait complete. 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f335b8740 20 client.15213 trim_cache size 0 max 16384 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24461, 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 88.141305860000003, 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 24, 2024-04-16T05:34:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 91, 2024-04-16T05:34:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T05:34:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 122.83962354800001 2024-04-16T05:34:21.821 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:34:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 -- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0f180a3bb0 msgr2=0x7f0f180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0f180a3bb0 0x7f0f180a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0f240c7430 tx=0x7f0f240095c0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 -- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 msgr2=0x7f0f2c19f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 0x7f0f2c19f940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0f2402c700 tx=0x7f0f24032f20 comp rx=0 tx=0).stop 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 -- 172.21.15.32:0/3520592736 shutdown_connections 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0f180a3bb0 0x7f0f180a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f2c107d20 0x7f0f2c19f940 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c1053f0 0x7f0f2c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 --2- 172.21.15.32:0/3520592736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102ac0 0x7f0f2c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 -- 172.21.15.32:0/3520592736 >> 172.21.15.32:0/3520592736 conn(0x7f0f2c0fc4f0 msgr2=0x7f0f2c103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 -- 172.21.15.32:0/3520592736 shutdown_connections 2024-04-16T05:34:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.815+0000 7f0f308f2640 1 -- 172.21.15.32:0/3520592736 wait complete. 2024-04-16T05:34:21.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2604004159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca01054b0 msgr2=0x7f7ca01078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2604004159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca01054b0 0x7f7ca01078a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009900 tx=0x7f7c8802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2604004159 shutdown_connections 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2604004159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0107de0 0x7f7ca010a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2604004159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca01054b0 0x7f7ca01078a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2604004159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0102b80 0x7f7ca0104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2604004159 >> 172.21.15.32:0/2604004159 conn(0x7f7ca00fc4b0 msgr2=0x7f7ca00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2604004159 shutdown_connections 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2604004159 wait complete. 2024-04-16T05:34:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.933+0000 7f7ca5a88640 1 Processor -- start 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7ca5a88640 1 -- start start 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7ca5a88640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca0102b80 0x7f7ca01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7ca5a88640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca01054b0 0x7f7ca0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7ca5a88640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0107de0 0x7f7ca019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7ca5a88640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010dce0 con 0x7f7ca0102b80 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7ca5a88640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010db60 con 0x7f7ca01054b0 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7ca5a88640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010de60 con 0x7f7ca0107de0 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca0102b80 0x7f7ca01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca0102b80 0x7f7ca01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53780/0 (socket says 172.21.15.32:53780) 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/2629210839 learned_addr learned my addr 172.21.15.32:0/2629210839 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0107de0 0x7f7ca019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca01054b0 0x7f7ca0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/2629210839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0107de0 msgr2=0x7f7ca019f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0107de0 0x7f7ca019f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/2629210839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca01054b0 msgr2=0x7f7ca0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca01054b0 0x7f7ca0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.934+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/2629210839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca019ffe0 con 0x7f7ca0102b80 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.935+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca0102b80 0x7f7ca01983c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c890 tx=0x7f7c9000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.935+0000 7f7c9d7fa640 1 -- 172.21.15.32:0/2629210839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90004220 con 0x7f7ca0102b80 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.935+0000 7f7c9d7fa640 1 -- 172.21.15.32:0/2629210839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c900043c0 con 0x7f7ca0102b80 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.935+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2629210839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca01a02d0 con 0x7f7ca0102b80 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.935+0000 7f7c9d7fa640 1 -- 172.21.15.32:0/2629210839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900049e0 con 0x7f7ca0102b80 2024-04-16T05:34:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.935+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2629210839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca01a0840 con 0x7f7ca0102b80 2024-04-16T05:34:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.937+0000 7f7c9d7fa640 1 -- 172.21.15.32:0/2629210839 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7c900079b0 con 0x7f7ca0102b80 2024-04-16T05:34:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7ca867c740 1 Processor -- start 2024-04-16T05:34:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7ca867c740 1 -- start start 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae5d60 0x564e6fae6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faeb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faedd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7ca867c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564e6f9f9240 con 0x564e6faeb8c0 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7ca867c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564e6fa569f0 con 0x564e6fae6750 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7ca867c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564e6fa119e0 con 0x564e6fae5d60 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c96ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faedd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c9d7fa640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7c8c0a3c30 0x7f7c8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c96ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faedd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53794/0 (socket says 172.21.15.32:53794) 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c96ffd640 1 -- 172.21.15.32:0/2869996857 learned_addr learned my addr 172.21.15.32:0/2869996857 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faeb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7c8c0a3c30 0x7f7c8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c9d7fa640 1 -- 172.21.15.32:0/2629210839 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7c900d06c0 con 0x7f7ca0102b80 2024-04-16T05:34:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae5d60 0x564e6fae6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c96ffd640 1 -- 172.21.15.32:0/2869996857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae5d60 msgr2=0x564e6fae6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae5d60 0x564e6fae6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.938+0000 7f7c96ffd640 1 -- 172.21.15.32:0/2869996857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 msgr2=0x564e6faeb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.939+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faeb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.939+0000 7f7c96ffd640 1 -- 172.21.15.32:0/2869996857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e6fa04bc0 con 0x564e6faeb8c0 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.939+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae5d60 0x564e6fae6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.939+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faedd50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7ca006b0c0 tx=0x7f7c88005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b06041c8400b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.939+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7c8c0a3c30 0x7f7c8c0a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x564e6fae6fb0 tx=0x7f7c98009840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.939+0000 7f7c95ffb640 1 -- 172.21.15.32:0/2869996857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8802cc80 con 0x564e6faeb8c0 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.939+0000 7f7c95ffb640 1 -- 172.21.15.32:0/2869996857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c88005040 con 0x564e6faeb8c0 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.939+0000 7f7c95ffb640 1 -- 172.21.15.32:0/2869996857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88004ae0 con 0x564e6faeb8c0 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 -- 172.21.15.32:0/2869996857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 msgr2=0x564e6faedd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faedd50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7ca006b0c0 tx=0x7f7c88005dd0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 -- 172.21.15.32:0/2869996857 shutdown_connections 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faedd50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faeb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2869996857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae5d60 0x564e6fae6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 -- 172.21.15.32:0/2869996857 >> 172.21.15.32:0/2869996857 conn(0x564e6fae12d0 msgr2=0x564e6fae16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 -- 172.21.15.32:0/2869996857 shutdown_connections 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 -- 172.21.15.32:0/2869996857 wait complete. 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 -- start start 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 Processor -- start 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.940+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae5d60 0x564e6faeffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae6750 0x564e6faf04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faf7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564e6fa500f0 con 0x564e6faeb8c0 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564e6fa4add0 con 0x564e6fae5d60 2024-04-16T05:34:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564e6fa58cd0 con 0x564e6fae6750 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 10 client.0 _ll_register_callbacks cb 0x564e6fa41570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faf7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faf7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53810/0 (socket says 172.21.15.32:53810) 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 -- 172.21.15.32:0/1690875211 learned_addr learned my addr 172.21.15.32:0/1690875211 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae5d60 0x564e6faeffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 -- 172.21.15.32:0/1690875211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae6750 msgr2=0x564e6faf04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae6750 0x564e6faf04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 -- 172.21.15.32:0/1690875211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae5d60 msgr2=0x564e6faeffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae5d60 0x564e6faeffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 -- 172.21.15.32:0/1690875211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e6f75e2d0 con 0x564e6faeb8c0 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faf7520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c98011a80 tx=0x7f7c98004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c1a353a08dab13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c7effd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c7effd640 1 -- 172.21.15.32:0/1690875211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9802a060 con 0x564e6faeb8c0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c7effd640 1 -- 172.21.15.32:0/1690875211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7c9801dc50 con 0x564e6faeb8c0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 1 -- 172.21.15.32:0/1690875211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c88005040 con 0x564e6faeb8c0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7c7effd640 1 -- 172.21.15.32:0/1690875211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c98031050 con 0x564e6faeb8c0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 10 client.15225 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.941+0000 7f7ca867c740 1 -- 172.21.15.32:0/1690875211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564e6f8286b0 con 0x564e6faeb8c0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7c7effd640 1 -- 172.21.15.32:0/1690875211 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c88005040 con 0x564e6faeb8c0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7c7effd640 1 -- 172.21.15.32:0/1690875211 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7c98019040 con 0x564e6faeb8c0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7ca867c740 10 client.15225 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7ca867c740 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7ca867c740 20 client.15225 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7ca867c740 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564e6fafcce0 0x564e6fafe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7ca867c740 4 client.15225 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7ca867c740 1 -- 172.21.15.32:0/1690875211 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e6f9d24c0 con 0x564e6fafcce0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564e6fafcce0 0x564e6fafe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7c7effd640 10 client.15225 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.942+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564e6fafcce0 0x564e6fafe150 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e70a832f2bc3a3 server_cookie=d7aee8c4d9af85ac in_seq=0 out_seq=0 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.943+0000 7f7c7effd640 1 -- 172.21.15.32:0/1690875211 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e6f9d24c0 con 0x564e6fafcce0 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.943+0000 7f7c7effd640 10 client.15225 handle_command_reply: tid=0 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.943+0000 7f7ca867c740 1 client.15225 shutdown 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.943+0000 7f7ca867c740 1 -- 172.21.15.32:0/1690875211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 msgr2=0x564e6faf7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.943+0000 7f7ca867c740 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faf7520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c98011a80 tx=0x7f7c98004ab0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 1 -- 172.21.15.32:0/1690875211 shutdown_connections 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6faeb8c0 0x564e6faf7520 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564e6fafcce0 0x564e6fafe150 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6fae6750 0x564e6faf04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7c7effd640 0 client.15225 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 1 --2- 172.21.15.32:0/1690875211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae5d60 0x564e6faeffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 1 -- 172.21.15.32:0/1690875211 >> 172.21.15.32:0/1690875211 conn(0x564e6fae12d0 msgr2=0x564e6fae0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 1 -- 172.21.15.32:0/1690875211 shutdown_connections 2024-04-16T05:34:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 1 -- 172.21.15.32:0/1690875211 wait complete. 2024-04-16T05:34:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.944+0000 7f7ca867c740 20 client.15225 trim_cache size 0 max 16384 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7ca867c740 1 Processor -- start 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7ca867c740 1 -- start start 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6fae7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faeffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7ca867c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564e6fafc490 con 0x564e6fae5d60 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7ca867c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564e6fafcb60 con 0x564e6fae6750 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7ca867c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564e6fafd230 con 0x564e6faeb8c0 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6fae7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6fae7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53818/0 (socket says 172.21.15.32:53818) 2024-04-16T05:34:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/3911047001 learned_addr learned my addr 172.21.15.32:0/3911047001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faeffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/3911047001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 msgr2=0x564e6faeffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faeffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/3911047001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 msgr2=0x564e6faf74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.962+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/3911047001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e6fae6b30 con 0x564e6fae5d60 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faeffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6fae7150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7c90007590 tx=0x7f7c900148a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eded9f5adf3f677f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7c957fa640 1 -- 172.21.15.32:0/3911047001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9001d040 con 0x564e6fae5d60 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7c957fa640 1 -- 172.21.15.32:0/3911047001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c900a09f0 con 0x564e6fae5d60 2024-04-16T05:34:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7c957fa640 1 -- 172.21.15.32:0/3911047001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90016db0 con 0x564e6fae5d60 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 -- 172.21.15.32:0/3911047001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 msgr2=0x564e6fae7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6fae7150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7c90007590 tx=0x7f7c900148a0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 -- 172.21.15.32:0/3911047001 shutdown_connections 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faeffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 --2- 172.21.15.32:0/3911047001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6fae7150 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 -- 172.21.15.32:0/3911047001 >> 172.21.15.32:0/3911047001 conn(0x564e6fae12d0 msgr2=0x564e6fae16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 -- 172.21.15.32:0/3911047001 shutdown_connections 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.963+0000 7f7ca867c740 1 -- 172.21.15.32:0/3911047001 wait complete. 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 1 -- start start 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 1 Processor -- start 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6faf2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faf5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564e6fafdfd0 con 0x564e6fae5d60 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564e6fafd900 con 0x564e6fae6750 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564e6fafe6a0 con 0x564e6faeb8c0 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7ca867c740 10 client.0 _ll_register_callbacks cb 0x7f7c98009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7c9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6faf2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7c9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6faf2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53820/0 (socket says 172.21.15.32:53820) 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/2106360372 learned_addr learned my addr 172.21.15.32:0/2106360372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faf5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/2106360372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 msgr2=0x564e6faf5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faf5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.964+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 msgr2=0x564e6faf2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c9f7fe640 1 -- 172.21.15.32:0/2106360372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e6faf63a0 con 0x564e6fae5d60 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c9ffff640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faf5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c9f7fe640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6faf2850 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7c90016000 tx=0x7f7c9009a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8093fd8ce5887f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c7effd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c7effd640 1 -- 172.21.15.32:0/2106360372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9001d030 con 0x564e6fae5d60 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c7effd640 1 -- 172.21.15.32:0/2106360372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7c900d03a0 con 0x564e6fae5d60 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 1 -- 172.21.15.32:0/2106360372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c900a09f0 con 0x564e6fae5d60 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c7effd640 1 -- 172.21.15.32:0/2106360372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c90098d60 con 0x564e6fae5d60 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 10 client.15231 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 1 -- 172.21.15.32:0/2106360372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564e6faf76a0 con 0x564e6fae5d60 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c7effd640 1 -- 172.21.15.32:0/2106360372 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c900a09f0 con 0x564e6fae5d60 2024-04-16T05:34:21.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7c7effd640 1 -- 172.21.15.32:0/2106360372 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7c90053bf0 con 0x564e6fae5d60 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 10 client.15231 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 20 client.15231 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564e6fb66f30 0x564e6fb65c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 4 client.15231 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.965+0000 7f7ca867c740 1 -- 172.21.15.32:0/2106360372 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "status"}) -- 0x564e6fa594e0 con 0x564e6fb66f30 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564e6fb66f30 0x564e6fb65c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7c7effd640 10 client.15231 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7c96ffd640 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564e6fb66f30 0x564e6fb65c40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc88f88d3707e8 server_cookie=2309b9231a8d145b in_seq=0 out_seq=0 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7c7effd640 1 -- 172.21.15.32:0/2106360372 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x564e6fa594e0 con 0x564e6fb66f30 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7c7effd640 10 client.15231 handle_command_reply: tid=0 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 client.15231 shutdown 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 -- 172.21.15.32:0/2106360372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 msgr2=0x564e6faf2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6faf2850 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7c90016000 tx=0x7f7c9009a9f0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 -- 172.21.15.32:0/2106360372 shutdown_connections 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564e6fb66f30 0x564e6fb65c40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e6faeb8c0 0x564e6faf5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e6fae6750 0x564e6faf2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7c7effd640 0 client.15231 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 --2- 172.21.15.32:0/2106360372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e6fae5d60 0x564e6faf2850 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.966+0000 7f7ca867c740 1 -- 172.21.15.32:0/2106360372 >> 172.21.15.32:0/2106360372 conn(0x564e6fae12d0 msgr2=0x564e6fae6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca867c740 1 -- 172.21.15.32:0/2106360372 shutdown_connections 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca867c740 1 -- 172.21.15.32:0/2106360372 wait complete. 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca867c740 20 client.15231 trim_cache size 0 max 16384 2024-04-16T05:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24494, 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 107.074226143, 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 24, 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 91, 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 111.420461589 2024-04-16T05:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:34:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7c8c0a3c30 msgr2=0x7f7c8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7c8c0a3c30 0x7f7c8c0a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x564e6fae6fb0 tx=0x7f7c98009840 comp rx=0 tx=0).stop 2024-04-16T05:34:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca0102b80 msgr2=0x7f7ca01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca0102b80 0x7f7ca01983c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c890 tx=0x7f7c9000cd50 comp rx=0 tx=0).stop 2024-04-16T05:34:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2629210839 shutdown_connections 2024-04-16T05:34:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f7c8c0a3c30 0x7f7c8c0a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0107de0 0x7f7ca019f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca01054b0 0x7f7ca0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 --2- 172.21.15.32:0/2629210839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ca0102b80 0x7f7ca01983c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2629210839 >> 172.21.15.32:0/2629210839 conn(0x7f7ca00fc4b0 msgr2=0x7f7ca01035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2629210839 shutdown_connections 2024-04-16T05:34:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:21.967+0000 7f7ca5a88640 1 -- 172.21.15.32:0/2629210839 wait complete. 2024-04-16T05:34:21.977 INFO:tasks.cephadm:Setup complete, yielding 2024-04-16T05:34:21.977 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-16T05:34:21.987 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:34:21.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch status' 2024-04-16T05:34:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:22 smithi032 ceph-mon[32351]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:22.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:22 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3783611626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:22.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:22 smithi134 ceph-mon[36153]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:22.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:22 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3783611626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:22 smithi082 ceph-mon[36195]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:22 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3783611626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 -- 172.21.15.32:0/2665053389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 msgr2=0x7f9e44102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2665053389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 0x7f9e44102f70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9e340099c0 tx=0x7f9e34009cd0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 -- 172.21.15.32:0/2665053389 shutdown_connections 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2665053389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e441046f0 0x7f9e4410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2665053389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 0x7f9e44102f70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2665053389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e44103d50 0x7f9e441041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 -- 172.21.15.32:0/2665053389 >> 172.21.15.32:0/2665053389 conn(0x7f9e440fe0f0 msgr2=0x7f9e44100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 -- 172.21.15.32:0/2665053389 shutdown_connections 2024-04-16T05:34:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.152+0000 7f9e4a670700 1 -- 172.21.15.32:0/2665053389 wait complete. 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e4a670700 1 Processor -- start 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e4a670700 1 -- start start 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e4a670700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 0x7f9e44076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e4a670700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e44103d50 0x7f9e44074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e4a670700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e441046f0 0x7f9e44075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e4a670700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4410e780 con 0x7f9e44102b50 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e4a670700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4410e600 con 0x7f9e44103d50 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e4a670700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4410e900 con 0x7f9e441046f0 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.153+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 0x7f9e44076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 0x7f9e44076890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53830/0 (socket says 172.21.15.32:53830) 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e43fff700 1 -- 172.21.15.32:0/2327519471 learned_addr learned my addr 172.21.15.32:0/2327519471 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e48c0d700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e441046f0 0x7f9e44075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e437fe700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e44103d50 0x7f9e44074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e43fff700 1 -- 172.21.15.32:0/2327519471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e441046f0 msgr2=0x7f9e44075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e43fff700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e441046f0 0x7f9e44075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e43fff700 1 -- 172.21.15.32:0/2327519471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e44103d50 msgr2=0x7f9e44074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e43fff700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e44103d50 0x7f9e44074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e43fff700 1 -- 172.21.15.32:0/2327519471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e44075cd0 con 0x7f9e44102b50 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e437fe700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e44103d50 0x7f9e44074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e43fff700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 0x7f9e44076890 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9e340098e0 tx=0x7f9e34009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e417fa700 1 -- 172.21.15.32:0/2327519471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e34027070 con 0x7f9e44102b50 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e417fa700 1 -- 172.21.15.32:0/2327519471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e34004150 con 0x7f9e44102b50 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e441a2f20 con 0x7f9e44102b50 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e417fa700 1 -- 172.21.15.32:0/2327519471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e34022e60 con 0x7f9e44102b50 2024-04-16T05:34:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.154+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e441a32c0 con 0x7f9e44102b50 2024-04-16T05:34:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.155+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4402dfe0 con 0x7f9e44102b50 2024-04-16T05:34:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.157+0000 7f9e417fa700 1 -- 172.21.15.32:0/2327519471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9e3402a070 con 0x7f9e44102b50 2024-04-16T05:34:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.158+0000 7f9e417fa700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9e2c0a3cc0 0x7f9e2c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.158+0000 7f9e437fe700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9e2c0a3cc0 0x7f9e2c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.158+0000 7f9e417fa700 1 -- 172.21.15.32:0/2327519471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9e340d15e0 con 0x7f9e44102b50 2024-04-16T05:34:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.158+0000 7f9e437fe700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9e2c0a3cc0 0x7f9e2c0a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005fd0 tx=0x7f9e38005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.162+0000 7f9e417fa700 1 -- 172.21.15.32:0/2327519471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e34013070 con 0x7f9e44102b50 2024-04-16T05:34:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.385+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9e44062f30 con 0x7f9e2c0a3cc0 2024-04-16T05:34:24.385 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:24 smithi032 ceph-mon[32351]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.386+0000 7f9e417fa700 1 -- 172.21.15.32:0/2327519471 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9e44062f30 con 0x7f9e2c0a3cc0 2024-04-16T05:34:24.386 INFO:teuthology.orchestra.run.smithi032.stdout:Backend: cephadm 2024-04-16T05:34:24.386 INFO:teuthology.orchestra.run.smithi032.stdout:Available: Yes 2024-04-16T05:34:24.386 INFO:teuthology.orchestra.run.smithi032.stdout:Paused: No 2024-04-16T05:34:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9e2c0a3cc0 msgr2=0x7f9e2c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9e2c0a3cc0 0x7f9e2c0a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005fd0 tx=0x7f9e38005ee0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 msgr2=0x7f9e44076890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 0x7f9e44076890 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9e340098e0 tx=0x7f9e34009580 comp rx=0 tx=0).stop 2024-04-16T05:34:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 shutdown_connections 2024-04-16T05:34:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f9e2c0a3cc0 0x7f9e2c0a6180 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e441046f0 0x7f9e44075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e44102b50 0x7f9e44076890 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 --2- 172.21.15.32:0/2327519471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e44103d50 0x7f9e44074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 >> 172.21.15.32:0/2327519471 conn(0x7f9e440fe0f0 msgr2=0x7f9e441070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.388+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 shutdown_connections 2024-04-16T05:34:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:24.389+0000 7f9e4a670700 1 -- 172.21.15.32:0/2327519471 wait complete. 2024-04-16T05:34:24.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:24 smithi134 ceph-mon[36153]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:24.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:24 smithi082 ceph-mon[36195]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:25.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-16T05:34:25.399 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:25.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:25 smithi032 ceph-mon[32351]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:25.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:25 smithi134 ceph-mon[36153]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:25.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:25 smithi082 ceph-mon[36195]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:26.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:26 smithi032 ceph-mon[32351]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:26.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:26 smithi134 ceph-mon[36153]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:26.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:26 smithi082 ceph-mon[36195]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:27.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 -- 172.21.15.32:0/1957178920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8105f20 msgr2=0x7f3cd8108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:27.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/1957178920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8105f20 0x7f3cd8108310 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009990 tx=0x7f3cc0009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 -- 172.21.15.32:0/1957178920 shutdown_connections 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/1957178920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd8108850 0x7f3cd810acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/1957178920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cd81035f0 0x7f3cd81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/1957178920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8105f20 0x7f3cd8108310 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 -- 172.21.15.32:0/1957178920 >> 172.21.15.32:0/1957178920 conn(0x7f3cd80fd020 msgr2=0x7f3cd80ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 -- 172.21.15.32:0/1957178920 shutdown_connections 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.142+0000 7f3cde5f4700 1 -- 172.21.15.32:0/1957178920 wait complete. 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cde5f4700 1 Processor -- start 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cde5f4700 1 -- start start 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cde5f4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cd81035f0 0x7f3cd8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cde5f4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd8105f20 0x7f3cd8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cde5f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8108850 0x7f3cd81a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cde5f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd810e740 con 0x7f3cd8108850 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cde5f4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd810e5c0 con 0x7f3cd8105f20 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cde5f4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd810e8c0 con 0x7f3cd81035f0 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8108850 0x7f3cd81a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8108850 0x7f3cd81a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53860/0 (socket says 172.21.15.32:53860) 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 -- 172.21.15.32:0/4141887187 learned_addr learned my addr 172.21.15.32:0/4141887187 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cd7fff700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cd81035f0 0x7f3cd8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3ccffff700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd8105f20 0x7f3cd8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 -- 172.21.15.32:0/4141887187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cd81035f0 msgr2=0x7f3cd8198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cd81035f0 0x7f3cd8198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 -- 172.21.15.32:0/4141887187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd8105f20 msgr2=0x7f3cd8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd8105f20 0x7f3cd8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 -- 172.21.15.32:0/4141887187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd81a0a70 con 0x7f3cd8108850 2024-04-16T05:34:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.143+0000 7f3cdcb91700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8108850 0x7f3cd81a03d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0007ef0 tx=0x7f3cd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.144+0000 7f3cd5ffb700 1 -- 172.21.15.32:0/4141887187 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0004bb0 con 0x7f3cd8108850 2024-04-16T05:34:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.144+0000 7f3cd5ffb700 1 -- 172.21.15.32:0/4141887187 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0004d50 con 0x7f3cd8108850 2024-04-16T05:34:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.144+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd81a0d60 con 0x7f3cd8108850 2024-04-16T05:34:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.144+0000 7f3cd5ffb700 1 -- 172.21.15.32:0/4141887187 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00044c0 con 0x7f3cd8108850 2024-04-16T05:34:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.144+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd81a1240 con 0x7f3cd8108850 2024-04-16T05:34:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.145+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd80697e0 con 0x7f3cd8108850 2024-04-16T05:34:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.146+0000 7f3cd5ffb700 1 -- 172.21.15.32:0/4141887187 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3cd001a030 con 0x7f3cd8108850 2024-04-16T05:34:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.146+0000 7f3cd5ffb700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3cc40a3cc0 0x7f3cc40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.147+0000 7f3cd5ffb700 1 -- 172.21.15.32:0/4141887187 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3cd00d0240 con 0x7f3cd8108850 2024-04-16T05:34:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.147+0000 7f3cd7fff700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3cc40a3cc0 0x7f3cc40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.147+0000 7f3cd7fff700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3cc40a3cc0 0x7f3cc40a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005950 tx=0x7f3cc800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.150+0000 7f3cd5ffb700 1 -- 172.21.15.32:0/4141887187 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cd0021020 con 0x7f3cd8108850 2024-04-16T05:34:27.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.369+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3cd8062ff0 con 0x7f3cc40a3cc0 2024-04-16T05:34:27.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.383+0000 7f3cd5ffb700 1 -- 172.21.15.32:0/4141887187 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f3cd8062ff0 con 0x7f3cc40a3cc0 2024-04-16T05:34:27.383 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-16T05:34:27.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.a smithi032 running (2m) 20s ago 119s 15.3M - 19.0.0-3101-ga87fcff6 3e786b67158f de09e00922c0 2024-04-16T05:34:27.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.b smithi082 running (117s) 50s ago 116s 25.3M - 19.0.0-3101-ga87fcff6 3e786b67158f e7a0a4ffdc18 2024-04-16T05:34:27.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.c smithi134 running (2m) 50s ago 2m 18.7M - 19.0.0-3101-ga87fcff6 3e786b67158f 428061409e05 2024-04-16T05:34:27.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.d smithi032 running (2m) 20s ago 2m 13.7M - 19.0.0-3101-ga87fcff6 3e786b67158f d1d1f5c2e005 2024-04-16T05:34:27.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.e smithi082 running (2m) 50s ago 2m 16.2M - 19.0.0-3101-ga87fcff6 3e786b67158f 263d0625fb42 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mds.f smithi134 running (2m) 50s ago 2m 16.3M - 19.0.0-3101-ga87fcff6 3e786b67158f 3ad03c21da9a 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mds.g smithi032 running (2m) 20s ago 2m 13.7M - 19.0.0-3101-ga87fcff6 3e786b67158f 694d2cbe0a4a 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mds.h smithi082 running (2m) 50s ago 2m 20.3M - 19.0.0-3101-ga87fcff6 3e786b67158f d9882b97cc8a 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mds.i smithi134 running (2m) 50s ago 2m 19.5M - 19.0.0-3101-ga87fcff6 3e786b67158f b310c02cc59f 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mds.j smithi032 running (2m) 20s ago 2m 12.9M - 19.0.0-3101-ga87fcff6 3e786b67158f ad68f6820de1 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mds.k smithi082 running (2m) 50s ago 2m 13.7M - 19.0.0-3101-ga87fcff6 3e786b67158f c7b28af03610 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mds.l smithi134 running (2m) 50s ago 2m 13.1M - 19.0.0-3101-ga87fcff6 3e786b67158f d50418adbc41 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.x smithi032 *:9283,8765 running (9m) 20s ago 9m 517M - 19.0.0-3101-ga87fcff6 3e786b67158f 9e101b4f648f 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.y smithi082 *:8443,8765 running (7m) 50s ago 7m 455M - 19.0.0-3101-ga87fcff6 3e786b67158f d58fb8e1390f 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.z smithi134 *:8443,8765 running (7m) 50s ago 7m 454M - 19.0.0-3101-ga87fcff6 3e786b67158f cea3fb9e5da5 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a smithi032 running (9m) 20s ago 9m 52.8M 2048M 19.0.0-3101-ga87fcff6 3e786b67158f 38b0b9542547 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b smithi082 running (8m) 50s ago 7m 38.5M 2048M 19.0.0-3101-ga87fcff6 3e786b67158f 2e947f49936b 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c smithi134 running (8m) 50s ago 8m 39.1M 2048M 19.0.0-3101-ga87fcff6 3e786b67158f 758f8f2d8501 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (7m) 20s ago 7m 51.1M 4273M 19.0.0-3101-ga87fcff6 3e786b67158f 71aa7cdf3208 2024-04-16T05:34:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi082 running (7m) 50s ago 7m 57.2M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 80dfc408f9f0 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi134 running (6m) 50s ago 6m 59.5M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 5b4986669f3e 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi032 running (6m) 20s ago 6m 44.5M 4273M 19.0.0-3101-ga87fcff6 3e786b67158f deab1c5a359b 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi082 running (5m) 50s ago 5m 42.6M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f f2c135dfe30b 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi134 running (5m) 50s ago 5m 57.6M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f c25ad392f21c 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 smithi032 running (4m) 20s ago 4m 53.5M 4273M 19.0.0-3101-ga87fcff6 3e786b67158f a8610e884ed8 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 smithi082 running (4m) 50s ago 4m 44.3M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 702efa35451f 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 smithi134 running (4m) 50s ago 4m 53.5M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 2fe94b80bd48 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 smithi032 running (3m) 20s ago 3m 45.5M 4273M 19.0.0-3101-ga87fcff6 3e786b67158f c149ca5a2661 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 smithi082 running (3m) 50s ago 3m 42.6M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 78dfe623d3aa 2024-04-16T05:34:27.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 smithi134 running (2m) 50s ago 2m 42.2M 4270M 19.0.0-3101-ga87fcff6 3e786b67158f 778bc6433199 2024-04-16T05:34:27.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3cc40a3cc0 msgr2=0x7f3cc40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:27.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3cc40a3cc0 0x7f3cc40a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005950 tx=0x7f3cc800a300 comp rx=0 tx=0).stop 2024-04-16T05:34:27.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8108850 msgr2=0x7f3cd81a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:27.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8108850 0x7f3cd81a03d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0007ef0 tx=0x7f3cd000d3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 shutdown_connections 2024-04-16T05:34:27.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3cc40a3cc0 0x7f3cc40a6180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd8108850 0x7f3cd81a03d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cd81035f0 0x7f3cd8198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 --2- 172.21.15.32:0/4141887187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd8105f20 0x7f3cd8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:27.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.385+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 >> 172.21.15.32:0/4141887187 conn(0x7f3cd80fd020 msgr2=0x7f3cd8103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:27.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.386+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 shutdown_connections 2024-04-16T05:34:27.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:27.386+0000 7f3cde5f4700 1 -- 172.21.15.32:0/4141887187 wait complete. 2024-04-16T05:34:28.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-16T05:34:28.498 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:28.954 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:28 smithi032 ceph-mon[32351]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:28.954 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:28 smithi032 ceph-mon[32351]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:29.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:28 smithi134 ceph-mon[36153]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:29.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:28 smithi134 ceph-mon[36153]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:29.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:28 smithi082 ceph-mon[36195]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:29.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:28 smithi082 ceph-mon[36195]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 -- 172.21.15.32:0/4049681438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99701088d0 msgr2=0x7f997010acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 --2- 172.21.15.32:0/4049681438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99701088d0 0x7f997010acc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f996800b0a0 tx=0x7f996800b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 -- 172.21.15.32:0/4049681438 shutdown_connections 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 --2- 172.21.15.32:0/4049681438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99701088d0 0x7f997010acc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 --2- 172.21.15.32:0/4049681438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9970103670 0x7f9970105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 --2- 172.21.15.32:0/4049681438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9970105fa0 0x7f9970108390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 -- 172.21.15.32:0/4049681438 >> 172.21.15.32:0/4049681438 conn(0x7f99700fd020 msgr2=0x7f99700ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 -- 172.21.15.32:0/4049681438 shutdown_connections 2024-04-16T05:34:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 -- 172.21.15.32:0/4049681438 wait complete. 2024-04-16T05:34:30.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.290+0000 7f9975a37700 1 Processor -- start 2024-04-16T05:34:30.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f9975a37700 1 -- start start 2024-04-16T05:34:30.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f9975a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9970103670 0x7f9970102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f9975a37700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9970105fa0 0x7f9970101620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f9975a37700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99701088d0 0x7f9970101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f9975a37700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f997010e790 con 0x7f9970103670 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f9975a37700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f997010e610 con 0x7f99701088d0 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f9975a37700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f997010e910 con 0x7f9970105fa0 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9970103670 0x7f9970102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9970103670 0x7f9970102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38796/0 (socket says 172.21.15.32:38796) 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996effd700 1 -- 172.21.15.32:0/3607711575 learned_addr learned my addr 172.21.15.32:0/3607711575 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996e7fc700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9970105fa0 0x7f9970101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996f7fe700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99701088d0 0x7f9970101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996effd700 1 -- 172.21.15.32:0/3607711575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9970105fa0 msgr2=0x7f9970101620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996effd700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9970105fa0 0x7f9970101620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996effd700 1 -- 172.21.15.32:0/3607711575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99701088d0 msgr2=0x7f9970101b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996effd700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99701088d0 0x7f9970101b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996effd700 1 -- 172.21.15.32:0/3607711575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9970102410 con 0x7f9970103670 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.291+0000 7f996e7fc700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9970105fa0 0x7f9970101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.292+0000 7f996effd700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9970103670 0x7f9970102fd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f996000c9b0 tx=0x7f996000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.292+0000 7f9974a35700 1 -- 172.21.15.32:0/3607711575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960007b00 con 0x7f9970103670 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.292+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99701a2ed0 con 0x7f9970103670 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.292+0000 7f9974a35700 1 -- 172.21.15.32:0/3607711575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9960007ca0 con 0x7f9970103670 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.292+0000 7f9974a35700 1 -- 172.21.15.32:0/3607711575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960018730 con 0x7f9970103670 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.292+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99701a32d0 con 0x7f9970103670 2024-04-16T05:34:30.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.293+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997002dfe0 con 0x7f9970103670 2024-04-16T05:34:30.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.294+0000 7f9974a35700 1 -- 172.21.15.32:0/3607711575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f99600188d0 con 0x7f9970103670 2024-04-16T05:34:30.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.295+0000 7f9974a35700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f995c0a3c40 0x7f995c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:30.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.295+0000 7f996e7fc700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f995c0a3c40 0x7f995c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:30.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.295+0000 7f9974a35700 1 -- 172.21.15.32:0/3607711575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f99600d0990 con 0x7f9970103670 2024-04-16T05:34:30.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.295+0000 7f996e7fc700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f995c0a3c40 0x7f995c0a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f99580099d0 tx=0x7f9958008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:30.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.298+0000 7f9974a35700 1 -- 172.21.15.32:0/3607711575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9960014030 con 0x7f9970103670 2024-04-16T05:34:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.517+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9970062f30 con 0x7f995c0a3c40 2024-04-16T05:34:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.524+0000 7f9974a35700 1 -- 172.21.15.32:0/3607711575 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9970062f30 con 0x7f995c0a3c40 2024-04-16T05:34:30.522 INFO:teuthology.orchestra.run.smithi032.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-16T05:34:30.523 INFO:teuthology.orchestra.run.smithi032.stdout:mds.all 12/12 53s ago 2m smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-16T05:34:30.523 INFO:teuthology.orchestra.run.smithi032.stdout:mgr 3/3 53s ago 7m smithi032=x;smithi082=y;smithi134=z;count:3 2024-04-16T05:34:30.523 INFO:teuthology.orchestra.run.smithi032.stdout:mon 3/3 53s ago 8m smithi032:172.21.15.32=a;smithi082:172.21.15.82=b;smithi134:172.21.15.134=c;count:3 2024-04-16T05:34:30.523 INFO:teuthology.orchestra.run.smithi032.stdout:osd 12 53s ago - 2024-04-16T05:34:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f995c0a3c40 msgr2=0x7f995c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f995c0a3c40 0x7f995c0a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f99580099d0 tx=0x7f9958008040 comp rx=0 tx=0).stop 2024-04-16T05:34:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9970103670 msgr2=0x7f9970102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9970103670 0x7f9970102fd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f996000c9b0 tx=0x7f996000ccc0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 shutdown_connections 2024-04-16T05:34:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f995c0a3c40 0x7f995c0a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99701088d0 0x7f9970101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9970103670 0x7f9970102fd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 --2- 172.21.15.32:0/3607711575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9970105fa0 0x7f9970101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.526+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 >> 172.21.15.32:0/3607711575 conn(0x7f99700fd020 msgr2=0x7f9970103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.527+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 shutdown_connections 2024-04-16T05:34:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:30.527+0000 7f9975a37700 1 -- 172.21.15.32:0/3607711575 wait complete. 2024-04-16T05:34:30.905 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:30 smithi032 ceph-mon[32351]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:31.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:30 smithi134 ceph-mon[36153]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:31.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:30 smithi082 ceph-mon[36195]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:31.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-16T05:34:31.633 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:31.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:31 smithi032 ceph-mon[32351]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:32.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:31 smithi134 ceph-mon[36153]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:31 smithi082 ceph-mon[36195]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:32.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:32 smithi032 ceph-mon[32351]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:32 smithi134 ceph-mon[36153]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:32 smithi082 ceph-mon[36195]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 -- 172.21.15.32:0/3303735312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 msgr2=0x7f28c4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 --2- 172.21.15.32:0/3303735312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 0x7f28c4101e70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f28b40099c0 tx=0x7f28b4009cd0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 -- 172.21.15.32:0/3303735312 shutdown_connections 2024-04-16T05:34:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 --2- 172.21.15.32:0/3303735312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c41035f0 0x7f28c4109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 --2- 172.21.15.32:0/3303735312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 0x7f28c4101e70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 --2- 172.21.15.32:0/3303735312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28c4102c50 0x7f28c41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 -- 172.21.15.32:0/3303735312 >> 172.21.15.32:0/3303735312 conn(0x7f28c40fcff0 msgr2=0x7f28c40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 -- 172.21.15.32:0/3303735312 shutdown_connections 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.361+0000 7f28c967f700 1 -- 172.21.15.32:0/3303735312 wait complete. 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c967f700 1 Processor -- start 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c967f700 1 -- start start 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c967f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 0x7f28c4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c967f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28c4102c50 0x7f28c41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c967f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c41035f0 0x7f28c41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c967f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f28c410d680 con 0x7f28c4101a50 2024-04-16T05:34:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c967f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f28c410d500 con 0x7f28c41035f0 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c967f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28c410d800 con 0x7f28c4102c50 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 0x7f28c4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 0x7f28c4198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38826/0 (socket says 172.21.15.32:38826) 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28c2ffd700 1 -- 172.21.15.32:0/676060936 learned_addr learned my addr 172.21.15.32:0/676060936 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.362+0000 7f28ba7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28c4102c50 0x7f28c41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c37fe700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c41035f0 0x7f28c41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c2ffd700 1 -- 172.21.15.32:0/676060936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28c4102c50 msgr2=0x7f28c41993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c2ffd700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28c4102c50 0x7f28c41993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c2ffd700 1 -- 172.21.15.32:0/676060936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c41035f0 msgr2=0x7f28c41a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c2ffd700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c41035f0 0x7f28c41a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c2ffd700 1 -- 172.21.15.32:0/676060936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c41a0aa0 con 0x7f28c4101a50 2024-04-16T05:34:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c2ffd700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 0x7f28c4198e90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005b10 tx=0x7f28b4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c0ff9700 1 -- 172.21.15.32:0/676060936 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4027070 con 0x7f28c4101a50 2024-04-16T05:34:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c0ff9700 1 -- 172.21.15.32:0/676060936 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b4004150 con 0x7f28c4101a50 2024-04-16T05:34:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c41a0d30 con 0x7f28c4101a50 2024-04-16T05:34:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c0ff9700 1 -- 172.21.15.32:0/676060936 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4022e00 con 0x7f28c4101a50 2024-04-16T05:34:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.363+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c41a1180 con 0x7f28c4101a50 2024-04-16T05:34:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.364+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c40697e0 con 0x7f28c4101a50 2024-04-16T05:34:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.366+0000 7f28c0ff9700 1 -- 172.21.15.32:0/676060936 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f28b402a070 con 0x7f28c4101a50 2024-04-16T05:34:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.366+0000 7f28c0ff9700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28b00a3c40 0x7f28b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.366+0000 7f28ba7fc700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28b00a3c40 0x7f28b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.366+0000 7f28c0ff9700 1 -- 172.21.15.32:0/676060936 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f28b40d1d90 con 0x7f28c4101a50 2024-04-16T05:34:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.367+0000 7f28ba7fc700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28b00a3c40 0x7f28b00a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005950 tx=0x7f28ac00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:33.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.370+0000 7f28c0ff9700 1 -- 172.21.15.32:0/676060936 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28b4031070 con 0x7f28c4101a50 2024-04-16T05:34:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.584+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f28c406aee0 con 0x7f28b00a3c40 2024-04-16T05:34:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.585+0000 7f28c0ff9700 1 -- 172.21.15.32:0/676060936 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f28c406aee0 con 0x7f28b00a3c40 2024-04-16T05:34:33.584 INFO:teuthology.orchestra.run.smithi032.stdout:HOST ADDR LABELS STATUS 2024-04-16T05:34:33.584 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 172.21.15.32 2024-04-16T05:34:33.584 INFO:teuthology.orchestra.run.smithi032.stdout:smithi082 172.21.15.82 2024-04-16T05:34:33.584 INFO:teuthology.orchestra.run.smithi032.stdout:smithi134 172.21.15.134 2024-04-16T05:34:33.584 INFO:teuthology.orchestra.run.smithi032.stdout:3 hosts in cluster 2024-04-16T05:34:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28b00a3c40 msgr2=0x7f28b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28b00a3c40 0x7f28b00a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005950 tx=0x7f28ac00a300 comp rx=0 tx=0).stop 2024-04-16T05:34:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 msgr2=0x7f28c4198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 0x7f28c4198e90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005b10 tx=0x7f28b4009580 comp rx=0 tx=0).stop 2024-04-16T05:34:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 shutdown_connections 2024-04-16T05:34:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f28b00a3c40 0x7f28b00a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c41035f0 0x7f28c41a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c4101a50 0x7f28c4198e90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 --2- 172.21.15.32:0/676060936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28c4102c50 0x7f28c41993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:33.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 >> 172.21.15.32:0/676060936 conn(0x7f28c40fcff0 msgr2=0x7f28c406a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:33.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 shutdown_connections 2024-04-16T05:34:33.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:33.587+0000 7f28c967f700 1 -- 172.21.15.32:0/676060936 wait complete. 2024-04-16T05:34:34.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-16T05:34:34.617 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:34.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:34 smithi134 ceph-mon[36153]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:34.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:34 smithi134 ceph-mon[36153]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:34 smithi032 ceph-mon[32351]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:34 smithi032 ceph-mon[32351]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:34 smithi082 ceph-mon[36195]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:35.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:34 smithi082 ceph-mon[36195]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 -- 172.21.15.32:0/713535404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec1088e0 msgr2=0x7fceec10acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/713535404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec1088e0 0x7fceec10acd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcee400b0a0 tx=0x7fcee400b3b0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 -- 172.21.15.32:0/713535404 shutdown_connections 2024-04-16T05:34:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/713535404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec1088e0 0x7fceec10acd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/713535404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceec103680 0x7fceec105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/713535404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec105fb0 0x7fceec1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 -- 172.21.15.32:0/713535404 >> 172.21.15.32:0/713535404 conn(0x7fceec0fd010 msgr2=0x7fceec0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 -- 172.21.15.32:0/713535404 shutdown_connections 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 -- 172.21.15.32:0/713535404 wait complete. 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 Processor -- start 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 -- start start 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceec103680 0x7fceec198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.384+0000 7fcef3dcd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec105fb0 0x7fceec199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef3dcd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1088e0 0x7fceec1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef3dcd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fceec10e780 con 0x7fceec105fb0 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef3dcd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceec10e600 con 0x7fceec1088e0 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef3dcd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fceec10e900 con 0x7fceec103680 2024-04-16T05:34:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec105fb0 0x7fceec199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec105fb0 0x7fceec199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38844/0 (socket says 172.21.15.32:38844) 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 -- 172.21.15.32:0/699591252 learned_addr learned my addr 172.21.15.32:0/699591252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1b69700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceec103680 0x7fceec198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef236a700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1088e0 0x7fceec1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 -- 172.21.15.32:0/699591252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceec103680 msgr2=0x7fceec198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceec103680 0x7fceec198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 -- 172.21.15.32:0/699591252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1088e0 msgr2=0x7fceec1a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1088e0 0x7fceec1a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 -- 172.21.15.32:0/699591252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec1a0950 con 0x7fceec105fb0 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef236a700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1088e0 0x7fceec1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef1368700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec105fb0 0x7fceec199240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c9b0 tx=0x7fcedc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcee2ffd700 1 -- 172.21.15.32:0/699591252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc007b00 con 0x7fceec105fb0 2024-04-16T05:34:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcee2ffd700 1 -- 172.21.15.32:0/699591252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcedc007ca0 con 0x7fceec105fb0 2024-04-16T05:34:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec1a0c40 con 0x7fceec105fb0 2024-04-16T05:34:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcee2ffd700 1 -- 172.21.15.32:0/699591252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc018730 con 0x7fceec105fb0 2024-04-16T05:34:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.385+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec1a1180 con 0x7fceec105fb0 2024-04-16T05:34:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.386+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceec0697e0 con 0x7fceec105fb0 2024-04-16T05:34:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.388+0000 7fcee2ffd700 1 -- 172.21.15.32:0/699591252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcedc0188d0 con 0x7fceec105fb0 2024-04-16T05:34:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.388+0000 7fcee2ffd700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fced80a3c40 0x7fced80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.388+0000 7fcef1b69700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fced80a3c40 0x7fced80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.388+0000 7fcee2ffd700 1 -- 172.21.15.32:0/699591252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcedc0d09d0 con 0x7fceec105fb0 2024-04-16T05:34:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.389+0000 7fcef1b69700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fced80a3c40 0x7fced80a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fceec19a1e0 tx=0x7fcee8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.392+0000 7fcee2ffd700 1 -- 172.21.15.32:0/699591252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcedc014030 con 0x7fceec105fb0 2024-04-16T05:34:36.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.608+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 --> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fceec062f30 con 0x7fced80a3c40 2024-04-16T05:34:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.610+0000 7fcee2ffd700 1 -- 172.21.15.32:0/699591252 <== mgr.14152 v2:172.21.15.32:6800/2875882657 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fceec062f30 con 0x7fced80a3c40 2024-04-16T05:34:36.609 INFO:teuthology.orchestra.run.smithi032.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-16T05:34:36.609 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402DN480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-16T05:34:36.609 INFO:teuthology.orchestra.run.smithi032.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-16T05:34:36.609 INFO:teuthology.orchestra.run.smithi032.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-16T05:34:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fced80a3c40 msgr2=0x7fced80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fced80a3c40 0x7fced80a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fceec19a1e0 tx=0x7fcee8009190 comp rx=0 tx=0).stop 2024-04-16T05:34:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec105fb0 msgr2=0x7fceec199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec105fb0 0x7fceec199240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c9b0 tx=0x7fcedc00ccc0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 shutdown_connections 2024-04-16T05:34:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fced80a3c40 0x7fced80a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1088e0 0x7fceec1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceec103680 0x7fceec198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 --2- 172.21.15.32:0/699591252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec105fb0 0x7fceec199240 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 >> 172.21.15.32:0/699591252 conn(0x7fceec0fd010 msgr2=0x7fceec103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 shutdown_connections 2024-04-16T05:34:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:36.612+0000 7fcef3dcd700 1 -- 172.21.15.32:0/699591252 wait complete. 2024-04-16T05:34:36.981 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:36 smithi032 ceph-mon[32351]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:37.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:36 smithi134 ceph-mon[36153]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:36 smithi082 ceph-mon[36195]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:37.462 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-16T05:34:37.470 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:34:37.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-16T05:34:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:37.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:37 smithi032 ceph-mon[32351]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:37 smithi134 ceph-mon[36153]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:38.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:37 smithi082 ceph-mon[36195]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:34:38.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:38 smithi032 ceph-mon[32351]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:38 smithi134 ceph-mon[36153]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:39.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:38 smithi082 ceph-mon[36195]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 -- 172.21.15.32:0/608146856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8105f70 msgr2=0x7f3df8108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/608146856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8105f70 0x7f3df8108360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009990 tx=0x7f3de0009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 -- 172.21.15.32:0/608146856 shutdown_connections 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/608146856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3df81088a0 0x7f3df810acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/608146856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3df8103640 0x7f3df8105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/608146856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8105f70 0x7f3df8108360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 -- 172.21.15.32:0/608146856 >> 172.21.15.32:0/608146856 conn(0x7f3df80fcff0 msgr2=0x7f3df80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 -- 172.21.15.32:0/608146856 shutdown_connections 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.451+0000 7f3dfcf85700 1 -- 172.21.15.32:0/608146856 wait complete. 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3dfcf85700 1 Processor -- start 2024-04-16T05:34:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3dfcf85700 1 -- start start 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3dfcf85700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8103640 0x7f3df819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3dfcf85700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3df8105f70 0x7f3df819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3dfcf85700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3df81088a0 0x7f3df81a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3dfcf85700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3df810e7b0 con 0x7f3df8103640 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3dfcf85700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3df810e630 con 0x7f3df8105f70 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3dfcf85700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3df810e930 con 0x7f3df81088a0 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3df659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8103640 0x7f3df819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3df659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8103640 0x7f3df819aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59704/0 (socket says 172.21.15.32:59704) 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3df659c700 1 -- 172.21.15.32:0/1151361648 learned_addr learned my addr 172.21.15.32:0/1151361648 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.452+0000 7f3df6d9d700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3df81088a0 0x7f3df81a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3df5d9b700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3df8105f70 0x7f3df819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3df659c700 1 -- 172.21.15.32:0/1151361648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3df81088a0 msgr2=0x7f3df81a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3df659c700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3df81088a0 0x7f3df81a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3df659c700 1 -- 172.21.15.32:0/1151361648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3df8105f70 msgr2=0x7f3df819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3df659c700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3df8105f70 0x7f3df819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3df659c700 1 -- 172.21.15.32:0/1151361648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df81a2c50 con 0x7f3df8103640 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3df659c700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8103640 0x7f3df819aff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ba80 tx=0x7f3de800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3def7fe700 1 -- 172.21.15.32:0/1151361648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8019070 con 0x7f3df8103640 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3def7fe700 1 -- 172.21.15.32:0/1151361648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de800ce10 con 0x7f3df8103640 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df81a2f40 con 0x7f3df8103640 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3def7fe700 1 -- 172.21.15.32:0/1151361648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8007370 con 0x7f3df8103640 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.453+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df81a3410 con 0x7f3df8103640 2024-04-16T05:34:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.454+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df80697e0 con 0x7f3df8103640 2024-04-16T05:34:39.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.456+0000 7f3def7fe700 1 -- 172.21.15.32:0/1151361648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3de801c070 con 0x7f3df8103640 2024-04-16T05:34:39.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.457+0000 7f3def7fe700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3de40a3c40 0x7f3de40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:39.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.457+0000 7f3def7fe700 1 -- 172.21.15.32:0/1151361648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3de80d0540 con 0x7f3df8103640 2024-04-16T05:34:39.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.457+0000 7f3df5d9b700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3de40a3c40 0x7f3de40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:39.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.458+0000 7f3df5d9b700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3de40a3c40 0x7f3de40a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3de000b5c0 tx=0x7f3de0003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.461+0000 7f3def7fe700 1 -- 172.21.15.32:0/1151361648 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3de8023030 con 0x7f3df8103640 2024-04-16T05:34:39.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.713+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3df8063700 con 0x7f3df8103640 2024-04-16T05:34:39.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.714+0000 7f3def7fe700 1 -- 172.21.15.32:0/1151361648 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3054 (secure 0 0 0) 0x7f3de809cbc0 con 0x7f3df8103640 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:e24 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 24 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-04-16T05:32:34.109354+0000 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-04-16T05:32:55.688634+0000 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-04-16T05:34:39.713 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 300 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 0 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 3 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1,2 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=24494,1=24461,2=24460} 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3,4] 2024-04-16T05:34:39.714 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data disabled 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 0 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 24494 members: 24460,24461,24494 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b{0:24494} state up:active seq 3 addr [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c{0:24442} state up:standby-replay seq 1 addr [v2:172.21.15.134:6836/2091760007,v1:172.21.15.134:6837/2091760007] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.h{1:24461} state up:active seq 10 addr [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.e{1:24434} state up:standby-replay seq 1 addr [v2:172.21.15.82:6834/1367430466,v1:172.21.15.82:6835/1367430466] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.i{2:24460} state up:active seq 10 addr [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.f{2:24430} state up:standby-replay seq 1 addr [v2:172.21.15.134:6834/1431936644,v1:172.21.15.134:6835/1431936644] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.32:6834/3476798351,v1:172.21.15.32:6835/3476798351] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.716 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.32:6836/3911772856,v1:172.21.15.32:6837/3911772856] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.716 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.g{-1:14592} state up:standby seq 1 addr [v2:172.21.15.32:6838/1699429500,v1:172.21.15.32:6839/1699429500] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.716 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a{-1:14604} state up:standby seq 1 addr [v2:172.21.15.32:6840/2946647099,v1:172.21.15.32:6841/2946647099] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.716 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.l{-1:24391} state up:standby seq 1 addr [v2:172.21.15.134:6832/1600676004,v1:172.21.15.134:6833/1600676004] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.716 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.k{-1:24401} state up:standby seq 1 addr [v2:172.21.15.82:6832/1596016884,v1:172.21.15.82:6833/1596016884] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T05:34:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3de40a3c40 msgr2=0x7f3de40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3de40a3c40 0x7f3de40a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3de000b5c0 tx=0x7f3de0003960 comp rx=0 tx=0).stop 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8103640 msgr2=0x7f3df819aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8103640 0x7f3df819aff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ba80 tx=0x7f3de800bd90 comp rx=0 tx=0).stop 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 shutdown_connections 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f3de40a3c40 0x7f3de40a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3df81088a0 0x7f3df81a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8103640 0x7f3df819aff0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 --2- 172.21.15.32:0/1151361648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3df8105f70 0x7f3df819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 >> 172.21.15.32:0/1151361648 conn(0x7f3df80fcff0 msgr2=0x7f3df8102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 shutdown_connections 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:39.717+0000 7f3dfcf85700 1 -- 172.21.15.32:0/1151361648 wait complete. 2024-04-16T05:34:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-16T05:34:40.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-16T05:34:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:40.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:40 smithi032 ceph-mon[32351]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:40.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:40 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1151361648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T05:34:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:40 smithi134 ceph-mon[36153]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:40 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1151361648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T05:34:41.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:40 smithi082 ceph-mon[36195]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:41.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:40 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1151361648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 -- 172.21.15.32:0/1820519338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028102b50 msgr2=0x7f8028102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 --2- 172.21.15.32:0/1820519338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028102b50 0x7f8028102f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f80180099c0 tx=0x7f8018009cd0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 -- 172.21.15.32:0/1820519338 shutdown_connections 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 --2- 172.21.15.32:0/1820519338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80281046f0 0x7f802810afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 --2- 172.21.15.32:0/1820519338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028102b50 0x7f8028102f70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 --2- 172.21.15.32:0/1820519338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8028103d50 0x7f80281041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 -- 172.21.15.32:0/1820519338 >> 172.21.15.32:0/1820519338 conn(0x7f80280fe0f0 msgr2=0x7f8028100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 -- 172.21.15.32:0/1820519338 shutdown_connections 2024-04-16T05:34:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.367+0000 7f802f0f6700 1 -- 172.21.15.32:0/1820519338 wait complete. 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802f0f6700 1 Processor -- start 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802f0f6700 1 -- start start 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802f0f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8028102b50 0x7f8028198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802f0f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028103d50 0x7f80281992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802f0f6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80281046f0 0x7f80281a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802f0f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f802810e730 con 0x7f8028103d50 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802f0f6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f802810e5b0 con 0x7f8028102b50 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802f0f6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f802810e8b0 con 0x7f80281046f0 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f801ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028103d50 0x7f80281992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f801ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028103d50 0x7f80281992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59724/0 (socket says 172.21.15.32:59724) 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f801ffff700 1 -- 172.21.15.32:0/3045446779 learned_addr learned my addr 172.21.15.32:0/3045446779 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802ce92700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8028102b50 0x7f8028198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f802d693700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80281046f0 0x7f80281a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f801ffff700 1 -- 172.21.15.32:0/3045446779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80281046f0 msgr2=0x7f80281a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f801ffff700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80281046f0 0x7f80281a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f801ffff700 1 -- 172.21.15.32:0/3045446779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8028102b50 msgr2=0x7f8028198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f801ffff700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8028102b50 0x7f8028198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.368+0000 7f801ffff700 1 -- 172.21.15.32:0/3045446779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80281a0a00 con 0x7f8028103d50 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.369+0000 7f801ffff700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028103d50 0x7f80281992e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f802400d980 tx=0x7f802400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.369+0000 7f801dffb700 1 -- 172.21.15.32:0/3045446779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024016070 con 0x7f8028103d50 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.369+0000 7f801dffb700 1 -- 172.21.15.32:0/3045446779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8024009db0 con 0x7f8028103d50 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.369+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80281a0cf0 con 0x7f8028103d50 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.369+0000 7f801dffb700 1 -- 172.21.15.32:0/3045446779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802400f3c0 con 0x7f8028103d50 2024-04-16T05:34:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.369+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a11d0 con 0x7f8028103d50 2024-04-16T05:34:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.370+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80280697e0 con 0x7f8028103d50 2024-04-16T05:34:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.371+0000 7f801dffb700 1 -- 172.21.15.32:0/3045446779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8024019070 con 0x7f8028103d50 2024-04-16T05:34:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.372+0000 7f801dffb700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f80100a3c40 0x7f80100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.372+0000 7f801dffb700 1 -- 172.21.15.32:0/3045446779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f80240d01b0 con 0x7f8028103d50 2024-04-16T05:34:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.372+0000 7f802ce92700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f80100a3c40 0x7f80100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.372+0000 7f802ce92700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f80100a3c40 0x7f80100a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f801800b580 tx=0x7f8018005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.375+0000 7f801dffb700 1 -- 172.21.15.32:0/3045446779 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f802409c830 con 0x7f8028103d50 2024-04-16T05:34:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.586+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f802802e1f0 con 0x7f8028103d50 2024-04-16T05:34:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.587+0000 7f801dffb700 1 -- 172.21.15.32:0/3045446779 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5589 (secure 0 0 0) 0x7f802409c130 con 0x7f8028103d50 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 91 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-04-16T05:24:37.692531+0000 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-04-16T05:33:34.757328+0000 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:crush_version 27 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:full_ratio 0.95 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:backfillfull_ratio 0.9 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:nearfull_ratio 0.85 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:require_min_compat_client luminous 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:min_compat_client luminous 2024-04-16T05:34:42.586 INFO:teuthology.orchestra.run.smithi032.stdout:require_osd_release squid 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:stretch_mode_enabled false 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:max_osd 12 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6802/4225872342,v1:172.21.15.32:6803/4225872342] [v2:172.21.15.32:6804/4225872342,v1:172.21.15.32:6805/4225872342] exists,up 3646a416-b8ca-4d84-9f20-cb3b2cb2273d 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6800/4042955203,v1:172.21.15.82:6801/4042955203] [v2:172.21.15.82:6802/4042955203,v1:172.21.15.82:6803/4042955203] exists,up 0bc55145-c8d6-445c-aa03-7617bf25e77e 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/2834744702,v1:172.21.15.134:6801/2834744702] [v2:172.21.15.134:6802/2834744702,v1:172.21.15.134:6803/2834744702] exists,up b6916f0e-af2f-4e04-a6e1-656f11cc3423 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6810/3929583360,v1:172.21.15.32:6811/3929583360] [v2:172.21.15.32:6812/3929583360,v1:172.21.15.32:6813/3929583360] exists,up 847ede2b-81d0-4993-895f-2e712666ea5a 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6808/3071889784,v1:172.21.15.82:6809/3071889784] [v2:172.21.15.82:6810/3071889784,v1:172.21.15.82:6811/3071889784] exists,up 7f816e73-9c2a-4392-80fd-ad98adc90aed 2024-04-16T05:34:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/1189980878,v1:172.21.15.134:6809/1189980878] [v2:172.21.15.134:6810/1189980878,v1:172.21.15.134:6811/1189980878] exists,up 49194b43-0fad-47b3-ad59-e6f24c8865c7 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6818/1227886227,v1:172.21.15.32:6819/1227886227] [v2:172.21.15.32:6820/1227886227,v1:172.21.15.32:6821/1227886227] exists,up cc3c0af7-426d-4be6-8d1a-87854c3eb8ed 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6816/529517800,v1:172.21.15.82:6817/529517800] [v2:172.21.15.82:6818/529517800,v1:172.21.15.82:6819/529517800] exists,up 5a3fe236-c01c-45f0-896a-f97bee5a4fb3 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/2280284325,v1:172.21.15.134:6817/2280284325] [v2:172.21.15.134:6818/2280284325,v1:172.21.15.134:6819/2280284325] exists,up b3917ee4-9f4e-400e-be3e-0e0558dd8881 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6826/192041066,v1:172.21.15.32:6827/192041066] [v2:172.21.15.32:6828/192041066,v1:172.21.15.32:6829/192041066] exists,up 981db02c-51a8-4779-bc37-1c111ef20bc9 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6824/4010044592,v1:172.21.15.82:6825/4010044592] [v2:172.21.15.82:6826/4010044592,v1:172.21.15.82:6827/4010044592] exists,up 13a54f6a-923a-4ce0-b6fb-f5331706fe76 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/1766870366,v1:172.21.15.134:6825/1766870366] [v2:172.21.15.134:6826/1766870366,v1:172.21.15.134:6827/1766870366] exists,up 67e3e8a5-0d88-445e-b37a-d7cd3db91a06 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.13 [2,11] 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.1d [8,11] 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.0 [5,9] 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.3 [4,9] 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.6 [4,6] 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.3c [4,9] 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 4.b [9,8] 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 4.e [5,8] 2024-04-16T05:34:42.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 4.16 [9,10] 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 4.17 [9,8] 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/2557553342 expires 2024-04-17T05:25:32.817074+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/1590295441 expires 2024-04-17T05:25:32.817074+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/1590295441 expires 2024-04-17T05:25:32.817074+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/2929016634 expires 2024-04-17T05:25:32.817074+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/26695350 expires 2024-04-17T05:25:04.672077+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3665169244 expires 2024-04-17T05:25:04.672077+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/3517252554 expires 2024-04-17T05:25:04.672077+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/1771946956 expires 2024-04-17T05:25:04.672077+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/3517252554 expires 2024-04-17T05:25:04.672077+0000 2024-04-16T05:34:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/1006904605 expires 2024-04-17T05:25:32.817074+0000 2024-04-16T05:34:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f80100a3c40 msgr2=0x7f80100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f80100a3c40 0x7f80100a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f801800b580 tx=0x7f8018005fb0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028103d50 msgr2=0x7f80281992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028103d50 0x7f80281992e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f802400d980 tx=0x7f802400dc90 comp rx=0 tx=0).stop 2024-04-16T05:34:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 shutdown_connections 2024-04-16T05:34:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f80100a3c40 0x7f80100a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80281046f0 0x7f80281a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8028102b50 0x7f8028198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 --2- 172.21.15.32:0/3045446779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028103d50 0x7f80281992e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 >> 172.21.15.32:0/3045446779 conn(0x7f80280fe0f0 msgr2=0x7f8028107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 shutdown_connections 2024-04-16T05:34:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:42.589+0000 7f802f0f6700 1 -- 172.21.15.32:0/3045446779 wait complete. 2024-04-16T05:34:42.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:42 smithi032 ceph-mon[32351]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:42.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:42 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3045446779' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T05:34:43.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:42 smithi134 ceph-mon[36153]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:43.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:42 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3045446779' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T05:34:43.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:42 smithi082 ceph-mon[36195]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:34:43.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:42 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3045446779' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T05:34:43.324 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-16T05:34:43.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 -- 172.21.15.32:0/1906946781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10841020d0 msgr2=0x7f1084102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 --2- 172.21.15.32:0/1906946781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10841020d0 0x7f1084102550 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1074009900 tx=0x7f107402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 -- 172.21.15.32:0/1906946781 shutdown_connections 2024-04-16T05:34:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 --2- 172.21.15.32:0/1906946781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084102a90 0x7f10841093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 --2- 172.21.15.32:0/1906946781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10841020d0 0x7f1084102550 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 --2- 172.21.15.32:0/1906946781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084100f00 0x7f1084101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 -- 172.21.15.32:0/1906946781 >> 172.21.15.32:0/1906946781 conn(0x7f10840fc500 msgr2=0x7f10840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 -- 172.21.15.32:0/1906946781 shutdown_connections 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.446+0000 7f108b5bd640 1 -- 172.21.15.32:0/1906946781 wait complete. 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f108b5bd640 1 Processor -- start 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f108b5bd640 1 -- start start 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f108b5bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1084100f00 0x7f108406b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f108b5bd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10841020d0 0x7f108419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f108b5bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084102a90 0x7f10841a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f108b5bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f108410cbd0 con 0x7f1084100f00 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f108b5bd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f108410ca50 con 0x7f10841020d0 2024-04-16T05:34:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f108b5bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f108410cd50 con 0x7f1084102a90 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1084100f00 0x7f108406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1084100f00 0x7f108406b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59736/0 (socket says 172.21.15.32:59736) 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 -- 172.21.15.32:0/4258779121 learned_addr learned my addr 172.21.15.32:0/4258779121 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1088b58640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10841020d0 0x7f108419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089b5a640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084102a90 0x7f10841a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 -- 172.21.15.32:0/4258779121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084102a90 msgr2=0x7f10841a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084102a90 0x7f10841a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 -- 172.21.15.32:0/4258779121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10841020d0 msgr2=0x7f108419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10841020d0 0x7f108419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 -- 172.21.15.32:0/4258779121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10841a2200 con 0x7f1084100f00 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089b5a640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084102a90 0x7f10841a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1088b58640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10841020d0 0x7f108419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f1089359640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1084100f00 0x7f108406b210 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f108000ea80 tx=0x7f108000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.447+0000 7f107a7fc640 1 -- 172.21.15.32:0/4258779121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108000ce40 con 0x7f1084100f00 2024-04-16T05:34:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.448+0000 7f107a7fc640 1 -- 172.21.15.32:0/4258779121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1080011950 con 0x7f1084100f00 2024-04-16T05:34:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.448+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10841a2490 con 0x7f1084100f00 2024-04-16T05:34:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.448+0000 7f107a7fc640 1 -- 172.21.15.32:0/4258779121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108001ab80 con 0x7f1084100f00 2024-04-16T05:34:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.448+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10841a29e0 con 0x7f1084100f00 2024-04-16T05:34:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.449+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1084108310 con 0x7f1084100f00 2024-04-16T05:34:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.450+0000 7f107a7fc640 1 -- 172.21.15.32:0/4258779121 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1080011b00 con 0x7f1084100f00 2024-04-16T05:34:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.450+0000 7f107a7fc640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f10700a3be0 0x7f10700a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.450+0000 7f107a7fc640 1 -- 172.21.15.32:0/4258779121 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f10800d0e40 con 0x7f1084100f00 2024-04-16T05:34:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.450+0000 7f1088b58640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f10700a3be0 0x7f10700a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.451+0000 7f1088b58640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f10700a3be0 0x7f10700a60a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1074036040 tx=0x7f1074000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.455+0000 7f107a7fc640 1 -- 172.21.15.32:0/4258779121 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1080016030 con 0x7f1084100f00 2024-04-16T05:34:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.645+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1084062ca0 con 0x7f1084100f00 2024-04-16T05:34:43.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.646+0000 7f107a7fc640 1 -- 172.21.15.32:0/4258779121 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24618 (secure 0 0 0) 0x7f108009d4c0 con 0x7f1084100f00 2024-04-16T05:34:43.645 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:43.645 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:33:34.757328+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f10700a3be0 msgr2=0x7f10700a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f10700a3be0 0x7f10700a60a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1074036040 tx=0x7f1074000bc0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1084100f00 msgr2=0x7f108406b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1084100f00 0x7f108406b210 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f108000ea80 tx=0x7f108000ef40 comp rx=0 tx=0).stop 2024-04-16T05:34:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 shutdown_connections 2024-04-16T05:34:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f10700a3be0 0x7f10700a60a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084102a90 0x7f10841a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10841020d0 0x7f108419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 --2- 172.21.15.32:0/4258779121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1084100f00 0x7f108406b210 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 >> 172.21.15.32:0/4258779121 conn(0x7f10840fc500 msgr2=0x7f10841054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 shutdown_connections 2024-04-16T05:34:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.648+0000 7f108b5bd640 1 -- 172.21.15.32:0/4258779121 wait complete. 2024-04-16T05:34:43.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:43.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/1699807468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc100f10 msgr2=0x7f2bcc101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/1699807468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc100f10 0x7f2bcc101350 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009900 tx=0x7f2bc002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/1699807468 shutdown_connections 2024-04-16T05:34:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/1699807468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bcc102aa0 0x7f2bcc1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/1699807468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bcc1020e0 0x7f2bcc102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/1699807468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc100f10 0x7f2bcc101350 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/1699807468 >> 172.21.15.32:0/1699807468 conn(0x7f2bcc0fc4f0 msgr2=0x7f2bcc0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/1699807468 shutdown_connections 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.769+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/1699807468 wait complete. 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd2ee4640 1 Processor -- start 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd2ee4640 1 -- start start 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd2ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bcc100f10 0x7f2bcc104f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd2ee4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bcc1020e0 0x7f2bcc103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd2ee4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc102aa0 0x7f2bcc103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd2ee4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10cc10 con 0x7f2bcc102aa0 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd2ee4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10ca90 con 0x7f2bcc100f10 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd2ee4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10cd90 con 0x7f2bcc1020e0 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc102aa0 0x7f2bcc103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc102aa0 0x7f2bcc103a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59756/0 (socket says 172.21.15.32:59756) 2024-04-16T05:34:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 -- 172.21.15.32:0/4185851770 learned_addr learned my addr 172.21.15.32:0/4185851770 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd0c80640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bcc100f10 0x7f2bcc104f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bcbfff640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bcc1020e0 0x7f2bcc103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 -- 172.21.15.32:0/4185851770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bcc1020e0 msgr2=0x7f2bcc103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bcc1020e0 0x7f2bcc103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 -- 172.21.15.32:0/4185851770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bcc100f10 msgr2=0x7f2bcc104f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bcc100f10 0x7f2bcc104f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 -- 172.21.15.32:0/4185851770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc104320 con 0x7f2bcc102aa0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bcbfff640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bcc1020e0 0x7f2bcc103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.770+0000 7f2bd1481640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc102aa0 0x7f2bcc103a90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400cce0 tx=0x7f2bc400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.771+0000 7f2bc9ffb640 1 -- 172.21.15.32:0/4185851770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400f450 con 0x7f2bcc102aa0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.771+0000 7f2bc9ffb640 1 -- 172.21.15.32:0/4185851770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc400fad0 con 0x7f2bcc102aa0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.771+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc1a2470 con 0x7f2bcc102aa0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.771+0000 7f2bc9ffb640 1 -- 172.21.15.32:0/4185851770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4018c10 con 0x7f2bcc102aa0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.771+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc1a29c0 con 0x7f2bcc102aa0 2024-04-16T05:34:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.772+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bcc1a2ce0 con 0x7f2bcc102aa0 2024-04-16T05:34:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.773+0000 7f2bc9ffb640 1 -- 172.21.15.32:0/4185851770 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2bc400f5f0 con 0x7f2bcc102aa0 2024-04-16T05:34:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.773+0000 7f2bc9ffb640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2bbc0a3c30 0x7f2bbc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.773+0000 7f2bd0c80640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2bbc0a3c30 0x7f2bbc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.773+0000 7f2bc9ffb640 1 -- 172.21.15.32:0/4185851770 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2bc40d0e80 con 0x7f2bcc102aa0 2024-04-16T05:34:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.774+0000 7f2bd0c80640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2bbc0a3c30 0x7f2bbc0a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bc002c240 tx=0x7f2bc002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:43.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.778+0000 7f2bc9ffb640 1 -- 172.21.15.32:0/4185851770 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bc4014030 con 0x7f2bcc102aa0 2024-04-16T05:34:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.965+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2bcc1a3070 con 0x7f2bcc102aa0 2024-04-16T05:34:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.965+0000 7f2bc9ffb640 1 -- 172.21.15.32:0/4185851770 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f2bc409d500 con 0x7f2bcc102aa0 2024-04-16T05:34:43.964 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:34:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2bbc0a3c30 msgr2=0x7f2bbc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2bbc0a3c30 0x7f2bbc0a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bc002c240 tx=0x7f2bc002df90 comp rx=0 tx=0).stop 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc102aa0 msgr2=0x7f2bcc103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc102aa0 0x7f2bcc103a90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400cce0 tx=0x7f2bc400a620 comp rx=0 tx=0).stop 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 shutdown_connections 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f2bbc0a3c30 0x7f2bbc0a60f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bcc102aa0 0x7f2bcc103a90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bcc1020e0 0x7f2bcc103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 --2- 172.21.15.32:0/4185851770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bcc100f10 0x7f2bcc104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.967+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 >> 172.21.15.32:0/4185851770 conn(0x7f2bcc0fc4f0 msgr2=0x7f2bcc0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.968+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 shutdown_connections 2024-04-16T05:34:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:43.968+0000 7f2bd2ee4640 1 -- 172.21.15.32:0/4185851770 wait complete. 2024-04-16T05:34:43.976 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:43 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4258779121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:43.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:34:44.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 -- 172.21.15.32:0/1996642987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 msgr2=0x7fbdc8104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 --2- 172.21.15.32:0/1996642987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 0x7fbdc8104ed0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009900 tx=0x7fbdc402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:44.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 -- 172.21.15.32:0/1996642987 shutdown_connections 2024-04-16T05:34:44.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 --2- 172.21.15.32:0/1996642987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8107d40 0x7fbdc810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 --2- 172.21.15.32:0/1996642987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc8105410 0x7fbdc8107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 --2- 172.21.15.32:0/1996642987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 0x7fbdc8104ed0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 -- 172.21.15.32:0/1996642987 >> 172.21.15.32:0/1996642987 conn(0x7fbdc80fc4f0 msgr2=0x7fbdc80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 -- 172.21.15.32:0/1996642987 shutdown_connections 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.087+0000 7fbdd0488640 1 -- 172.21.15.32:0/1996642987 wait complete. 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdd0488640 1 Processor -- start 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdd0488640 1 -- start start 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdd0488640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 0x7fbdc8102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdd0488640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8105410 0x7fbdc8100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdd0488640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc8107d40 0x7fbdc8100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdd0488640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc810dcf0 con 0x7fbdc8102ae0 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdd0488640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc810db70 con 0x7fbdc8107d40 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdd0488640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc810de70 con 0x7fbdc8105410 2024-04-16T05:34:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdce224640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 0x7fbdc8102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdce224640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 0x7fbdc8102450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59778/0 (socket says 172.21.15.32:59778) 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdce224640 1 -- 172.21.15.32:0/4127708460 learned_addr learned my addr 172.21.15.32:0/4127708460 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdce224640 1 -- 172.21.15.32:0/4127708460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8105410 msgr2=0x7fbdc8100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdce224640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8105410 0x7fbdc8100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdce224640 1 -- 172.21.15.32:0/4127708460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc8107d40 msgr2=0x7fbdc8100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdce224640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc8107d40 0x7fbdc8100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.088+0000 7fbdce224640 1 -- 172.21.15.32:0/4127708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc8101860 con 0x7fbdc8102ae0 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.089+0000 7fbdce224640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 0x7fbdc8102450 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009230 tx=0x7fbdc4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.089+0000 7fbdbf7fe640 1 -- 172.21.15.32:0/4127708460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4041070 con 0x7fbdc8102ae0 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.089+0000 7fbdbf7fe640 1 -- 172.21.15.32:0/4127708460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc40054e0 con 0x7fbdc8102ae0 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.089+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc81a23f0 con 0x7fbdc8102ae0 2024-04-16T05:34:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.089+0000 7fbdbf7fe640 1 -- 172.21.15.32:0/4127708460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40346a0 con 0x7fbdc8102ae0 2024-04-16T05:34:44.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.089+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc81a2940 con 0x7fbdc8102ae0 2024-04-16T05:34:44.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.090+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc80fdfe0 con 0x7fbdc8102ae0 2024-04-16T05:34:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.091+0000 7fbdbf7fe640 1 -- 172.21.15.32:0/4127708460 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbdc4044070 con 0x7fbdc8102ae0 2024-04-16T05:34:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.091+0000 7fbdbf7fe640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbdb40a3bb0 0x7fbdb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.092+0000 7fbdcda23640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbdb40a3bb0 0x7fbdb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.092+0000 7fbdbf7fe640 1 -- 172.21.15.32:0/4127708460 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbdc40f29b0 con 0x7fbdc8102ae0 2024-04-16T05:34:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.092+0000 7fbdcda23640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbdb40a3bb0 0x7fbdb40a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbdc81021c0 tx=0x7fbdb8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.095+0000 7fbdbf7fe640 1 -- 172.21.15.32:0/4127708460 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbdc40befd0 con 0x7fbdc8102ae0 2024-04-16T05:34:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:43 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4258779121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:43 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4258779121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.281+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbdc802d620 con 0x7fbdc8102ae0 2024-04-16T05:34:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.282+0000 7fbdbf7fe640 1 -- 172.21.15.32:0/4127708460 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fbdc40be8d0 con 0x7fbdc8102ae0 2024-04-16T05:34:44.281 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbdb40a3bb0 msgr2=0x7fbdb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbdb40a3bb0 0x7fbdb40a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbdc81021c0 tx=0x7fbdb8005c70 comp rx=0 tx=0).stop 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 msgr2=0x7fbdc8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 0x7fbdc8102450 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009230 tx=0x7fbdc4004040 comp rx=0 tx=0).stop 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 shutdown_connections 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbdb40a3bb0 0x7fbdb40a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc8107d40 0x7fbdc8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8105410 0x7fbdc8100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 --2- 172.21.15.32:0/4127708460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8102ae0 0x7fbdc8102450 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 >> 172.21.15.32:0/4127708460 conn(0x7fbdc80fc4f0 msgr2=0x7fbdc8103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 shutdown_connections 2024-04-16T05:34:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.284+0000 7fbdd0488640 1 -- 172.21.15.32:0/4127708460 wait complete. 2024-04-16T05:34:44.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.404+0000 7f2260b5f640 1 -- 172.21.15.32:0/3197044433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 msgr2=0x7f225c101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.404+0000 7f2260b5f640 1 --2- 172.21.15.32:0/3197044433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 0x7f225c101d40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2244009900 tx=0x7f224402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 -- 172.21.15.32:0/3197044433 shutdown_connections 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 --2- 172.21.15.32:0/3197044433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f225c102280 0x7f225c108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 --2- 172.21.15.32:0/3197044433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 0x7f225c101d40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 --2- 172.21.15.32:0/3197044433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1006d0 0x7f225c100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 -- 172.21.15.32:0/3197044433 >> 172.21.15.32:0/3197044433 conn(0x7f225c0fbc40 msgr2=0x7f225c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 -- 172.21.15.32:0/3197044433 shutdown_connections 2024-04-16T05:34:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 -- 172.21.15.32:0/3197044433 wait complete. 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 Processor -- start 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.405+0000 7f2260b5f640 1 -- start start 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2260b5f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1006d0 0x7f225c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2260b5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 0x7f225c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2260b5f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f225c102280 0x7f225c19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2260b5f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f225c10bae0 con 0x7f225c1018c0 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2260b5f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f225c10b960 con 0x7f225c1006d0 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2260b5f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f225c10bc60 con 0x7f225c102280 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 0x7f225c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 0x7f225c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59800/0 (socket says 172.21.15.32:59800) 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 -- 172.21.15.32:0/719772936 learned_addr learned my addr 172.21.15.32:0/719772936 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f225ad9d640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f225c102280 0x7f225c19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f225a59c640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1006d0 0x7f225c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 -- 172.21.15.32:0/719772936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f225c102280 msgr2=0x7f225c19f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f225c102280 0x7f225c19f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 -- 172.21.15.32:0/719772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1006d0 msgr2=0x7f225c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1006d0 0x7f225c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 -- 172.21.15.32:0/719772936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c19ff40 con 0x7f225c1018c0 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f225ad9d640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f225c102280 0x7f225c19f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f225a59c640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1006d0 0x7f225c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2259d9b640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 0x7f225c198860 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2244009230 tx=0x7f2244005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f22537fe640 1 -- 172.21.15.32:0/719772936 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244041070 con 0x7f225c1018c0 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f22537fe640 1 -- 172.21.15.32:0/719772936 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22440054e0 con 0x7f225c1018c0 2024-04-16T05:34:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c1a01d0 con 0x7f225c1018c0 2024-04-16T05:34:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f22537fe640 1 -- 172.21.15.32:0/719772936 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244034440 con 0x7f225c1018c0 2024-04-16T05:34:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.406+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c1a06c0 con 0x7f225c1018c0 2024-04-16T05:34:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.407+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c1079f0 con 0x7f225c1018c0 2024-04-16T05:34:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.408+0000 7f22537fe640 1 -- 172.21.15.32:0/719772936 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2244044070 con 0x7f225c1018c0 2024-04-16T05:34:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.409+0000 7f22537fe640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f22480a3bb0 0x7f22480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.409+0000 7f225a59c640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f22480a3bb0 0x7f22480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.409+0000 7f22537fe640 1 -- 172.21.15.32:0/719772936 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f22440f2590 con 0x7f225c1018c0 2024-04-16T05:34:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.410+0000 7f225a59c640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f22480a3bb0 0x7f22480a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2254005d00 tx=0x7f2254005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.414+0000 7f22537fe640 1 -- 172.21.15.32:0/719772936 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22440bebb0 con 0x7f225c1018c0 2024-04-16T05:34:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.599+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f225c062c00 con 0x7f225c1018c0 2024-04-16T05:34:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.600+0000 7f22537fe640 1 -- 172.21.15.32:0/719772936 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f22440be4b0 con 0x7f225c1018c0 2024-04-16T05:34:44.599 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:34:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.601+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f22480a3bb0 msgr2=0x7f22480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.601+0000 7f2260b5f640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f22480a3bb0 0x7f22480a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2254005d00 tx=0x7f2254005ef0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.601+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 msgr2=0x7f225c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.601+0000 7f2260b5f640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 0x7f225c198860 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2244009230 tx=0x7f2244005dd0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.602+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 shutdown_connections 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.602+0000 7f2260b5f640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f22480a3bb0 0x7f22480a6070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.602+0000 7f2260b5f640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f225c102280 0x7f225c19f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.602+0000 7f2260b5f640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c1018c0 0x7f225c198860 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.602+0000 7f2260b5f640 1 --2- 172.21.15.32:0/719772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1006d0 0x7f225c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.602+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 >> 172.21.15.32:0/719772936 conn(0x7f225c0fbc40 msgr2=0x7f225c104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.602+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 shutdown_connections 2024-04-16T05:34:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.602+0000 7f2260b5f640 1 -- 172.21.15.32:0/719772936 wait complete. 2024-04-16T05:34:44.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:34:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 -- 172.21.15.32:0/2254415089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e00100f10 msgr2=0x7f4e00101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 --2- 172.21.15.32:0/2254415089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e00100f10 0x7f4e00101350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009900 tx=0x7f4df002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 -- 172.21.15.32:0/2254415089 shutdown_connections 2024-04-16T05:34:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 --2- 172.21.15.32:0/2254415089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00102aa0 0x7f4e001093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 --2- 172.21.15.32:0/2254415089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e001020e0 0x7f4e00102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 --2- 172.21.15.32:0/2254415089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e00100f10 0x7f4e00101350 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 -- 172.21.15.32:0/2254415089 >> 172.21.15.32:0/2254415089 conn(0x7f4e000fc4f0 msgr2=0x7f4e000fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 -- 172.21.15.32:0/2254415089 shutdown_connections 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.722+0000 7f4e069e6640 1 -- 172.21.15.32:0/2254415089 wait complete. 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e069e6640 1 Processor -- start 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e069e6640 1 -- start start 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e069e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00100f10 0x7f4e001972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e069e6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001020e0 0x7f4e00197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e069e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00102aa0 0x7f4e0019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e069e6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010cc30 con 0x7f4e001020e0 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e069e6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010cab0 con 0x7f4e00100f10 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e069e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010cdb0 con 0x7f4e00102aa0 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4dff7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001020e0 0x7f4e00197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4dff7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001020e0 0x7f4e00197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59820/0 (socket says 172.21.15.32:59820) 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4dff7fe640 1 -- 172.21.15.32:0/3045933998 learned_addr learned my addr 172.21.15.32:0/3045933998 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00100f10 0x7f4e001972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.723+0000 7f4e04f83640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00102aa0 0x7f4e0019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dff7fe640 1 -- 172.21.15.32:0/3045933998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00102aa0 msgr2=0x7f4e0019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00102aa0 0x7f4e0019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dff7fe640 1 -- 172.21.15.32:0/3045933998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00100f10 msgr2=0x7f4e001972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00100f10 0x7f4e001972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dff7fe640 1 -- 172.21.15.32:0/3045933998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0019ef10 con 0x7f4e001020e0 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001020e0 0x7f4e00197830 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4df400c970 tx=0x7f4df400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dfd7fa640 1 -- 172.21.15.32:0/3045933998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4007bb0 con 0x7f4e001020e0 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dfd7fa640 1 -- 172.21.15.32:0/3045933998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df4007d50 con 0x7f4e001020e0 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0019f200 con 0x7f4e001020e0 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4dfd7fa640 1 -- 172.21.15.32:0/3045933998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df400f450 con 0x7f4e001020e0 2024-04-16T05:34:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.724+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0019f700 con 0x7f4e001020e0 2024-04-16T05:34:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.725+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e00108370 con 0x7f4e001020e0 2024-04-16T05:34:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.726+0000 7f4dfd7fa640 1 -- 172.21.15.32:0/3045933998 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4df4005020 con 0x7f4e001020e0 2024-04-16T05:34:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.727+0000 7f4dfd7fa640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4de80a3bb0 0x7f4de80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.727+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4de80a3bb0 0x7f4de80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.727+0000 7f4dfd7fa640 1 -- 172.21.15.32:0/3045933998 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4df40d0560 con 0x7f4e001020e0 2024-04-16T05:34:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.727+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4de80a3bb0 0x7f4de80a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4df002c170 tx=0x7f4df002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.731+0000 7f4dfd7fa640 1 -- 172.21.15.32:0/3045933998 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4df4014030 con 0x7f4e001020e0 2024-04-16T05:34:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.920+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4e00062c00 con 0x7f4e001020e0 2024-04-16T05:34:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.920+0000 7f4dfd7fa640 1 -- 172.21.15.32:0/3045933998 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f4df409cbe0 con 0x7f4e001020e0 2024-04-16T05:34:44.919 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4de80a3bb0 msgr2=0x7f4de80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4de80a3bb0 0x7f4de80a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4df002c170 tx=0x7f4df002df90 comp rx=0 tx=0).stop 2024-04-16T05:34:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001020e0 msgr2=0x7f4e00197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001020e0 0x7f4e00197830 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4df400c970 tx=0x7f4df400ce30 comp rx=0 tx=0).stop 2024-04-16T05:34:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 shutdown_connections 2024-04-16T05:34:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4de80a3bb0 0x7f4de80a6070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00102aa0 0x7f4e0019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001020e0 0x7f4e00197830 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 --2- 172.21.15.32:0/3045933998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00100f10 0x7f4e001972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 >> 172.21.15.32:0/3045933998 conn(0x7f4e000fc4f0 msgr2=0x7f4e00105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 shutdown_connections 2024-04-16T05:34:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:44.922+0000 7f4e069e6640 1 -- 172.21.15.32:0/3045933998 wait complete. 2024-04-16T05:34:44.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:44 smithi032 ceph-mon[32351]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:44.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:44 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4185851770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:44.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:44 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4127708460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:44.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:44 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/719772936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:44.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 -- 172.21.15.32:0/564817729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 msgr2=0x7f38ac104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/564817729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 0x7f38ac104f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f389c009900 tx=0x7f389c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 -- 172.21.15.32:0/564817729 shutdown_connections 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/564817729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38ac107da0 0x7f38ac10a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/564817729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38ac105470 0x7f38ac107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/564817729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 0x7f38ac104f30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 -- 172.21.15.32:0/564817729 >> 172.21.15.32:0/564817729 conn(0x7f38ac0fc550 msgr2=0x7f38ac0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 -- 172.21.15.32:0/564817729 shutdown_connections 2024-04-16T05:34:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 -- 172.21.15.32:0/564817729 wait complete. 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.040+0000 7f38b1ce4640 1 Processor -- start 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38b1ce4640 1 -- start start 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38b1ce4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 0x7f38ac1024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38b1ce4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38ac105470 0x7f38ac100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38b1ce4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38ac107da0 0x7f38ac101030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38b1ce4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac10dce0 con 0x7f38ac102b40 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38b1ce4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac10db60 con 0x7f38ac105470 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38b1ce4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac10de60 con 0x7f38ac107da0 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 0x7f38ac1024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 0x7f38ac1024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59852/0 (socket says 172.21.15.32:59852) 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 -- 172.21.15.32:0/4132063128 learned_addr learned my addr 172.21.15.32:0/4132063128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38abfff640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38ac107da0 0x7f38ac101030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38aaffd640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38ac105470 0x7f38ac100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 -- 172.21.15.32:0/4132063128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38ac107da0 msgr2=0x7f38ac101030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38ac107da0 0x7f38ac101030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 -- 172.21.15.32:0/4132063128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38ac105470 msgr2=0x7f38ac100af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38ac105470 0x7f38ac100af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 -- 172.21.15.32:0/4132063128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac1018c0 con 0x7f38ac102b40 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38abfff640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38ac107da0 0x7f38ac101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38aaffd640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38ac105470 0x7f38ac100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38ab7fe640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 0x7f38ac1024b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f389c009230 tx=0x7f389c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.041+0000 7f38a8ff9640 1 -- 172.21.15.32:0/4132063128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c041070 con 0x7f38ac102b40 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.042+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac1a2310 con 0x7f38ac102b40 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.042+0000 7f38a8ff9640 1 -- 172.21.15.32:0/4132063128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c0054e0 con 0x7f38ac102b40 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.042+0000 7f38a8ff9640 1 -- 172.21.15.32:0/4132063128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0346a0 con 0x7f38ac102b40 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.042+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac1a28b0 con 0x7f38ac102b40 2024-04-16T05:34:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.043+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac0fe040 con 0x7f38ac102b40 2024-04-16T05:34:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.044+0000 7f38a8ff9640 1 -- 172.21.15.32:0/4132063128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f389c044070 con 0x7f38ac102b40 2024-04-16T05:34:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.044+0000 7f38a8ff9640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f38940a3bb0 0x7f38940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.045+0000 7f38aaffd640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f38940a3bb0 0x7f38940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.045+0000 7f38a8ff9640 1 -- 172.21.15.32:0/4132063128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f389c0f2950 con 0x7f38ac102b40 2024-04-16T05:34:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.045+0000 7f38aaffd640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f38940a3bb0 0x7f38940a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f38ac102220 tx=0x7f38a0005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.048+0000 7f38a8ff9640 1 -- 172.21.15.32:0/4132063128 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f389c0befd0 con 0x7f38ac102b40 2024-04-16T05:34:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:44 smithi134 ceph-mon[36153]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:44 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4185851770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:44 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4127708460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:45.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:44 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/719772936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:45.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:44 smithi082 ceph-mon[36195]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T05:34:45.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:44 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4185851770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:45.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:44 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4127708460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:45.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:44 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/719772936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:45.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.273+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38ac02d620 con 0x7f38ac102b40 2024-04-16T05:34:45.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.274+0000 7f38a8ff9640 1 -- 172.21.15.32:0/4132063128 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11876 (secure 0 0 0) 0x7f389c0be8d0 con 0x7f38ac102b40 2024-04-16T05:34:45.273 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:45.273 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:32:55.688634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24461,24494]},"id":1}]} 2024-04-16T05:34:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-16T05:34:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f38940a3bb0 msgr2=0x7f38940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f38940a3bb0 0x7f38940a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f38ac102220 tx=0x7f38a0005d90 comp rx=0 tx=0).stop 2024-04-16T05:34:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 msgr2=0x7f38ac1024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 0x7f38ac1024b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f389c009230 tx=0x7f389c004040 comp rx=0 tx=0).stop 2024-04-16T05:34:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 shutdown_connections 2024-04-16T05:34:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f38940a3bb0 0x7f38940a6070 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38ac107da0 0x7f38ac101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38ac105470 0x7f38ac100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 --2- 172.21.15.32:0/4132063128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ac102b40 0x7f38ac1024b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 >> 172.21.15.32:0/4132063128 conn(0x7f38ac0fc550 msgr2=0x7f38ac103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:45.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 shutdown_connections 2024-04-16T05:34:45.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.276+0000 7f38b1ce4640 1 -- 172.21.15.32:0/4132063128 wait complete. 2024-04-16T05:34:45.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.393+0000 7fb7bbdca640 1 -- 172.21.15.32:0/20602328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 msgr2=0x7fb7b4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.393+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/20602328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 0x7fb7b4102470 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009ca0 tx=0x7fb7a402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.393+0000 7fb7bbdca640 1 -- 172.21.15.32:0/20602328 shutdown_connections 2024-04-16T05:34:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.393+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/20602328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7b4103c00 0x7fb7b410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.393+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/20602328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b4103240 0x7fb7b41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.393+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/20602328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 0x7fb7b4102470 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.393+0000 7fb7bbdca640 1 -- 172.21.15.32:0/20602328 >> 172.21.15.32:0/20602328 conn(0x7fb7b40fd5e0 msgr2=0x7fb7b40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 -- 172.21.15.32:0/20602328 shutdown_connections 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 -- 172.21.15.32:0/20602328 wait complete. 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 Processor -- start 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 -- start start 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 0x7fb7b4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7b4103240 0x7fb7b41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b4103c00 0x7fb7b419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b410dce0 con 0x7fb7b4102050 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b410db60 con 0x7fb7b4103240 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7bbdca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b410de60 con 0x7fb7b4103c00 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7b9b66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 0x7fb7b4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7b9b66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 0x7fb7b4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59876/0 (socket says 172.21.15.32:59876) 2024-04-16T05:34:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.394+0000 7fb7b9b66640 1 -- 172.21.15.32:0/279934228 learned_addr learned my addr 172.21.15.32:0/279934228 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7ba367640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b4103c00 0x7fb7b419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b9365640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7b4103240 0x7fb7b41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b9b66640 1 -- 172.21.15.32:0/279934228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b4103c00 msgr2=0x7fb7b419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b9b66640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b4103c00 0x7fb7b419f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b9b66640 1 -- 172.21.15.32:0/279934228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7b4103240 msgr2=0x7fb7b41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b9b66640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7b4103240 0x7fb7b41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b9b66640 1 -- 172.21.15.32:0/279934228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b419ff90 con 0x7fb7b4102050 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7ba367640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b4103c00 0x7fb7b419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b9365640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7b4103240 0x7fb7b41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b9b66640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 0x7fb7b4198370 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40324c0 tx=0x7fb7a4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b2ffd640 1 -- 172.21.15.32:0/279934228 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4039040 con 0x7fb7b4102050 2024-04-16T05:34:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b41a0220 con 0x7fb7b4102050 2024-04-16T05:34:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b2ffd640 1 -- 172.21.15.32:0/279934228 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4044070 con 0x7fb7b4102050 2024-04-16T05:34:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7b2ffd640 1 -- 172.21.15.32:0/279934228 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40042a0 con 0x7fb7b4102050 2024-04-16T05:34:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.395+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b41a0710 con 0x7fb7b4102050 2024-04-16T05:34:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.396+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b4109370 con 0x7fb7b4102050 2024-04-16T05:34:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.397+0000 7fb7b2ffd640 1 -- 172.21.15.32:0/279934228 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb7a4003970 con 0x7fb7b4102050 2024-04-16T05:34:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.398+0000 7fb7b2ffd640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb79c0a3bb0 0x7fb79c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.398+0000 7fb7b2ffd640 1 -- 172.21.15.32:0/279934228 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb7a40f3790 con 0x7fb7b4102050 2024-04-16T05:34:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.398+0000 7fb7b9365640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb79c0a3bb0 0x7fb79c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.398+0000 7fb7b9365640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb79c0a3bb0 0x7fb79c0a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80058c0 tx=0x7fb7a8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.401+0000 7fb7b2ffd640 1 -- 172.21.15.32:0/279934228 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7a40bfe40 con 0x7fb7b4102050 2024-04-16T05:34:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.589+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7b4062c00 con 0x7fb7b4102050 2024-04-16T05:34:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.590+0000 7fb7b2ffd640 1 -- 172.21.15.32:0/279934228 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24618 (secure 0 0 0) 0x7fb7a40bf740 con 0x7fb7b4102050 2024-04-16T05:34:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:45.589 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:33:34.757328+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb79c0a3bb0 msgr2=0x7fb79c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb79c0a3bb0 0x7fb79c0a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80058c0 tx=0x7fb7a8005850 comp rx=0 tx=0).stop 2024-04-16T05:34:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 msgr2=0x7fb7b4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 0x7fb7b4198370 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40324c0 tx=0x7fb7a4032f20 comp rx=0 tx=0).stop 2024-04-16T05:34:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 shutdown_connections 2024-04-16T05:34:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb79c0a3bb0 0x7fb79c0a6070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b4103c00 0x7fb7b419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7b4103240 0x7fb7b41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 --2- 172.21.15.32:0/279934228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7b4102050 0x7fb7b4198370 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.592+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 >> 172.21.15.32:0/279934228 conn(0x7fb7b40fd5e0 msgr2=0x7fb7b4106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.593+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 shutdown_connections 2024-04-16T05:34:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.593+0000 7fb7bbdca640 1 -- 172.21.15.32:0/279934228 wait complete. 2024-04-16T05:34:45.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:45.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- 172.21.15.32:0/3118470727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870107d80 msgr2=0x7fe87010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 --2- 172.21.15.32:0/3118470727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870107d80 0x7fe87010a170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe868009ca0 tx=0x7fe86802c100 comp rx=0 tx=0).stop 2024-04-16T05:34:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- 172.21.15.32:0/3118470727 shutdown_connections 2024-04-16T05:34:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 --2- 172.21.15.32:0/3118470727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870107d80 0x7fe87010a170 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 --2- 172.21.15.32:0/3118470727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe870105450 0x7fe870107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 --2- 172.21.15.32:0/3118470727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe870102b20 0x7fe870104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- 172.21.15.32:0/3118470727 >> 172.21.15.32:0/3118470727 conn(0x7fe8700fc4f0 msgr2=0x7fe8700fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- 172.21.15.32:0/3118470727 shutdown_connections 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- 172.21.15.32:0/3118470727 wait complete. 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 Processor -- start 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- start start 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe870102b20 0x7fe870198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870105450 0x7fe8701987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe870107d80 0x7fe87019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe87010dcd0 con 0x7fe870105450 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe87010db50 con 0x7fe870102b20 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.714+0000 7fe877ad3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe87010de50 con 0x7fe870107d80 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870105450 0x7fe8701987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870105450 0x7fe8701987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59900/0 (socket says 172.21.15.32:59900) 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 -- 172.21.15.32:0/4121888411 learned_addr learned my addr 172.21.15.32:0/4121888411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe876070640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe870107d80 0x7fe87019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87586f640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe870102b20 0x7fe870198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 -- 172.21.15.32:0/4121888411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe870107d80 msgr2=0x7fe87019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe870107d80 0x7fe87019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 -- 172.21.15.32:0/4121888411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe870102b20 msgr2=0x7fe870198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe870102b20 0x7fe870198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 -- 172.21.15.32:0/4121888411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87019feb0 con 0x7fe870105450 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe876070640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe870107d80 0x7fe87019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87586f640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe870102b20 0x7fe870198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe87506e640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870105450 0x7fe8701987d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c970 tx=0x7fe86000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe866ffd640 1 -- 172.21.15.32:0/4121888411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860014070 con 0x7fe870105450 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe866ffd640 1 -- 172.21.15.32:0/4121888411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe860004440 con 0x7fe870105450 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8701a01a0 con 0x7fe870105450 2024-04-16T05:34:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe866ffd640 1 -- 172.21.15.32:0/4121888411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86000f450 con 0x7fe870105450 2024-04-16T05:34:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.715+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8701a06f0 con 0x7fe870105450 2024-04-16T05:34:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.716+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8700fdfe0 con 0x7fe870105450 2024-04-16T05:34:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.718+0000 7fe866ffd640 1 -- 172.21.15.32:0/4121888411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe860017070 con 0x7fe870105450 2024-04-16T05:34:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.719+0000 7fe866ffd640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe85c0a3bb0 0x7fe85c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.719+0000 7fe87586f640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe85c0a3bb0 0x7fe85c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.719+0000 7fe866ffd640 1 -- 172.21.15.32:0/4121888411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe8600d0b10 con 0x7fe870105450 2024-04-16T05:34:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.719+0000 7fe87586f640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe85c0a3bb0 0x7fe85c0a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0058c0 tx=0x7fe86c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.722+0000 7fe866ffd640 1 -- 172.21.15.32:0/4121888411 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe86001e020 con 0x7fe870105450 2024-04-16T05:34:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.912+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe87002d8b0 con 0x7fe870105450 2024-04-16T05:34:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.912+0000 7fe866ffd640 1 -- 172.21.15.32:0/4121888411 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fe86009d130 con 0x7fe870105450 2024-04-16T05:34:45.911 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:34:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe85c0a3bb0 msgr2=0x7fe85c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe85c0a3bb0 0x7fe85c0a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0058c0 tx=0x7fe86c005d30 comp rx=0 tx=0).stop 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870105450 msgr2=0x7fe8701987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870105450 0x7fe8701987d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c970 tx=0x7fe86000ce30 comp rx=0 tx=0).stop 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 shutdown_connections 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe85c0a3bb0 0x7fe85c0a6070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe870107d80 0x7fe87019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe870105450 0x7fe8701987d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 --2- 172.21.15.32:0/4121888411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe870102b20 0x7fe870198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 >> 172.21.15.32:0/4121888411 conn(0x7fe8700fc4f0 msgr2=0x7fe870103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 shutdown_connections 2024-04-16T05:34:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:45.915+0000 7fe877ad3640 1 -- 172.21.15.32:0/4121888411 wait complete. 2024-04-16T05:34:45.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:45 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3045933998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:45.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:45 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4132063128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:45.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:45 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/279934228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:45.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:34:46.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2680930568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b341006e0 msgr2=0x7f0b34100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2680930568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b341006e0 0x7f0b34100b00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009900 tx=0x7f0b2402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2680930568 shutdown_connections 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2680930568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b34102290 0x7f0b34108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2680930568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b341018d0 0x7f0b34101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2680930568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b341006e0 0x7f0b34100b00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2680930568 >> 172.21.15.32:0/2680930568 conn(0x7f0b340fbc30 msgr2=0x7f0b340fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2680930568 shutdown_connections 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.034+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2680930568 wait complete. 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.035+0000 7f0b39f5f640 1 Processor -- start 2024-04-16T05:34:46.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.035+0000 7f0b39f5f640 1 -- start start 2024-04-16T05:34:46.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.035+0000 7f0b39f5f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b341006e0 0x7f0b34198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.035+0000 7f0b39f5f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b341018d0 0x7f0b34198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.035+0000 7f0b39f5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b34102290 0x7f0b3419f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.035+0000 7f0b39f5f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3410baf0 con 0x7f0b34102290 2024-04-16T05:34:46.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.035+0000 7f0b39f5f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3410b970 con 0x7f0b341018d0 2024-04-16T05:34:46.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.035+0000 7f0b39f5f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3410bc70 con 0x7f0b341006e0 2024-04-16T05:34:46.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b34102290 0x7f0b3419f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b337fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b341006e0 0x7f0b34198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b34102290 0x7f0b3419f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59928/0 (socket says 172.21.15.32:59928) 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 -- 172.21.15.32:0/2574399399 learned_addr learned my addr 172.21.15.32:0/2574399399 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 -- 172.21.15.32:0/2574399399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b341006e0 msgr2=0x7f0b34198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b341006e0 0x7f0b34198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b32ffd640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b341018d0 0x7f0b34198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 -- 172.21.15.32:0/2574399399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b341018d0 msgr2=0x7f0b34198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b341018d0 0x7f0b34198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 -- 172.21.15.32:0/2574399399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3419ffd0 con 0x7f0b34102290 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b33fff640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b34102290 0x7f0b3419f960 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00dd70 tx=0x7f0b2c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b30ff9640 1 -- 172.21.15.32:0/2574399399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c004d20 con 0x7f0b34102290 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b341a02c0 con 0x7f0b34102290 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b30ff9640 1 -- 172.21.15.32:0/2574399399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b2c00be40 con 0x7f0b34102290 2024-04-16T05:34:46.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b30ff9640 1 -- 172.21.15.32:0/2574399399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c005720 con 0x7f0b34102290 2024-04-16T05:34:46.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.036+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b341a0810 con 0x7f0b34102290 2024-04-16T05:34:46.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.037+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b341a0ad0 con 0x7f0b34102290 2024-04-16T05:34:46.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.039+0000 7f0b30ff9640 1 -- 172.21.15.32:0/2574399399 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0b2c0058c0 con 0x7f0b34102290 2024-04-16T05:34:46.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.039+0000 7f0b30ff9640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b1c0a3bb0 0x7f0b1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.039+0000 7f0b337fe640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b1c0a3bb0 0x7f0b1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.039+0000 7f0b30ff9640 1 -- 172.21.15.32:0/2574399399 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0b2c0d0ff0 con 0x7f0b34102290 2024-04-16T05:34:46.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.040+0000 7f0b337fe640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b1c0a3bb0 0x7f0b1c0a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0b24000c00 tx=0x7f0b24005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:46.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.043+0000 7f0b30ff9640 1 -- 172.21.15.32:0/2574399399 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b2c051ff0 con 0x7f0b34102290 2024-04-16T05:34:46.051 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:45 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3045933998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:46.051 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:45 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4132063128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:46.051 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:45 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/279934228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:45 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3045933998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:45 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4132063128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:45 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/279934228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:46.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.229+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0b341a0ca0 con 0x7f0b34102290 2024-04-16T05:34:46.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.229+0000 7f0b30ff9640 1 -- 172.21.15.32:0/2574399399 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f0b2c09d650 con 0x7f0b34102290 2024-04-16T05:34:46.228 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:46.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b1c0a3bb0 msgr2=0x7f0b1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b1c0a3bb0 0x7f0b1c0a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0b24000c00 tx=0x7f0b24005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b34102290 msgr2=0x7f0b3419f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b34102290 0x7f0b3419f960 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00dd70 tx=0x7f0b2c00b730 comp rx=0 tx=0).stop 2024-04-16T05:34:46.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 shutdown_connections 2024-04-16T05:34:46.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f0b1c0a3bb0 0x7f0b1c0a6070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b34102290 0x7f0b3419f960 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b341018d0 0x7f0b34198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 --2- 172.21.15.32:0/2574399399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b341006e0 0x7f0b34198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 >> 172.21.15.32:0/2574399399 conn(0x7f0b340fbc30 msgr2=0x7f0b34106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:46.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 shutdown_connections 2024-04-16T05:34:46.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.231+0000 7f0b39f5f640 1 -- 172.21.15.32:0/2574399399 wait complete. 2024-04-16T05:34:46.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:34:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.351+0000 7fcc37152640 1 -- 172.21.15.32:0/1242757153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 msgr2=0x7fcc3010a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.351+0000 7fcc37152640 1 --2- 172.21.15.32:0/1242757153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 0x7fcc3010a630 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009900 tx=0x7fcc2802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.351+0000 7fcc37152640 1 -- 172.21.15.32:0/1242757153 shutdown_connections 2024-04-16T05:34:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.351+0000 7fcc37152640 1 --2- 172.21.15.32:0/1242757153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 0x7fcc3010a630 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.351+0000 7fcc37152640 1 --2- 172.21.15.32:0/1242757153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc301000c0 0x7fcc30100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.351+0000 7fcc37152640 1 --2- 172.21.15.32:0/1242757153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc30103650 0x7fcc30103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.351+0000 7fcc37152640 1 -- 172.21.15.32:0/1242757153 >> 172.21.15.32:0/1242757153 conn(0x7fcc300fbc40 msgr2=0x7fcc300fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 -- 172.21.15.32:0/1242757153 shutdown_connections 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 -- 172.21.15.32:0/1242757153 wait complete. 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 Processor -- start 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 -- start start 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc301000c0 0x7fcc30198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 0x7fcc30198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30103650 0x7fcc3019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3010dd30 con 0x7fcc30100a80 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3010dbb0 con 0x7fcc301000c0 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc37152640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3010deb0 con 0x7fcc30103650 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 0x7fcc30198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 0x7fcc30198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59934/0 (socket says 172.21.15.32:59934) 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.352+0000 7fcc2ffff640 1 -- 172.21.15.32:0/434434461 learned_addr learned my addr 172.21.15.32:0/434434461 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc34eee640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc301000c0 0x7fcc30198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2ffff640 1 -- 172.21.15.32:0/434434461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30103650 msgr2=0x7fcc3019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2ffff640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30103650 0x7fcc3019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2ffff640 1 -- 172.21.15.32:0/434434461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc301000c0 msgr2=0x7fcc30198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2ffff640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc301000c0 0x7fcc30198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2ffff640 1 -- 172.21.15.32:0/434434461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3019ff40 con 0x7fcc30100a80 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2ffff640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 0x7fcc30198860 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400ea80 tx=0x7fcc2400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2dffb640 1 -- 172.21.15.32:0/434434461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400ce40 con 0x7fcc30100a80 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc301a0230 con 0x7fcc30100a80 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2dffb640 1 -- 172.21.15.32:0/434434461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc24005020 con 0x7fcc30100a80 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc2dffb640 1 -- 172.21.15.32:0/434434461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24011760 con 0x7fcc30100a80 2024-04-16T05:34:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.353+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc301a0730 con 0x7fcc30100a80 2024-04-16T05:34:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.354+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc30109510 con 0x7fcc30100a80 2024-04-16T05:34:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.356+0000 7fcc2dffb640 1 -- 172.21.15.32:0/434434461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcc240051d0 con 0x7fcc30100a80 2024-04-16T05:34:46.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.356+0000 7fcc2dffb640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fcc180a3bb0 0x7fcc180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.356+0000 7fcc34eee640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fcc180a3bb0 0x7fcc180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.356+0000 7fcc2dffb640 1 -- 172.21.15.32:0/434434461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcc240d0eb0 con 0x7fcc30100a80 2024-04-16T05:34:46.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.357+0000 7fcc34eee640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fcc180a3bb0 0x7fcc180a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcc20005db0 tx=0x7fcc20005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:46.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.361+0000 7fcc2dffb640 1 -- 172.21.15.32:0/434434461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc24016030 con 0x7fcc30100a80 2024-04-16T05:34:46.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.546+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcc30062ca0 con 0x7fcc30100a80 2024-04-16T05:34:46.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.546+0000 7fcc2dffb640 1 -- 172.21.15.32:0/434434461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fcc2409d4d0 con 0x7fcc30100a80 2024-04-16T05:34:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:34:46.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fcc180a3bb0 msgr2=0x7fcc180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fcc180a3bb0 0x7fcc180a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcc20005db0 tx=0x7fcc20005d40 comp rx=0 tx=0).stop 2024-04-16T05:34:46.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 msgr2=0x7fcc30198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 0x7fcc30198860 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400ea80 tx=0x7fcc2400ef40 comp rx=0 tx=0).stop 2024-04-16T05:34:46.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 shutdown_connections 2024-04-16T05:34:46.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fcc180a3bb0 0x7fcc180a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30103650 0x7fcc3019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc30100a80 0x7fcc30198860 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 --2- 172.21.15.32:0/434434461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc301000c0 0x7fcc30198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.548+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 >> 172.21.15.32:0/434434461 conn(0x7fcc300fbc40 msgr2=0x7fcc301066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:46.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.549+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 shutdown_connections 2024-04-16T05:34:46.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.549+0000 7fcc37152640 1 -- 172.21.15.32:0/434434461 wait complete. 2024-04-16T05:34:46.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 -- 172.21.15.32:0/4096095229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 msgr2=0x7f8608104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 --2- 172.21.15.32:0/4096095229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 0x7f8608104f10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009900 tx=0x7f85f802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 -- 172.21.15.32:0/4096095229 shutdown_connections 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 --2- 172.21.15.32:0/4096095229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8608107d80 0x7f860810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 --2- 172.21.15.32:0/4096095229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608105450 0x7f8608107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 --2- 172.21.15.32:0/4096095229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 0x7f8608104f10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 -- 172.21.15.32:0/4096095229 >> 172.21.15.32:0/4096095229 conn(0x7f86080fc4b0 msgr2=0x7f86080fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 -- 172.21.15.32:0/4096095229 shutdown_connections 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.668+0000 7f860ea35640 1 -- 172.21.15.32:0/4096095229 wait complete. 2024-04-16T05:34:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860ea35640 1 Processor -- start 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860ea35640 1 -- start start 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860ea35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 0x7f860819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860ea35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608105450 0x7f860819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860ea35640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8608107d80 0x7f86081a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860ea35640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f860810dcd0 con 0x7f8608102b20 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860ea35640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f860810db50 con 0x7f8608105450 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860ea35640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f860810de50 con 0x7f8608107d80 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f8607fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 0x7f860819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f8607fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 0x7f860819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59954/0 (socket says 172.21.15.32:59954) 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f8607fff640 1 -- 172.21.15.32:0/488900682 learned_addr learned my addr 172.21.15.32:0/488900682 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f860cfd2640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8608107d80 0x7f86081a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f86077fe640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608105450 0x7f860819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f8607fff640 1 -- 172.21.15.32:0/488900682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8608107d80 msgr2=0x7f86081a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f8607fff640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8608107d80 0x7f86081a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f8607fff640 1 -- 172.21.15.32:0/488900682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608105450 msgr2=0x7f860819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f8607fff640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608105450 0x7f860819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f8607fff640 1 -- 172.21.15.32:0/488900682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86081a21e0 con 0x7f8608102b20 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.669+0000 7f86077fe640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608105450 0x7f860819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.670+0000 7f8607fff640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 0x7f860819a5c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009230 tx=0x7f85f8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.670+0000 7f86057fa640 1 -- 172.21.15.32:0/488900682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8041070 con 0x7f8608102b20 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.670+0000 7f86057fa640 1 -- 172.21.15.32:0/488900682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f80054e0 con 0x7f8608102b20 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.670+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86081a2470 con 0x7f8608102b20 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.670+0000 7f86057fa640 1 -- 172.21.15.32:0/488900682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8034440 con 0x7f8608102b20 2024-04-16T05:34:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.670+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a2960 con 0x7f8608102b20 2024-04-16T05:34:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.671+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8608109870 con 0x7f8608102b20 2024-04-16T05:34:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.672+0000 7f86057fa640 1 -- 172.21.15.32:0/488900682 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f85f8044070 con 0x7f8608102b20 2024-04-16T05:34:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.673+0000 7f86057fa640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85f00a3bb0 0x7f85f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.673+0000 7f86077fe640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85f00a3bb0 0x7f85f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.673+0000 7f86057fa640 1 -- 172.21.15.32:0/488900682 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f85f80f2530 con 0x7f8608102b20 2024-04-16T05:34:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.673+0000 7f86077fe640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85f00a3bb0 0x7f85f00a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0058c0 tx=0x7f85fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.677+0000 7f86057fa640 1 -- 172.21.15.32:0/488900682 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85f80bebb0 con 0x7f8608102b20 2024-04-16T05:34:46.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.868+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f860802d840 con 0x7f8608102b20 2024-04-16T05:34:46.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.868+0000 7f86057fa640 1 -- 172.21.15.32:0/488900682 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f85f80be4b0 con 0x7f8608102b20 2024-04-16T05:34:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85f00a3bb0 msgr2=0x7f85f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85f00a3bb0 0x7f85f00a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0058c0 tx=0x7f85fc005850 comp rx=0 tx=0).stop 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 msgr2=0x7f860819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 0x7f860819a5c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009230 tx=0x7f85f8005dd0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 shutdown_connections 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f85f00a3bb0 0x7f85f00a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8608107d80 0x7f86081a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608105450 0x7f860819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 --2- 172.21.15.32:0/488900682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608102b20 0x7f860819a5c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 >> 172.21.15.32:0/488900682 conn(0x7f86080fc4b0 msgr2=0x7f86081021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:46.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 shutdown_connections 2024-04-16T05:34:46.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.870+0000 7f860ea35640 1 -- 172.21.15.32:0/488900682 wait complete. 2024-04-16T05:34:46.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:46 smithi032 ceph-mon[32351]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:46.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:46 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4121888411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:46.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:46 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2574399399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:46.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:46 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/434434461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:46.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 -- 172.21.15.32:0/2614899082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 msgr2=0x7f4e3410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 --2- 172.21.15.32:0/2614899082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 0x7f4e3410a490 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009900 tx=0x7f4e2402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 -- 172.21.15.32:0/2614899082 shutdown_connections 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 --2- 172.21.15.32:0/2614899082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 0x7f4e3410a490 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 --2- 172.21.15.32:0/2614899082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e34103240 0x7f4e341036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 --2- 172.21.15.32:0/2614899082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e34102050 0x7f4e34102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 -- 172.21.15.32:0/2614899082 >> 172.21.15.32:0/2614899082 conn(0x7f4e340fd600 msgr2=0x7f4e340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 -- 172.21.15.32:0/2614899082 shutdown_connections 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 -- 172.21.15.32:0/2614899082 wait complete. 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.989+0000 7f4e394c7640 1 Processor -- start 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e394c7640 1 -- start start 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e394c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e34102050 0x7f4e34198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e394c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e34103240 0x7f4e34198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e394c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 0x7f4e3419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e394c7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3410dce0 con 0x7f4e34103c00 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e394c7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3410db60 con 0x7f4e34103240 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e394c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3410de60 con 0x7f4e34102050 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 0x7f4e3419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 0x7f4e3419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59966/0 (socket says 172.21.15.32:59966) 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 -- 172.21.15.32:0/642928021 learned_addr learned my addr 172.21.15.32:0/642928021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e327fc640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e34103240 0x7f4e34198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e32ffd640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e34102050 0x7f4e34198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 -- 172.21.15.32:0/642928021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e34102050 msgr2=0x7f4e34198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e34102050 0x7f4e34198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 -- 172.21.15.32:0/642928021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e34103240 msgr2=0x7f4e34198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e34103240 0x7f4e34198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 -- 172.21.15.32:0/642928021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3419ff20 con 0x7f4e34103c00 2024-04-16T05:34:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e327fc640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e34103240 0x7f4e34198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.990+0000 7f4e337fe640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 0x7f4e3419f8b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e2402ce70 tx=0x7f4e24005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.991+0000 7f4e2bfff640 1 -- 172.21.15.32:0/642928021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24041070 con 0x7f4e34103c00 2024-04-16T05:34:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.991+0000 7f4e2bfff640 1 -- 172.21.15.32:0/642928021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e240054e0 con 0x7f4e34103c00 2024-04-16T05:34:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.991+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e341a01b0 con 0x7f4e34103c00 2024-04-16T05:34:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.991+0000 7f4e2bfff640 1 -- 172.21.15.32:0/642928021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e240344a0 con 0x7f4e34103c00 2024-04-16T05:34:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.991+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e341a06a0 con 0x7f4e34103c00 2024-04-16T05:34:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.992+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e34109370 con 0x7f4e34103c00 2024-04-16T05:34:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.993+0000 7f4e2bfff640 1 -- 172.21.15.32:0/642928021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e24044070 con 0x7f4e34103c00 2024-04-16T05:34:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.993+0000 7f4e2bfff640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e200a3bb0 0x7f4e200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.993+0000 7f4e32ffd640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e200a3bb0 0x7f4e200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.994+0000 7f4e2bfff640 1 -- 172.21.15.32:0/642928021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4e240f2570 con 0x7f4e34103c00 2024-04-16T05:34:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.994+0000 7f4e32ffd640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e200a3bb0 0x7f4e200a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e341030a0 tx=0x7f4e2c009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:46.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:46.998+0000 7f4e2bfff640 1 -- 172.21.15.32:0/642928021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e240bebf0 con 0x7f4e34103c00 2024-04-16T05:34:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:46 smithi134 ceph-mon[36153]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:46 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4121888411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:46 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2574399399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:46 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/434434461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:47.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:46 smithi082 ceph-mon[36195]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T05:34:47.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:46 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4121888411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:47.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:46 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2574399399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:47.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:46 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/434434461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.225+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e34062c00 con 0x7f4e34103c00 2024-04-16T05:34:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.226+0000 7f4e2bfff640 1 -- 172.21.15.32:0/642928021 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11876 (secure 0 0 0) 0x7f4e240be4f0 con 0x7f4e34103c00 2024-04-16T05:34:47.224 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:47.224 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:32:55.688634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24461,24494]},"id":1}]} 2024-04-16T05:34:47.225 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e200a3bb0 msgr2=0x7f4e200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e200a3bb0 0x7f4e200a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e341030a0 tx=0x7f4e2c009870 comp rx=0 tx=0).stop 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 msgr2=0x7f4e3419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 0x7f4e3419f8b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e2402ce70 tx=0x7f4e24005e40 comp rx=0 tx=0).stop 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 shutdown_connections 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f4e200a3bb0 0x7f4e200a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e34103c00 0x7f4e3419f8b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e34103240 0x7f4e34198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 --2- 172.21.15.32:0/642928021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e34102050 0x7f4e34198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 >> 172.21.15.32:0/642928021 conn(0x7f4e340fd600 msgr2=0x7f4e34106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 shutdown_connections 2024-04-16T05:34:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.228+0000 7f4e394c7640 1 -- 172.21.15.32:0/642928021 wait complete. 2024-04-16T05:34:47.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:47.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 -- 172.21.15.32:0/1820938182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea8100f00 msgr2=0x7f5ea8101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 --2- 172.21.15.32:0/1820938182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea8100f00 0x7f5ea8101340 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009900 tx=0x7f5e9802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 -- 172.21.15.32:0/1820938182 shutdown_connections 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 --2- 172.21.15.32:0/1820938182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ea8102a90 0x7f5ea81093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 --2- 172.21.15.32:0/1820938182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea81020d0 0x7f5ea8102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 --2- 172.21.15.32:0/1820938182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea8100f00 0x7f5ea8101340 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 -- 172.21.15.32:0/1820938182 >> 172.21.15.32:0/1820938182 conn(0x7f5ea80fc4e0 msgr2=0x7f5ea80fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 -- 172.21.15.32:0/1820938182 shutdown_connections 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.345+0000 7f5eadd40640 1 -- 172.21.15.32:0/1820938182 wait complete. 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5eadd40640 1 Processor -- start 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5eadd40640 1 -- start start 2024-04-16T05:34:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5eadd40640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100f00 0x7f5ea806b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5eadd40640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea81020d0 0x7f5ea819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5eadd40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ea8102a90 0x7f5ea819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5eadd40640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810cbb0 con 0x7f5ea81020d0 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5eadd40640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810ca30 con 0x7f5ea8102a90 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5eadd40640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810cd30 con 0x7f5ea8100f00 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5ea6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea81020d0 0x7f5ea819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5ea6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea81020d0 0x7f5ea819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59982/0 (socket says 172.21.15.32:59982) 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5ea6ffd640 1 -- 172.21.15.32:0/801587404 learned_addr learned my addr 172.21.15.32:0/801587404 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5ea77fe640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100f00 0x7f5ea806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.346+0000 7f5ea7fff640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ea8102a90 0x7f5ea819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea6ffd640 1 -- 172.21.15.32:0/801587404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100f00 msgr2=0x7f5ea806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea6ffd640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100f00 0x7f5ea806b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea6ffd640 1 -- 172.21.15.32:0/801587404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ea8102a90 msgr2=0x7f5ea819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea6ffd640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ea8102a90 0x7f5ea819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea6ffd640 1 -- 172.21.15.32:0/801587404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea819ffb0 con 0x7f5ea81020d0 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea7fff640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ea8102a90 0x7f5ea819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea6ffd640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea81020d0 0x7f5ea819d080 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000d950 tx=0x7f5e9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea4ff9640 1 -- 172.21.15.32:0/801587404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90017070 con 0x7f5ea81020d0 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea4ff9640 1 -- 172.21.15.32:0/801587404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e90009c30 con 0x7f5ea81020d0 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5ea4ff9640 1 -- 172.21.15.32:0/801587404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000f6b0 con 0x7f5ea81020d0 2024-04-16T05:34:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea81a02a0 con 0x7f5ea81020d0 2024-04-16T05:34:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.347+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea81a07f0 con 0x7f5ea81020d0 2024-04-16T05:34:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.348+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea8108330 con 0x7f5ea81020d0 2024-04-16T05:34:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.349+0000 7f5ea4ff9640 1 -- 172.21.15.32:0/801587404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5e900109d0 con 0x7f5ea81020d0 2024-04-16T05:34:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.350+0000 7f5ea4ff9640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5e940a3bb0 0x7f5e940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.350+0000 7f5ea77fe640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5e940a3bb0 0x7f5e940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.350+0000 7f5ea4ff9640 1 -- 172.21.15.32:0/801587404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5e900d04b0 con 0x7f5ea81020d0 2024-04-16T05:34:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.350+0000 7f5ea77fe640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5e940a3bb0 0x7f5e940a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e9802c7b0 tx=0x7f5e98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.354+0000 7f5ea4ff9640 1 -- 172.21.15.32:0/801587404 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e90009de0 con 0x7f5ea81020d0 2024-04-16T05:34:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.544+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ea8062c00 con 0x7f5ea81020d0 2024-04-16T05:34:47.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.546+0000 7f5ea4ff9640 1 -- 172.21.15.32:0/801587404 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24618 (secure 0 0 0) 0x7f5e9009cad0 con 0x7f5ea81020d0 2024-04-16T05:34:47.545 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:47.545 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:33:34.757328+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:47.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5e940a3bb0 msgr2=0x7f5e940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5e940a3bb0 0x7f5e940a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e9802c7b0 tx=0x7f5e98005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea81020d0 msgr2=0x7f5ea819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea81020d0 0x7f5ea819d080 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000d950 tx=0x7f5e9000de10 comp rx=0 tx=0).stop 2024-04-16T05:34:47.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 shutdown_connections 2024-04-16T05:34:47.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f5e940a3bb0 0x7f5e940a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ea8102a90 0x7f5ea819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea81020d0 0x7f5ea819d080 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 --2- 172.21.15.32:0/801587404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100f00 0x7f5ea806b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 >> 172.21.15.32:0/801587404 conn(0x7f5ea80fc4e0 msgr2=0x7f5ea8105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 shutdown_connections 2024-04-16T05:34:47.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.548+0000 7f5eadd40640 1 -- 172.21.15.32:0/801587404 wait complete. 2024-04-16T05:34:47.559 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-16T05:32:34.109354+0000', 'modified': '2024-04-16T05:32:55.688634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24461, 'mds_2': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/1431936644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 1431936644}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1431936644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/1367430466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 1367430466}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 1367430466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/2091760007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 2091760007}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2091760007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.134:6839/3919403886', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3919403886}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3919403886}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.82:6837/1413733447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 1413733447}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 1413733447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6839/2453097092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2453097092}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2453097092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24460, 24461, 24494]} 2024-04-16T05:34:47.559 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-16T05:34:47.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-16T05:34:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.668+0000 7f20fe416640 1 -- 172.21.15.32:0/3493285815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8101210 msgr2=0x7f20f8101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.668+0000 7f20fe416640 1 --2- 172.21.15.32:0/3493285815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8101210 0x7f20f8101690 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009900 tx=0x7f20e002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 -- 172.21.15.32:0/3493285815 shutdown_connections 2024-04-16T05:34:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 --2- 172.21.15.32:0/3493285815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8101bd0 0x7f20f8109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 --2- 172.21.15.32:0/3493285815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8101210 0x7f20f8101690 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 --2- 172.21.15.32:0/3493285815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f81024e0 0x7f20f8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 -- 172.21.15.32:0/3493285815 >> 172.21.15.32:0/3493285815 conn(0x7f20f80fbc40 msgr2=0x7f20f80fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 -- 172.21.15.32:0/3493285815 shutdown_connections 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 -- 172.21.15.32:0/3493285815 wait complete. 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 Processor -- start 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 -- start start 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8101210 0x7f20f8105130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8101bd0 0x7f20f8103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f81024e0 0x7f20f8103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810cbc0 con 0x7f20f81024e0 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810ca40 con 0x7f20f8101210 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fe416640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810cd40 con 0x7f20f8101bd0 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fc9b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f81024e0 0x7f20f8103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fc9b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f81024e0 0x7f20f8103cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60000/0 (socket says 172.21.15.32:60000) 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.669+0000 7f20fc9b3640 1 -- 172.21.15.32:0/919841136 learned_addr learned my addr 172.21.15.32:0/919841136 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20f7fff640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8101210 0x7f20f8105130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20effff640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8101bd0 0x7f20f8103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20fc9b3640 1 -- 172.21.15.32:0/919841136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8101bd0 msgr2=0x7f20f8103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20fc9b3640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8101bd0 0x7f20f8103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20fc9b3640 1 -- 172.21.15.32:0/919841136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8101210 msgr2=0x7f20f8105130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20fc9b3640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8101210 0x7f20f8105130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20fc9b3640 1 -- 172.21.15.32:0/919841136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f8104540 con 0x7f20f81024e0 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20fc9b3640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f81024e0 0x7f20f8103cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009fd0 tx=0x7f20f000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20f5ffb640 1 -- 172.21.15.32:0/919841136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0017070 con 0x7f20f81024e0 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20f5ffb640 1 -- 172.21.15.32:0/919841136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20f0003e50 con 0x7f20f81024e0 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20fe416640 1 -- 172.21.15.32:0/919841136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f81a24e0 con 0x7f20f81024e0 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20f5ffb640 1 -- 172.21.15.32:0/919841136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f00106c0 con 0x7f20f81024e0 2024-04-16T05:34:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.670+0000 7f20fe416640 1 -- 172.21.15.32:0/919841136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81a29b0 con 0x7f20f81024e0 2024-04-16T05:34:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.672+0000 7f20f5ffb640 1 -- 172.21.15.32:0/919841136 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f20f0003970 con 0x7f20f81024e0 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f2100ffd740 1 Processor -- start 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f2100ffd740 1 -- start start 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6bd60 0x5574fec6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6c750 0x5574fec71250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fec73d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f2100ffd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5574feb7f240 con 0x5574fec6c750 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f2100ffd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5574febdc9f0 con 0x5574fec718c0 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f2100ffd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5574feb979e0 con 0x5574fec6bd60 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6c750 0x5574fec71250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6c750 0x5574fec71250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60010/0 (socket says 172.21.15.32:60010) 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f7fff640 1 -- 172.21.15.32:0/3444830869 learned_addr learned my addr 172.21.15.32:0/3444830869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20effff640 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6bd60 0x5574fec6c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20effff640 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fec73d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f5ffb640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f20e40a3bb0 0x7f20e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f7fff640 1 -- 172.21.15.32:0/3444830869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6bd60 msgr2=0x5574fec6c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f7fff640 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6bd60 0x5574fec6c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f7fff640 1 -- 172.21.15.32:0/3444830869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 msgr2=0x5574fec73d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f7fff640 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fec73d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20f7fff640 1 -- 172.21.15.32:0/3444830869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574feb8abc0 con 0x5574fec6c750 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20effff640 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fec73d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.673+0000 7f20fc9b3640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f20e40a3bb0 0x7f20e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.674+0000 7f20f5ffb640 1 -- 172.21.15.32:0/919841136 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f20f00d04d0 con 0x7f20f81024e0 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.674+0000 7f20f7fff640 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6c750 0x5574fec71250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009c20 tx=0x7f20e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555e86008be65bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.674+0000 7f20ee7fc640 1 -- 172.21.15.32:0/3444830869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8004030 con 0x5574fec6c750 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.674+0000 7f20ee7fc640 1 -- 172.21.15.32:0/3444830869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e80041d0 con 0x5574fec6c750 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.674+0000 7f20ee7fc640 1 -- 172.21.15.32:0/3444830869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80197c0 con 0x5574fec6c750 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.674+0000 7f2100ffd740 1 -- 172.21.15.32:0/3444830869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6c750 msgr2=0x5574fec71250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.674+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6c750 0x5574fec71250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009c20 tx=0x7f20e8011810 comp rx=0 tx=0).stop 2024-04-16T05:34:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.674+0000 7f20fc9b3640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f20e40a3bb0 0x7f20e40a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f20f0018860 tx=0x7f20f006ae50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 -- 172.21.15.32:0/3444830869 shutdown_connections 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fec73d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6c750 0x5574fec71250 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3444830869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6bd60 0x5574fec6c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 -- 172.21.15.32:0/3444830869 >> 172.21.15.32:0/3444830869 conn(0x5574fec672d0 msgr2=0x5574fec676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 -- 172.21.15.32:0/3444830869 shutdown_connections 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 -- 172.21.15.32:0/3444830869 wait complete. 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 -- start start 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 Processor -- start 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec75f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec6c750 0x5574fec76490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec718c0 0x5574fec7d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5574febd60f0 con 0x5574fec6bd60 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5574febd0dd0 con 0x5574fec6c750 2024-04-16T05:34:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5574febdecd0 con 0x5574fec718c0 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.675+0000 7f2100ffd740 10 client.0 _ll_register_callbacks cb 0x5574febc7570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec75f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec75f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60016/0 (socket says 172.21.15.32:60016) 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 -- 172.21.15.32:0/3459332959 learned_addr learned my addr 172.21.15.32:0/3459332959 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20f7fff640 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec6c750 0x5574fec76490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec718c0 0x5574fec7d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 -- 172.21.15.32:0/3459332959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec718c0 msgr2=0x5574fec7d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec718c0 0x5574fec7d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 -- 172.21.15.32:0/3459332959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec6c750 msgr2=0x5574fec76490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec6c750 0x5574fec76490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 -- 172.21.15.32:0/3459332959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fe8e42d0 con 0x5574fec6bd60 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20f7fff640 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec6c750 0x5574fec76490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20effff640 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec75f50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009900 tx=0x7f20e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f87cfc95c97f7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20d77fe640 1 -- 172.21.15.32:0/3459332959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0048020 con 0x5574fec6bd60 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20d77fe640 1 -- 172.21.15.32:0/3459332959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f20e0005250 con 0x5574fec6bd60 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f2100ffd740 1 -- 172.21.15.32:0/3459332959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e80041d0 con 0x5574fec6bd60 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f20d77fe640 1 -- 172.21.15.32:0/3459332959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f20e0053050 con 0x5574fec6bd60 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f2100ffd740 10 client.15363 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.676+0000 7f2100ffd740 1 -- 172.21.15.32:0/3459332959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5574fe9ae6b0 con 0x5574fec6bd60 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f20d77fe640 1 -- 172.21.15.32:0/3459332959 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e80041d0 con 0x5574fec6bd60 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f20d77fe640 1 -- 172.21.15.32:0/3459332959 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f20e0003850 con 0x5574fec6bd60 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f2100ffd740 10 client.15363 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f2100ffd740 10 client.15363 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f2100ffd740 20 client.15363 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5574fec82c50 0x5574fec840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f2100ffd740 4 client.15363 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f2100ffd740 1 -- 172.21.15.32:0/3459332959 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574feb584c0 con 0x5574fec82c50 2024-04-16T05:34:47.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.677+0000 7f20f7fff640 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5574fec82c50 0x5574fec840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.678+0000 7f20d77fe640 10 client.15363 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.678+0000 7f20f7fff640 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5574fec82c50 0x5574fec840c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b742b0b619bb1a3 server_cookie=b7a3c88c64faa7ac in_seq=0 out_seq=0 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.678+0000 7f20d77fe640 1 -- 172.21.15.32:0/3459332959 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574feb584c0 con 0x5574fec82c50 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.678+0000 7f20d77fe640 10 client.15363 handle_command_reply: tid=0 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.678+0000 7f2100ffd740 1 client.15363 shutdown 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 -- 172.21.15.32:0/3459332959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 msgr2=0x5574fec75f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec75f50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009900 tx=0x7f20e0009260 comp rx=0 tx=0).stop 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 -- 172.21.15.32:0/3459332959 shutdown_connections 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5574fec82c50 0x5574fec840c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec718c0 0x5574fec7d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec6c750 0x5574fec76490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 --2- 172.21.15.32:0/3459332959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec75f50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f20d77fe640 0 client.15363 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 -- 172.21.15.32:0/3459332959 >> 172.21.15.32:0/3459332959 conn(0x5574fec672d0 msgr2=0x5574fec66410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 -- 172.21.15.32:0/3459332959 shutdown_connections 2024-04-16T05:34:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 1 -- 172.21.15.32:0/3459332959 wait complete. 2024-04-16T05:34:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.679+0000 7f2100ffd740 20 client.15363 trim_cache size 0 max 16384 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f2100ffd740 1 Processor -- start 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f2100ffd740 1 -- start start 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 0x5574fec7d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fec73cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f2100ffd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5574fec82400 con 0x5574fec6bd60 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f2100ffd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5574fec82ad0 con 0x5574fec718c0 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f2100ffd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5574fec831a0 con 0x5574fec6c750 2024-04-16T05:34:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f20effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f20effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec763f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60030/0 (socket says 172.21.15.32:60030) 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f20effff640 1 -- 172.21.15.32:0/1045004548 learned_addr learned my addr 172.21.15.32:0/1045004548 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f20effff640 1 -- 172.21.15.32:0/1045004548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 msgr2=0x5574fec7d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f20effff640 1 --2- 172.21.15.32:0/1045004548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 0x5574fec7d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.697+0000 7f20effff640 1 -- 172.21.15.32:0/1045004548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 msgr2=0x5574fec73cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.698+0000 7f20effff640 1 --2- 172.21.15.32:0/1045004548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fec73cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.698+0000 7f20effff640 1 -- 172.21.15.32:0/1045004548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fec68040 con 0x5574fec6bd60 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.698+0000 7f20effff640 1 --2- 172.21.15.32:0/1045004548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec763f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f20e0001eb0 tx=0x7f20e002cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c2992a4aa2624d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.698+0000 7f20edffb640 1 -- 172.21.15.32:0/1045004548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0048070 con 0x5574fec6bd60 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.698+0000 7f20edffb640 1 -- 172.21.15.32:0/1045004548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e0042440 con 0x5574fec6bd60 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.698+0000 7f20edffb640 1 -- 172.21.15.32:0/1045004548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00042c0 con 0x5574fec6bd60 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- 172.21.15.32:0/1045004548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 msgr2=0x5574fec763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 --2- 172.21.15.32:0/1045004548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec763f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f20e0001eb0 tx=0x7f20e002cd70 comp rx=0 tx=0).stop 2024-04-16T05:34:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- 172.21.15.32:0/1045004548 shutdown_connections 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 --2- 172.21.15.32:0/1045004548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fec73cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 --2- 172.21.15.32:0/1045004548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 0x5574fec7d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 --2- 172.21.15.32:0/1045004548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec763f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- 172.21.15.32:0/1045004548 >> 172.21.15.32:0/1045004548 conn(0x5574fec672d0 msgr2=0x5574fec676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- 172.21.15.32:0/1045004548 shutdown_connections 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- 172.21.15.32:0/1045004548 wait complete. 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- start start 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 Processor -- start 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec70620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 0x5574fec7d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fece6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5574fec83f40 con 0x5574fec6bd60 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5574fec83870 con 0x5574fec718c0 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5574fec84610 con 0x5574fec6c750 2024-04-16T05:34:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f2100ffd740 10 client.0 _ll_register_callbacks cb 0x7f20e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.699+0000 7f20effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec70620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec70620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60046/0 (socket says 172.21.15.32:60046) 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 -- 172.21.15.32:0/2707274608 learned_addr learned my addr 172.21.15.32:0/2707274608 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20f7fff640 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 0x5574fec7d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fece6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 -- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 msgr2=0x5574fec7d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 0x5574fec7d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 -- 172.21.15.32:0/2707274608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 msgr2=0x5574fece6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fece6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 -- 172.21.15.32:0/2707274608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fece7440 con 0x5574fec6bd60 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20f7fff640 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 0x5574fec7d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20effff640 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec70620 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f20e0003d40 tx=0x7f20e002ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e321aa3a61da9de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.700+0000 7f20d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f20d77fe640 1 -- 172.21.15.32:0/2707274608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0048070 con 0x5574fec6bd60 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f20d77fe640 1 -- 172.21.15.32:0/2707274608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f20e00425f0 con 0x5574fec6bd60 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f2100ffd740 1 -- 172.21.15.32:0/2707274608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e0042440 con 0x5574fec6bd60 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f20d77fe640 1 -- 172.21.15.32:0/2707274608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f20e00427f0 con 0x5574fec6bd60 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f2100ffd740 10 client.15369 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f2100ffd740 1 -- 172.21.15.32:0/2707274608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5574fece8740 con 0x5574fec6bd60 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f20d77fe640 1 -- 172.21.15.32:0/2707274608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e0042440 con 0x5574fec6bd60 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f20d77fe640 1 -- 172.21.15.32:0/2707274608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f20e0058bf0 con 0x5574fec6bd60 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f2100ffd740 10 client.15369 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f2100ffd740 10 client.15369 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f2100ffd740 20 client.15369 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f2100ffd740 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5574fecece50 0x5574fecebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.701+0000 7f2100ffd740 4 client.15369 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.702+0000 7f2100ffd740 1 -- 172.21.15.32:0/2707274608 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "status"}) -- 0x5574febdf4e0 con 0x5574fecece50 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.702+0000 7f20f7fff640 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5574fecece50 0x5574fecebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.702+0000 7f20d77fe640 10 client.15369 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.702+0000 7f20f7fff640 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5574fecece50 0x5574fecebb60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecb712961b71e64e server_cookie=ea3f30da0c47b12 in_seq=0 out_seq=0 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.702+0000 7f20d77fe640 1 -- 172.21.15.32:0/2707274608 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5574febdf4e0 con 0x5574fecece50 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.702+0000 7f20d77fe640 10 client.15369 handle_command_reply: tid=0 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.702+0000 7f2100ffd740 1 client.15369 shutdown 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 -- 172.21.15.32:0/2707274608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 msgr2=0x5574fec70620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec70620 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f20e0003d40 tx=0x7f20e002ce80 comp rx=0 tx=0).stop 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 -- 172.21.15.32:0/2707274608 shutdown_connections 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5574fecece50 0x5574fecebb60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574fec718c0 0x5574fece6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574fec6c750 0x5574fec7d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20d77fe640 0 client.15369 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 --2- 172.21.15.32:0/2707274608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574fec6bd60 0x5574fec70620 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 -- 172.21.15.32:0/2707274608 >> 172.21.15.32:0/2707274608 conn(0x5574fec672d0 msgr2=0x5574fec81ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 -- 172.21.15.32:0/2707274608 shutdown_connections 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 1 -- 172.21.15.32:0/2707274608 wait complete. 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f2100ffd740 20 client.15369 trim_cache size 0 max 16384 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2024-04-16T05:34:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24460, 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 113.022796755, 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 24, 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 91, 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 145.910806686 2024-04-16T05:34:47.708 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 -- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f20e40a3bb0 msgr2=0x7f20e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f20e40a3bb0 0x7f20e40a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f20f0018860 tx=0x7f20f006ae50 comp rx=0 tx=0).stop 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 -- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f81024e0 msgr2=0x7f20f8103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f81024e0 0x7f20f8103cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009fd0 tx=0x7f20f000b260 comp rx=0 tx=0).stop 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 -- 172.21.15.32:0/919841136 shutdown_connections 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f20e40a3bb0 0x7f20e40a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f81024e0 0x7f20f8103cb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8101bd0 0x7f20f8103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 --2- 172.21.15.32:0/919841136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8101210 0x7f20f8105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.703+0000 7f20fe416640 1 -- 172.21.15.32:0/919841136 >> 172.21.15.32:0/919841136 conn(0x7f20f80fbc40 msgr2=0x7f20f8100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.704+0000 7f20fe416640 1 -- 172.21.15.32:0/919841136 shutdown_connections 2024-04-16T05:34:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.704+0000 7f20fe416640 1 -- 172.21.15.32:0/919841136 wait complete. 2024-04-16T05:34:47.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-16T05:34:47.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.820+0000 7f31939f2640 1 -- 172.21.15.32:0/2112426295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c1008b0 msgr2=0x7f318c100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.820+0000 7f31939f2640 1 --2- 172.21.15.32:0/2112426295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c1008b0 0x7f318c100cd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f317c009ca0 tx=0x7f317c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 -- 172.21.15.32:0/2112426295 shutdown_connections 2024-04-16T05:34:47.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 --2- 172.21.15.32:0/2112426295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c101bb0 0x7f318c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 --2- 172.21.15.32:0/2112426295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c101210 0x7f318c101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 --2- 172.21.15.32:0/2112426295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c1008b0 0x7f318c100cd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 -- 172.21.15.32:0/2112426295 >> 172.21.15.32:0/2112426295 conn(0x7f318c0fc4b0 msgr2=0x7f318c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 -- 172.21.15.32:0/2112426295 shutdown_connections 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 -- 172.21.15.32:0/2112426295 wait complete. 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 Processor -- start 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 -- start start 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c1008b0 0x7f318c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c101210 0x7f318c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c101bb0 0x7f318c1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f318c10fee0 con 0x7f318c101bb0 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f318c10fd60 con 0x7f318c101210 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f31939f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f318c110060 con 0x7f318c1008b0 2024-04-16T05:34:47.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.821+0000 7f3191f8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c101bb0 0x7f318c1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3191f8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c101bb0 0x7f318c1a1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60076/0 (socket says 172.21.15.32:60076) 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3191f8f640 1 -- 172.21.15.32:0/3630712146 learned_addr learned my addr 172.21.15.32:0/3630712146 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3190f8d640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c101210 0x7f318c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f319178e640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c1008b0 0x7f318c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3191f8f640 1 -- 172.21.15.32:0/3630712146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c1008b0 msgr2=0x7f318c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3191f8f640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c1008b0 0x7f318c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3191f8f640 1 -- 172.21.15.32:0/3630712146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c101210 msgr2=0x7f318c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3191f8f640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c101210 0x7f318c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3191f8f640 1 -- 172.21.15.32:0/3630712146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c1a21a0 con 0x7f318c101bb0 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f319178e640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c1008b0 0x7f318c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f3191f8f640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c101bb0 0x7f318c1a1b30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f318400c970 tx=0x7f318400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f318a7fc640 1 -- 172.21.15.32:0/3630712146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184007bb0 con 0x7f318c101bb0 2024-04-16T05:34:47.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f31939f2640 1 -- 172.21.15.32:0/3630712146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c1a2490 con 0x7f318c101bb0 2024-04-16T05:34:47.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f318a7fc640 1 -- 172.21.15.32:0/3630712146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3184007d50 con 0x7f318c101bb0 2024-04-16T05:34:47.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f318a7fc640 1 -- 172.21.15.32:0/3630712146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318400f450 con 0x7f318c101bb0 2024-04-16T05:34:47.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.822+0000 7f31939f2640 1 -- 172.21.15.32:0/3630712146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c1a2990 con 0x7f318c101bb0 2024-04-16T05:34:47.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.824+0000 7f318a7fc640 1 -- 172.21.15.32:0/3630712146 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3184004f60 con 0x7f318c101bb0 2024-04-16T05:34:47.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f31965da740 1 Processor -- start 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f31965da740 1 -- start start 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f31965da740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f31965da740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f31965da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e68c0 0x56259f8e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f31965da740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56259f7f4240 con 0x56259f8e0d60 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f31965da740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56259f8519f0 con 0x56259f8e1750 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f31965da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56259f80c9e0 con 0x56259f8e68c0 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f3190f8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f3190f8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60090/0 (socket says 172.21.15.32:60090) 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f3190f8d640 1 -- 172.21.15.32:0/4148126840 learned_addr learned my addr 172.21.15.32:0/4148126840 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f318a7fc640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31740a3bb0 0x7f31740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f319178e640 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f3191f8f640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31740a3bb0 0x7f31740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f3190f8d640 1 -- 172.21.15.32:0/4148126840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e68c0 msgr2=0x56259f8e8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f3190f8d640 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e68c0 0x56259f8e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.825+0000 7f3190f8d640 1 -- 172.21.15.32:0/4148126840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 msgr2=0x56259f8e6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f3190f8d640 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f3190f8d640 1 -- 172.21.15.32:0/4148126840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56259f7ffbc0 con 0x56259f8e0d60 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f319178e640 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f318a7fc640 1 -- 172.21.15.32:0/3630712146 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f31840d0530 con 0x7f318c101bb0 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f3190f8d640 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8e1180 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3180009d40 tx=0x7f3180013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7201abc04f60e455 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f31737fe640 1 -- 172.21.15.32:0/4148126840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318001c470 con 0x56259f8e0d60 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f3191f8f640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31740a3bb0 0x7f31740a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f318401e000 tx=0x7f3184096e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f31737fe640 1 -- 172.21.15.32:0/4148126840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3180017070 con 0x56259f8e0d60 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.826+0000 7f31737fe640 1 -- 172.21.15.32:0/4148126840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318000f6b0 con 0x56259f8e0d60 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- 172.21.15.32:0/4148126840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 msgr2=0x56259f8e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8e1180 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3180009d40 tx=0x7f3180013830 comp rx=0 tx=0).stop 2024-04-16T05:34:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- 172.21.15.32:0/4148126840 shutdown_connections 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e68c0 0x56259f8e8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 --2- 172.21.15.32:0/4148126840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8e1180 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- 172.21.15.32:0/4148126840 >> 172.21.15.32:0/4148126840 conn(0x56259f8dc2d0 msgr2=0x56259f8dc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- 172.21.15.32:0/4148126840 shutdown_connections 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- 172.21.15.32:0/4148126840 wait complete. 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- start start 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 Processor -- start 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8eb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e68c0 0x56259f8f2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56259f84b0f0 con 0x56259f8e0d60 2024-04-16T05:34:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56259f845dd0 con 0x56259f8e1750 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.827+0000 7f31965da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56259f853cd0 con 0x56259f8e68c0 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f31965da740 10 client.0 _ll_register_callbacks cb 0x56259f83c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8eafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60104/0 (socket says 172.21.15.32:60104) 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 -- 172.21.15.32:0/3202198457 learned_addr learned my addr 172.21.15.32:0/3202198457 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f319178e640 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 -- 172.21.15.32:0/3202198457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e68c0 msgr2=0x56259f8f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e68c0 0x56259f8f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 -- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 msgr2=0x56259f8eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8eb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 -- 172.21.15.32:0/3202198457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56259f5592d0 con 0x56259f8e0d60 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f319178e640 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3190f8d640 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8eafc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3180006010 tx=0x7f3180009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618665c276ed5c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3170ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3170ff9640 1 -- 172.21.15.32:0/3202198457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180004220 con 0x56259f8e0d60 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3170ff9640 1 -- 172.21.15.32:0/3202198457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f31800043c0 con 0x56259f8e0d60 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f31965da740 1 -- 172.21.15.32:0/3202198457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3180017070 con 0x56259f8e0d60 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f3170ff9640 1 -- 172.21.15.32:0/3202198457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3180005020 con 0x56259f8e0d60 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f31965da740 10 client.15381 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.828+0000 7f31965da740 1 -- 172.21.15.32:0/3202198457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56259f6236b0 con 0x56259f8e0d60 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f3170ff9640 1 -- 172.21.15.32:0/3202198457 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3180017070 con 0x56259f8e0d60 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f3170ff9640 1 -- 172.21.15.32:0/3202198457 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f31800035f0 con 0x56259f8e0d60 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f31965da740 10 client.15381 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f31965da740 10 client.15381 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f31965da740 20 client.15381 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f31965da740 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56259f8f78f0 0x56259f8f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f31965da740 4 client.15381 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f31965da740 1 -- 172.21.15.32:0/3202198457 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56259f7cd4c0 con 0x56259f8f78f0 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.829+0000 7f319178e640 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56259f8f78f0 0x56259f8f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f319178e640 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56259f8f78f0 0x56259f8f8d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f1c50f0bb9b7cb3 server_cookie=3fa17a5006455157 in_seq=0 out_seq=0 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f3170ff9640 10 client.15381 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f3170ff9640 1 -- 172.21.15.32:0/3202198457 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56259f7cd4c0 con 0x56259f8f78f0 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f3170ff9640 10 client.15381 handle_command_reply: tid=0 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 client.15381 shutdown 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 -- 172.21.15.32:0/3202198457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 msgr2=0x56259f8eafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8eafc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3180006010 tx=0x7f3180009b30 comp rx=0 tx=0).stop 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 -- 172.21.15.32:0/3202198457 shutdown_connections 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56259f8f78f0 0x56259f8f8d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e68c0 0x56259f8f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f3170ff9640 0 client.15381 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 --2- 172.21.15.32:0/3202198457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e0d60 0x56259f8eafc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.830+0000 7f31965da740 1 -- 172.21.15.32:0/3202198457 >> 172.21.15.32:0/3202198457 conn(0x56259f8dc2d0 msgr2=0x56259f8db410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.831+0000 7f31965da740 1 -- 172.21.15.32:0/3202198457 shutdown_connections 2024-04-16T05:34:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.831+0000 7f31965da740 1 -- 172.21.15.32:0/3202198457 wait complete. 2024-04-16T05:34:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.831+0000 7f31965da740 20 client.15381 trim_cache size 0 max 16384 2024-04-16T05:34:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f31965da740 1 Processor -- start 2024-04-16T05:34:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f31965da740 1 -- start start 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f31965da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 0x56259f8dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f31965da740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e1750 0x56259f8e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f31965da740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e68c0 0x56259f8e3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f31965da740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56259f8f70a0 con 0x56259f8e1750 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f31965da740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56259f8f7770 con 0x56259f8e68c0 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f31965da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56259f8f7e40 con 0x56259f8e0d60 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f319178e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e1750 0x56259f8e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f319178e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e1750 0x56259f8e2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60108/0 (socket says 172.21.15.32:60108) 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f319178e640 1 -- 172.21.15.32:0/3842021502 learned_addr learned my addr 172.21.15.32:0/3842021502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f3190f8d640 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e68c0 0x56259f8e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.849+0000 7f3190f8d640 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 0x56259f8dc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f319178e640 1 -- 172.21.15.32:0/3842021502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 msgr2=0x56259f8dc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f319178e640 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 0x56259f8dc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f319178e640 1 -- 172.21.15.32:0/3842021502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e68c0 msgr2=0x56259f8e3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f319178e640 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e68c0 0x56259f8e3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f319178e640 1 -- 172.21.15.32:0/3842021502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56259f8dd040 con 0x56259f8e1750 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f3190f8d640 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e68c0 0x56259f8e3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f319178e640 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e1750 0x56259f8e2cf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f317c000f80 tx=0x7f317c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68ae3f116cda338 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f3172ffd640 1 -- 172.21.15.32:0/3842021502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c039040 con 0x56259f8e1750 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f3172ffd640 1 -- 172.21.15.32:0/3842021502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f317c049030 con 0x56259f8e1750 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.850+0000 7f3172ffd640 1 -- 172.21.15.32:0/3842021502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c0454d0 con 0x56259f8e1750 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- 172.21.15.32:0/3842021502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e1750 msgr2=0x56259f8e2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e1750 0x56259f8e2cf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f317c000f80 tx=0x7f317c009800 comp rx=0 tx=0).stop 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- 172.21.15.32:0/3842021502 shutdown_connections 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e68c0 0x56259f8e3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e1750 0x56259f8e2cf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 --2- 172.21.15.32:0/3842021502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 0x56259f8dc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- 172.21.15.32:0/3842021502 >> 172.21.15.32:0/3842021502 conn(0x56259f8dc2d0 msgr2=0x56259f8e21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- 172.21.15.32:0/3842021502 shutdown_connections 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- 172.21.15.32:0/3842021502 wait complete. 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- start start 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 Processor -- start 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 0x56259f8e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e68c0 0x56259f8f0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56259f8f8be0 con 0x56259f8e68c0 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56259f8f8510 con 0x56259f8e1750 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56259f8f92b0 con 0x56259f8e0d60 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.851+0000 7f31965da740 10 client.0 _ll_register_callbacks cb 0x7f3180009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e68c0 0x56259f8f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e68c0 0x56259f8f0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60114/0 (socket says 172.21.15.32:60114) 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 -- 172.21.15.32:0/937494915 learned_addr learned my addr 172.21.15.32:0/937494915 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 0x56259f8e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f319178e640 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 -- 172.21.15.32:0/937494915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 msgr2=0x56259f8e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 0x56259f8e48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 -- 172.21.15.32:0/937494915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 msgr2=0x56259f8e4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 -- 172.21.15.32:0/937494915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56259f8f0c10 con 0x56259f8e68c0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3190f8d640 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e68c0 0x56259f8f0620 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f318000a010 tx=0x7f3180026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7502c2fabc948dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3170ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3170ff9640 1 -- 172.21.15.32:0/937494915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31800055d0 con 0x56259f8e68c0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f31965da740 1 -- 172.21.15.32:0/937494915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f317c049030 con 0x56259f8e68c0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3170ff9640 1 -- 172.21.15.32:0/937494915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3180004820 con 0x56259f8e68c0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3170ff9640 1 -- 172.21.15.32:0/937494915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3180004a20 con 0x56259f8e68c0 2024-04-16T05:34:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f31965da740 10 client.15387 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f31965da740 1 -- 172.21.15.32:0/937494915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56259f95d8a0 con 0x56259f8e68c0 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.852+0000 7f3170ff9640 1 -- 172.21.15.32:0/937494915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f317c049030 con 0x56259f8e68c0 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f3170ff9640 1 -- 172.21.15.32:0/937494915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3180030040 con 0x56259f8e68c0 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f31965da740 10 client.15387 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f31965da740 10 client.15387 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f31965da740 20 client.15387 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f31965da740 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56259f9623f0 0x56259f961100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f31965da740 4 client.15387 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f31965da740 1 -- 172.21.15.32:0/937494915 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "status"}) -- 0x56259f8544e0 con 0x56259f9623f0 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f319178e640 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56259f9623f0 0x56259f961100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f319178e640 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56259f9623f0 0x56259f961100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60850a76fad15500 server_cookie=9e665b97aa77aff6 in_seq=0 out_seq=0 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.853+0000 7f3170ff9640 10 client.15387 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f3170ff9640 1 -- 172.21.15.32:0/937494915 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56259f8544e0 con 0x56259f9623f0 2024-04-16T05:34:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f3170ff9640 10 client.15387 handle_command_reply: tid=0 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 client.15387 shutdown 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 -- 172.21.15.32:0/937494915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e68c0 msgr2=0x56259f8f0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e68c0 0x56259f8f0620 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f318000a010 tx=0x7f3180026040 comp rx=0 tx=0).stop 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 -- 172.21.15.32:0/937494915 shutdown_connections 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56259f9623f0 0x56259f961100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56259f8e68c0 0x56259f8f0620 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56259f8e1750 0x56259f8e4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 --2- 172.21.15.32:0/937494915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56259f8e0d60 0x56259f8e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f3170ff9640 0 client.15387 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 -- 172.21.15.32:0/937494915 >> 172.21.15.32:0/937494915 conn(0x56259f8dc2d0 msgr2=0x56259f8dc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 -- 172.21.15.32:0/937494915 shutdown_connections 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 1 -- 172.21.15.32:0/937494915 wait complete. 2024-04-16T05:34:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31965da740 20 client.15387 trim_cache size 0 max 16384 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24461, 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 114.18055472, 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 24, 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 91, 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 148.87887165699999 2024-04-16T05:34:47.860 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:34:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31939f2640 1 -- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31740a3bb0 msgr2=0x7f31740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31939f2640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31740a3bb0 0x7f31740a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f318401e000 tx=0x7f3184096e00 comp rx=0 tx=0).stop 2024-04-16T05:34:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31939f2640 1 -- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c101bb0 msgr2=0x7f318c1a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.854+0000 7f31939f2640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c101bb0 0x7f318c1a1b30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f318400c970 tx=0x7f318400ce30 comp rx=0 tx=0).stop 2024-04-16T05:34:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.855+0000 7f31939f2640 1 -- 172.21.15.32:0/3630712146 shutdown_connections 2024-04-16T05:34:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.855+0000 7f31939f2640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31740a3bb0 0x7f31740a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.855+0000 7f31939f2640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c101bb0 0x7f318c1a1b30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.855+0000 7f31939f2640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c101210 0x7f318c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.855+0000 7f31939f2640 1 --2- 172.21.15.32:0/3630712146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c1008b0 0x7f318c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.855+0000 7f31939f2640 1 -- 172.21.15.32:0/3630712146 >> 172.21.15.32:0/3630712146 conn(0x7f318c0fc4b0 msgr2=0x7f318c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.855+0000 7f31939f2640 1 -- 172.21.15.32:0/3630712146 shutdown_connections 2024-04-16T05:34:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.855+0000 7f31939f2640 1 -- 172.21.15.32:0/3630712146 wait complete. 2024-04-16T05:34:47.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:47 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/488900682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:47.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:47 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/642928021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:47.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:47 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/801587404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:47.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T05:34:47.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.972+0000 7f31229f2640 1 -- 172.21.15.32:0/3582376901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 msgr2=0x7f311c10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.972+0000 7f31229f2640 1 --2- 172.21.15.32:0/3582376901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 0x7f311c10a190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3114009ca0 tx=0x7f311402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 -- 172.21.15.32:0/3582376901 shutdown_connections 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 --2- 172.21.15.32:0/3582376901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 0x7f311c10a190 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 --2- 172.21.15.32:0/3582376901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f311c105470 0x7f311c107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 --2- 172.21.15.32:0/3582376901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f311c102b40 0x7f311c104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 -- 172.21.15.32:0/3582376901 >> 172.21.15.32:0/3582376901 conn(0x7f311c0fc4b0 msgr2=0x7f311c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 -- 172.21.15.32:0/3582376901 shutdown_connections 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 -- 172.21.15.32:0/3582376901 wait complete. 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 Processor -- start 2024-04-16T05:34:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.973+0000 7f31229f2640 1 -- start start 2024-04-16T05:34:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31229f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f311c102b40 0x7f311c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31229f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f311c105470 0x7f311c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31229f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 0x7f311c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31229f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f311c10dcd0 con 0x7f311c107da0 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31229f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f311c10db50 con 0x7f311c105470 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31229f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f311c10de50 con 0x7f311c102b40 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 0x7f311c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 0x7f311c19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60124/0 (socket says 172.21.15.32:60124) 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 -- 172.21.15.32:0/413371504 learned_addr learned my addr 172.21.15.32:0/413371504 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f311bfff640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f311c102b40 0x7f311c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f311b7fe640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f311c105470 0x7f311c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 -- 172.21.15.32:0/413371504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f311c102b40 msgr2=0x7f311c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f311c102b40 0x7f311c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 -- 172.21.15.32:0/413371504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f311c105470 msgr2=0x7f311c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f311c105470 0x7f311c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 -- 172.21.15.32:0/413371504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311c19ff10 con 0x7f311c107da0 2024-04-16T05:34:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f3120f8f640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 0x7f311c19f8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3114037040 tx=0x7f3114032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31197fa640 1 -- 172.21.15.32:0/413371504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114041020 con 0x7f311c107da0 2024-04-16T05:34:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31197fa640 1 -- 172.21.15.32:0/413371504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3114034d90 con 0x7f311c107da0 2024-04-16T05:34:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31229f2640 1 -- 172.21.15.32:0/413371504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311c1a01a0 con 0x7f311c107da0 2024-04-16T05:34:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31197fa640 1 -- 172.21.15.32:0/413371504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114039040 con 0x7f311c107da0 2024-04-16T05:34:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.974+0000 7f31229f2640 1 -- 172.21.15.32:0/413371504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311c1a0690 con 0x7f311c107da0 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31197fa640 1 -- 172.21.15.32:0/413371504 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3114044020 con 0x7f311c107da0 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31255dc740 1 Processor -- start 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31255dc740 1 -- start start 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31255dc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519170 0x55bc31519590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31255dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519b60 0x55bc3151e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31255dc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc31521160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31255dc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bc31428240 con 0x55bc31519170 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31255dc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bc314859f0 con 0x55bc3151ecd0 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31255dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc314409e0 con 0x55bc31519b60 2024-04-16T05:34:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519170 0x55bc31519590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f31197fa640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31040a3bb0 0x7f31040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519170 0x55bc31519590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60136/0 (socket says 172.21.15.32:60136) 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f311b7fe640 1 -- 172.21.15.32:0/2649096895 learned_addr learned my addr 172.21.15.32:0/2649096895 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.977+0000 7f3120f8f640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31040a3bb0 0x7f31040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2649096895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc31521160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f31197fa640 1 -- 172.21.15.32:0/413371504 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f31140f26b0 con 0x7f311c107da0 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f311b7fe640 1 -- 172.21.15.32:0/2649096895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519b60 msgr2=0x55bc3151e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2649096895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519b60 0x55bc3151e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f311b7fe640 1 -- 172.21.15.32:0/2649096895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 msgr2=0x55bc31521160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2649096895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc31521160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f311b7fe640 1 -- 172.21.15.32:0/2649096895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc31433bc0 con 0x55bc31519170 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2649096895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519170 0x55bc31519590 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f31100144f0 tx=0x7f311000b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd3821750b662fc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f3120f8f640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31040a3bb0 0x7f31040a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f31140be330 tx=0x7f31140b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f31027fc640 1 -- 172.21.15.32:0/2649096895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110005020 con 0x55bc31519170 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f31027fc640 1 -- 172.21.15.32:0/2649096895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31100047a0 con 0x55bc31519170 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.978+0000 7f31027fc640 1 -- 172.21.15.32:0/2649096895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110012710 con 0x55bc31519170 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 -- 172.21.15.32:0/2649096895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519170 msgr2=0x55bc31519590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 --2- 172.21.15.32:0/2649096895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519170 0x55bc31519590 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f31100144f0 tx=0x7f311000b4e0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 -- 172.21.15.32:0/2649096895 shutdown_connections 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 --2- 172.21.15.32:0/2649096895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc31521160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 --2- 172.21.15.32:0/2649096895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519b60 0x55bc3151e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 --2- 172.21.15.32:0/2649096895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519170 0x55bc31519590 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 -- 172.21.15.32:0/2649096895 >> 172.21.15.32:0/2649096895 conn(0x55bc315102d0 msgr2=0x55bc315106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 -- 172.21.15.32:0/2649096895 shutdown_connections 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 -- 172.21.15.32:0/2649096895 wait complete. 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 -- start start 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.979+0000 7f31255dc740 1 Processor -- start 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f31255dc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc31519170 0x55bc31523360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f31255dc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc315238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f31255dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc3151ecd0 0x55bc3152a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f31255dc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bc3147f0f0 con 0x55bc31519b60 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f31255dc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bc31479dd0 con 0x55bc31519170 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f31255dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc31487cd0 con 0x55bc3151ecd0 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f31255dc740 10 client.0 _ll_register_callbacks cb 0x55bc31470570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc315238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc315238a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60138/0 (socket says 172.21.15.32:60138) 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 -- 172.21.15.32:0/2874847483 learned_addr learned my addr 172.21.15.32:0/2874847483 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc31519170 0x55bc31523360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc3151ecd0 0x55bc3152a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 -- 172.21.15.32:0/2874847483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc3151ecd0 msgr2=0x55bc3152a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc3151ecd0 0x55bc3152a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 -- 172.21.15.32:0/2874847483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc31519170 msgr2=0x55bc31523360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc31519170 0x55bc31523360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 -- 172.21.15.32:0/2874847483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc3118d2d0 con 0x55bc31519b60 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc3151ecd0 0x55bc3152a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f311bfff640 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc315238a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f310c0116c0 tx=0x7f310c011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b435ebd5af6c615 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f30fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f30fb7fe640 1 -- 172.21.15.32:0/2874847483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c00fc50 con 0x55bc31519b60 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f30fb7fe640 1 -- 172.21.15.32:0/2874847483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f310c00fdf0 con 0x55bc31519b60 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f31255dc740 1 -- 172.21.15.32:0/2874847483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31100047a0 con 0x55bc31519b60 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.980+0000 7f30fb7fe640 1 -- 172.21.15.32:0/2874847483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f310c024050 con 0x55bc31519b60 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f31255dc740 10 client.15399 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f31255dc740 1 -- 172.21.15.32:0/2874847483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bc312576b0 con 0x55bc31519b60 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f30fb7fe640 1 -- 172.21.15.32:0/2874847483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31100047a0 con 0x55bc31519b60 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f30fb7fe640 1 -- 172.21.15.32:0/2874847483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f310c019430 con 0x55bc31519b60 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f31255dc740 10 client.15399 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f31255dc740 10 client.15399 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f31255dc740 20 client.15399 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f31255dc740 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bc315300c0 0x55bc31531530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f31255dc740 4 client.15399 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f31255dc740 1 -- 172.21.15.32:0/2874847483 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc314014c0 con 0x55bc315300c0 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.981+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bc315300c0 0x55bc31531530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f311b7fe640 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bc315300c0 0x55bc31531530 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e80113959d5c80f server_cookie=50d78695282c7f97 in_seq=0 out_seq=0 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f30fb7fe640 10 client.15399 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f30fb7fe640 1 -- 172.21.15.32:0/2874847483 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc314014c0 con 0x55bc315300c0 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f30fb7fe640 10 client.15399 handle_command_reply: tid=0 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 client.15399 shutdown 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 -- 172.21.15.32:0/2874847483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 msgr2=0x55bc315238a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc315238a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f310c0116c0 tx=0x7f310c011b80 comp rx=0 tx=0).stop 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 -- 172.21.15.32:0/2874847483 shutdown_connections 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bc315300c0 0x55bc31531530 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc3151ecd0 0x55bc3152a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f30fb7fe640 0 client.15399 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc315238a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 --2- 172.21.15.32:0/2874847483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc31519170 0x55bc31523360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.982+0000 7f31255dc740 1 -- 172.21.15.32:0/2874847483 >> 172.21.15.32:0/2874847483 conn(0x55bc315102d0 msgr2=0x55bc3150f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.983+0000 7f31255dc740 1 -- 172.21.15.32:0/2874847483 shutdown_connections 2024-04-16T05:34:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.983+0000 7f31255dc740 1 -- 172.21.15.32:0/2874847483 wait complete. 2024-04-16T05:34:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:47.983+0000 7f31255dc740 20 client.15399 trim_cache size 0 max 16384 2024-04-16T05:34:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f31255dc740 1 Processor -- start 2024-04-16T05:34:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f31255dc740 1 -- start start 2024-04-16T05:34:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f31255dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f31255dc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc3152a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f31255dc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc3152f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f31255dc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bc3152f870 con 0x55bc31519b60 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f31255dc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bc3152ff40 con 0x55bc3151ecd0 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f31255dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc31530610 con 0x55bc31519170 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc3152a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc3152a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60150/0 (socket says 172.21.15.32:60150) 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311bfff640 1 -- 172.21.15.32:0/686583769 learned_addr learned my addr 172.21.15.32:0/686583769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311b7fe640 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311b7fe640 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc3152f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311bfff640 1 -- 172.21.15.32:0/686583769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 msgr2=0x55bc315106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311bfff640 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311bfff640 1 -- 172.21.15.32:0/686583769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 msgr2=0x55bc3152f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311bfff640 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc3152f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311bfff640 1 -- 172.21.15.32:0/686583769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc31511040 con 0x55bc31519b60 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.001+0000 7f311b7fe640 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f311bfff640 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc3152a8a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f310c000c00 tx=0x7f310c019dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d38e2cdae816996 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f3101ffb640 1 -- 172.21.15.32:0/686583769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c019630 con 0x55bc31519b60 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f3101ffb640 1 -- 172.21.15.32:0/686583769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c005bf0 con 0x55bc31519b60 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f3101ffb640 1 -- 172.21.15.32:0/686583769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c003970 con 0x55bc31519b60 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 -- 172.21.15.32:0/686583769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 msgr2=0x55bc3152a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc3152a8a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f310c000c00 tx=0x7f310c019dd0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 -- 172.21.15.32:0/686583769 shutdown_connections 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc3152f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc3152a8a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 --2- 172.21.15.32:0/686583769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 -- 172.21.15.32:0/686583769 >> 172.21.15.32:0/686583769 conn(0x55bc315102d0 msgr2=0x55bc3150b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 -- 172.21.15.32:0/686583769 shutdown_connections 2024-04-16T05:34:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.002+0000 7f31255dc740 1 -- 172.21.15.32:0/686583769 wait complete. 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 1 -- start start 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 1 Processor -- start 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc31525ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc31594060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bc315313b0 con 0x55bc31519b60 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bc31530ce0 con 0x55bc3151ecd0 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc31531a80 con 0x55bc31519170 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f31255dc740 10 client.0 _ll_register_callbacks cb 0x7f310c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc31525ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc31525ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60166/0 (socket says 172.21.15.32:60166) 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311bfff640 1 -- 172.21.15.32:0/866345646 learned_addr learned my addr 172.21.15.32:0/866345646 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311b7fe640 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311bfff640 1 -- 172.21.15.32:0/866345646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 msgr2=0x55bc315257a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311bfff640 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315257a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311bfff640 1 -- 172.21.15.32:0/866345646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 msgr2=0x55bc31594060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311bfff640 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc31594060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.003+0000 7f311bfff640 1 -- 172.21.15.32:0/866345646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc315945a0 con 0x55bc31519b60 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f311bfff640 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc31525ce0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f310c0198a0 tx=0x7f310c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b6475d2536f4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f30fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f30fb7fe640 1 -- 172.21.15.32:0/866345646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c0050c0 con 0x55bc31519b60 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f30fb7fe640 1 -- 172.21.15.32:0/866345646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f310c005260 con 0x55bc31519b60 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 1 -- 172.21.15.32:0/866345646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc314884e0 con 0x55bc31519b60 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f30fb7fe640 1 -- 172.21.15.32:0/866345646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f310c0054a0 con 0x55bc31519b60 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 10 client.15405 fetch_fsmap learned FSMap version 24 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 1 -- 172.21.15.32:0/866345646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bc31595780 con 0x55bc31519b60 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f30fb7fe640 1 -- 172.21.15.32:0/866345646 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bc314884e0 con 0x55bc31519b60 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f30fb7fe640 1 -- 172.21.15.32:0/866345646 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f310c0288c0 con 0x55bc31519b60 2024-04-16T05:34:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 10 client.15405 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 10 client.15405 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 20 client.15405 populate_metadata read hostname 'smithi032' 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bc3159a130 0x55bc31598e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 4 client.15405 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.004+0000 7f31255dc740 1 -- 172.21.15.32:0/866345646 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "status"}) -- 0x55bc31482350 con 0x55bc3159a130 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f311b7fe640 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bc3159a130 0x55bc31598e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f311b7fe640 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bc3159a130 0x55bc31598e40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da3635d8d13e12b server_cookie=e8253036a26f9383 in_seq=0 out_seq=0 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f30fb7fe640 10 client.15405 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f30fb7fe640 1 -- 172.21.15.32:0/866345646 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bc31482350 con 0x55bc3159a130 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f30fb7fe640 10 client.15405 handle_command_reply: tid=0 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 client.15405 shutdown 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 -- 172.21.15.32:0/866345646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 msgr2=0x55bc31525ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc31525ce0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f310c0198a0 tx=0x7f310c0196e0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 -- 172.21.15.32:0/866345646 shutdown_connections 2024-04-16T05:34:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bc3151ecd0 0x55bc31594060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc31519b60 0x55bc31525ce0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bc3159a130 0x55bc31598e40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 --2- 172.21.15.32:0/866345646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc31519170 0x55bc315257a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f30fb7fe640 0 client.15405 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 -- 172.21.15.32:0/866345646 >> 172.21.15.32:0/866345646 conn(0x55bc315102d0 msgr2=0x55bc315106b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.005+0000 7f31255dc740 1 -- 172.21.15.32:0/866345646 shutdown_connections 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31255dc740 1 -- 172.21.15.32:0/866345646 wait complete. 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31255dc740 20 client.15405 trim_cache size 0 max 16384 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "578203a8-fbb1-11ee-bc8f-c7b262605968", 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24494, 2024-04-16T05:34:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-16T05:34:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-16T05:34:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-16T05:34:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 133.11318555099999, 2024-04-16T05:34:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 24, 2024-04-16T05:34:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 91, 2024-04-16T05:34:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T05:34:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 137.45942136100001 2024-04-16T05:34:48.012 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 -- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31040a3bb0 msgr2=0x7f31040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31040a3bb0 0x7f31040a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f31140be330 tx=0x7f31140b0040 comp rx=0 tx=0).stop 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 -- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 msgr2=0x7f311c19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 0x7f311c19f8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3114037040 tx=0x7f3114032f20 comp rx=0 tx=0).stop 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 -- 172.21.15.32:0/413371504 shutdown_connections 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f31040a3bb0 0x7f31040a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311c107da0 0x7f311c19f8a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f311c105470 0x7f311c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 --2- 172.21.15.32:0/413371504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f311c102b40 0x7f311c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 -- 172.21.15.32:0/413371504 >> 172.21.15.32:0/413371504 conn(0x7f311c0fc4b0 msgr2=0x7f311c103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:48.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 -- 172.21.15.32:0/413371504 shutdown_connections 2024-04-16T05:34:48.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.006+0000 7f31229f2640 1 -- 172.21.15.32:0/413371504 wait complete. 2024-04-16T05:34:48.016 INFO:teuthology.run_tasks:Running task sequential... 2024-04-16T05:34:48.025 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-16T05:34:48.025 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-16T05:34:48.025 INFO:teuthology.task.print:Enabling mgr modules 2024-04-16T05:34:48.025 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-16T05:34:48.026 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T05:34:48.026 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:34:48.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-16T05:34:48.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:47 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/488900682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:48.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:47 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/642928021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:48.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:47 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/801587404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 -- 172.21.15.32:0/3278841293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c107dc0 msgr2=0x7fe13c10a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 --2- 172.21.15.32:0/3278841293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c107dc0 0x7fe13c10a1b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe134009ca0 tx=0x7fe13402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 -- 172.21.15.32:0/3278841293 shutdown_connections 2024-04-16T05:34:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 --2- 172.21.15.32:0/3278841293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c107dc0 0x7fe13c10a1b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 --2- 172.21.15.32:0/3278841293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe13c105490 0x7fe13c107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 --2- 172.21.15.32:0/3278841293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe13c102b60 0x7fe13c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 -- 172.21.15.32:0/3278841293 >> 172.21.15.32:0/3278841293 conn(0x7fe13c0fc4f0 msgr2=0x7fe13c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 -- 172.21.15.32:0/3278841293 shutdown_connections 2024-04-16T05:34:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.138+0000 7fe140d59640 1 -- 172.21.15.32:0/3278841293 wait complete. 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe140d59640 1 Processor -- start 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe140d59640 1 -- start start 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe140d59640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c102b60 0x7fe13c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe140d59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe13c105490 0x7fe13c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe140d59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe13c107dc0 0x7fe13c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe140d59640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c10dcf0 con 0x7fe13c102b60 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe140d59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c10db70 con 0x7fe13c107dc0 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe140d59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c10de70 con 0x7fe13c105490 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe13a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c102b60 0x7fe13c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe13a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c102b60 0x7fe13c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60184/0 (socket says 172.21.15.32:60184) 2024-04-16T05:34:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe13a59c640 1 -- 172.21.15.32:0/766283118 learned_addr learned my addr 172.21.15.32:0/766283118 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.139+0000 7fe133fff640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe13c105490 0x7fe13c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe13a59c640 1 -- 172.21.15.32:0/766283118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe13c105490 msgr2=0x7fe13c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe13a59c640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe13c105490 0x7fe13c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe13a59c640 1 -- 172.21.15.32:0/766283118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe13c107dc0 msgr2=0x7fe13c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe13a59c640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe13c107dc0 0x7fe13c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe13a59c640 1 -- 172.21.15.32:0/766283118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c19fe40 con 0x7fe13c102b60 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe13a59c640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c102b60 0x7fe13c198220 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00e9f0 tx=0x7fe12c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe1337fe640 1 -- 172.21.15.32:0/766283118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00cd80 con 0x7fe13c102b60 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13c1a0130 con 0x7fe13c102b60 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe1337fe640 1 -- 172.21.15.32:0/766283118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe12c005500 con 0x7fe13c102b60 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe1337fe640 1 -- 172.21.15.32:0/766283118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c010690 con 0x7fe13c102b60 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.140+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13c1a0630 con 0x7fe13c102b60 2024-04-16T05:34:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.141+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe13c0fdfe0 con 0x7fe13c102b60 2024-04-16T05:34:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.142+0000 7fe1337fe640 1 -- 172.21.15.32:0/766283118 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe12c0046c0 con 0x7fe13c102b60 2024-04-16T05:34:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.143+0000 7fe1337fe640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1280a3c50 0x7fe1280a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.143+0000 7fe1337fe640 1 -- 172.21.15.32:0/766283118 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe12c0d0ed0 con 0x7fe13c102b60 2024-04-16T05:34:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.143+0000 7fe133fff640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1280a3c50 0x7fe1280a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.143+0000 7fe133fff640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1280a3c50 0x7fe1280a6110 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe124005fd0 tx=0x7fe124005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.146+0000 7fe1337fe640 1 -- 172.21.15.32:0/766283118 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe12c053ff0 con 0x7fe13c102b60 2024-04-16T05:34:48.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:47 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/488900682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:48.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:47 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/642928021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:48.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:47 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/801587404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.386+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fe13c062ca0 con 0x7fe13c102b60 2024-04-16T05:34:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.706+0000 7fe1337fe640 1 -- 172.21.15.32:0/766283118 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fe12c01f340 con 0x7fe13c102b60 2024-04-16T05:34:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1280a3c50 msgr2=0x7fe1280a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1280a3c50 0x7fe1280a6110 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe124005fd0 tx=0x7fe124005850 comp rx=0 tx=0).stop 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c102b60 msgr2=0x7fe13c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c102b60 0x7fe13c198220 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00e9f0 tx=0x7fe12c00edf0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 shutdown_connections 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fe1280a3c50 0x7fe1280a6110 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe13c107dc0 0x7fe13c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe13c105490 0x7fe13c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 --2- 172.21.15.32:0/766283118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13c102b60 0x7fe13c198220 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 >> 172.21.15.32:0/766283118 conn(0x7fe13c0fc4f0 msgr2=0x7fe13c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 shutdown_connections 2024-04-16T05:34:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.710+0000 7fe140d59640 1 -- 172.21.15.32:0/766283118 wait complete. 2024-04-16T05:34:48.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-16T05:34:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 -- 172.21.15.32:0/3167608095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 msgr2=0x7fb9a4104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 --2- 172.21.15.32:0/3167608095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 0x7fb9a4104f70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb994009900 tx=0x7fb99402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 -- 172.21.15.32:0/3167608095 shutdown_connections 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 --2- 172.21.15.32:0/3167608095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9a4107de0 0x7fb9a410a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 --2- 172.21.15.32:0/3167608095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a41054b0 0x7fb9a41078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 --2- 172.21.15.32:0/3167608095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 0x7fb9a4104f70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 -- 172.21.15.32:0/3167608095 >> 172.21.15.32:0/3167608095 conn(0x7fb9a40fc4d0 msgr2=0x7fb9a40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 -- 172.21.15.32:0/3167608095 shutdown_connections 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.842+0000 7fb9a9641640 1 -- 172.21.15.32:0/3167608095 wait complete. 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a9641640 1 Processor -- start 2024-04-16T05:34:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a9641640 1 -- start start 2024-04-16T05:34:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a9641640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 0x7fb9a4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a9641640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9a41054b0 0x7fb9a41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a9641640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a4107de0 0x7fb9a419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a9641640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a410dd10 con 0x7fb9a4102b80 2024-04-16T05:34:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a9641640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a410db90 con 0x7fb9a41054b0 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a9641640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a410de90 con 0x7fb9a4107de0 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.843+0000 7fb9a2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 0x7fb9a4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 0x7fb9a4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40200/0 (socket says 172.21.15.32:40200) 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a2ffd640 1 -- 172.21.15.32:0/1929324824 learned_addr learned my addr 172.21.15.32:0/1929324824 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb99a7fc640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9a41054b0 0x7fb9a41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a37fe640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a4107de0 0x7fb9a419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a2ffd640 1 -- 172.21.15.32:0/1929324824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a4107de0 msgr2=0x7fb9a419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a2ffd640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a4107de0 0x7fb9a419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a2ffd640 1 -- 172.21.15.32:0/1929324824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9a41054b0 msgr2=0x7fb9a41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a2ffd640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9a41054b0 0x7fb9a41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a2ffd640 1 -- 172.21.15.32:0/1929324824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a419ff40 con 0x7fb9a4102b80 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a2ffd640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 0x7fb9a4198370 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb994009230 tx=0x7fb994005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a0ff9640 1 -- 172.21.15.32:0/1929324824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994041070 con 0x7fb9a4102b80 2024-04-16T05:34:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a0ff9640 1 -- 172.21.15.32:0/1929324824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9940044d0 con 0x7fb9a4102b80 2024-04-16T05:34:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a0ff9640 1 -- 172.21.15.32:0/1929324824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994034440 con 0x7fb9a4102b80 2024-04-16T05:34:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a41a01d0 con 0x7fb9a4102b80 2024-04-16T05:34:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.844+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a41a06c0 con 0x7fb9a4102b80 2024-04-16T05:34:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.845+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a40fdfc0 con 0x7fb9a4102b80 2024-04-16T05:34:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.846+0000 7fb9a0ff9640 1 -- 172.21.15.32:0/1929324824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb994044070 con 0x7fb9a4102b80 2024-04-16T05:34:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.847+0000 7fb9a0ff9640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb9900a3ca0 0x7fb9900a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.847+0000 7fb99a7fc640 1 -- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb9900a3ca0 msgr2=0x7fb9900a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.847+0000 7fb9a0ff9640 1 -- 172.21.15.32:0/1929324824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb9940f2730 con 0x7fb9a4102b80 2024-04-16T05:34:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.847+0000 7fb99a7fc640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb9900a3ca0 0x7fb9900a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:34:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:48.851+0000 7fb9a0ff9640 1 -- 172.21.15.32:0/1929324824 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9940be760 con 0x7fb9a4102b80 2024-04-16T05:34:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:48 smithi032 ceph-mon[32351]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:48.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:48 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/766283118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T05:34:48.983 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: ignoring --setuser ceph since I am not root 2024-04-16T05:34:48.983 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: ignoring --setgroup ceph since I am not root 2024-04-16T05:34:49.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.042+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb9a402d620 con 0x7fb9a4102b80 2024-04-16T05:34:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.046+0000 7fb9a0ff9640 1 -- 172.21.15.32:0/1929324824 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fb9940c2a60 con 0x7fb9a4102b80 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb99a7fc640 1 -- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb9900a3ca0 msgr2=0x7fb9900a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb99a7fc640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb9900a3ca0 0x7fb9900a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb9900a3ca0 msgr2=0x7fb9900a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb9900a3ca0 0x7fb9900a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 msgr2=0x7fb9a4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 0x7fb9a4198370 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb994009230 tx=0x7fb994005d50 comp rx=0 tx=0).stop 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 shutdown_connections 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fb9900a3ca0 0x7fb9900a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a4107de0 0x7fb9a419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9a41054b0 0x7fb9a41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 --2- 172.21.15.32:0/1929324824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a4102b80 0x7fb9a4198370 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 >> 172.21.15.32:0/1929324824 conn(0x7fb9a40fc4d0 msgr2=0x7fb9a41034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 shutdown_connections 2024-04-16T05:34:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.048+0000 7fb9a9641640 1 -- 172.21.15.32:0/1929324824 wait complete. 2024-04-16T05:34:49.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-16T05:34:49.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: ignoring --setuser ceph since I am not root 2024-04-16T05:34:49.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: ignoring --setgroup ceph since I am not root 2024-04-16T05:34:49.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:48 smithi134 ceph-mgr[38044]: -- 172.21.15.134:0/1167444334 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9b28c65a0 con 0x55d9b289b000 2024-04-16T05:34:49.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:48 smithi134 ceph-mon[36153]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:49.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:48 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/766283118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T05:34:49.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:48 smithi082 ceph-mon[36195]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T05:34:49.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:48 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/766283118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T05:34:49.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: ignoring --setuser ceph since I am not root 2024-04-16T05:34:49.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: ignoring --setgroup ceph since I am not root 2024-04-16T05:34:49.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:48 smithi082 ceph-mgr[37585]: -- 172.21.15.82:0/1534459152 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599101da5a0 con 0x5599101af000 2024-04-16T05:34:49.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.167+0000 7f87ca29a640 1 -- 172.21.15.32:0/641587673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c4102aa0 msgr2=0x7f87c41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:49.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.167+0000 7f87ca29a640 1 --2- 172.21.15.32:0/641587673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c4102aa0 0x7f87c41093c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009ca0 tx=0x7f87bc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 -- 172.21.15.32:0/641587673 shutdown_connections 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 --2- 172.21.15.32:0/641587673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c4102aa0 0x7f87c41093c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 --2- 172.21.15.32:0/641587673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c41020e0 0x7f87c4102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 --2- 172.21.15.32:0/641587673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c4100f10 0x7f87c4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 -- 172.21.15.32:0/641587673 >> 172.21.15.32:0/641587673 conn(0x7f87c40fc4d0 msgr2=0x7f87c40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 -- 172.21.15.32:0/641587673 shutdown_connections 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 -- 172.21.15.32:0/641587673 wait complete. 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 Processor -- start 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 -- start start 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.168+0000 7f87ca29a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4100f10 0x7f87c406a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87ca29a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c41020e0 0x7f87c419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87ca29a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c4102aa0 0x7f87c419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87ca29a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410cbb0 con 0x7f87c41020e0 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87ca29a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410ca30 con 0x7f87c4102aa0 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87ca29a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410cd30 con 0x7f87c4100f10 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c41020e0 0x7f87c419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c41020e0 0x7f87c419cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40222/0 (socket says 172.21.15.32:40222) 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 -- 172.21.15.32:0/1707361406 learned_addr learned my addr 172.21.15.32:0/1707361406 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c3fff640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4100f10 0x7f87c406a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c8837640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c4102aa0 0x7f87c419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 -- 172.21.15.32:0/1707361406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4100f10 msgr2=0x7f87c406a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4100f10 0x7f87c406a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 -- 172.21.15.32:0/1707361406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c4102aa0 msgr2=0x7f87c419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c4102aa0 0x7f87c419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 -- 172.21.15.32:0/1707361406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c419fe90 con 0x7f87c41020e0 2024-04-16T05:34:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c37fe640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c41020e0 0x7f87c419cf60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f87b800d950 tx=0x7f87b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c17fa640 1 -- 172.21.15.32:0/1707361406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8017070 con 0x7f87c41020e0 2024-04-16T05:34:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c41a0120 con 0x7f87c41020e0 2024-04-16T05:34:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c17fa640 1 -- 172.21.15.32:0/1707361406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b8009c30 con 0x7f87c41020e0 2024-04-16T05:34:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87c17fa640 1 -- 172.21.15.32:0/1707361406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b800f6b0 con 0x7f87c41020e0 2024-04-16T05:34:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.169+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c41a0670 con 0x7f87c41020e0 2024-04-16T05:34:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.170+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c4108240 con 0x7f87c41020e0 2024-04-16T05:34:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.172+0000 7f87c17fa640 1 -- 172.21.15.32:0/1707361406 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f87b80109d0 con 0x7f87c41020e0 2024-04-16T05:34:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.172+0000 7f87c17fa640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f87ac0a3ca0 0x7f87ac0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.172+0000 7f87c3fff640 1 -- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f87ac0a3ca0 msgr2=0x7f87ac0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.172+0000 7f87c17fa640 1 -- 172.21.15.32:0/1707361406 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f87b80d0370 con 0x7f87c41020e0 2024-04-16T05:34:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.173+0000 7f87c3fff640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f87ac0a3ca0 0x7f87ac0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:34:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.180+0000 7f87c17fa640 1 -- 172.21.15.32:0/1707361406 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87b8009de0 con 0x7f87c41020e0 2024-04-16T05:34:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.369+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f87c4062ca0 con 0x7f87c41020e0 2024-04-16T05:34:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.373+0000 7f87c17fa640 1 -- 172.21.15.32:0/1707361406 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f87b809c3a0 con 0x7f87c41020e0 2024-04-16T05:34:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.374+0000 7f87c3fff640 1 -- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f87ac0a3ca0 msgr2=0x7f87ac0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.374+0000 7f87c3fff640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f87ac0a3ca0 0x7f87ac0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:34:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f87ac0a3ca0 msgr2=0x7f87ac0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:34:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f87ac0a3ca0 0x7f87ac0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c41020e0 msgr2=0x7f87c419cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c41020e0 0x7f87c419cf60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f87b800d950 tx=0x7f87b800de10 comp rx=0 tx=0).stop 2024-04-16T05:34:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 shutdown_connections 2024-04-16T05:34:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f87ac0a3ca0 0x7f87ac0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c4102aa0 0x7f87c419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c41020e0 0x7f87c419cf60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 --2- 172.21.15.32:0/1707361406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4100f10 0x7f87c406a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 >> 172.21.15.32:0/1707361406 conn(0x7f87c40fc4d0 msgr2=0x7f87c4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 shutdown_connections 2024-04-16T05:34:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:49.375+0000 7f87ca29a640 1 -- 172.21.15.32:0/1707361406 wait complete. 2024-04-16T05:34:49.385 INFO:teuthology.run_tasks:Running task kclient... 2024-04-16T05:34:49.400 INFO:tasks.kclient:Mounting kernel clients... 2024-04-16T05:34:49.400 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-16T05:34:49.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:49.400 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-16T05:34:49.416 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-16T05:34:49.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:49.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T05:34:49.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:49.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-16T05:34:49.589 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:49.312+0000 7f970cd9f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:34:49.589 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:49.411+0000 7f970cd9f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:34:49.597 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-16T05:34:49.597 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T05:34:49.597 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T05:34:49.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph fs ls 2024-04-16T05:34:49.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:49 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:49.275+0000 7f2964640200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:34:49.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:49 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:49.367+0000 7f2964640200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:34:49.649 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:49 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:49.273+0000 7f5e0d396200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T05:34:49.649 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:49 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:49.367+0000 7f5e0d396200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T05:34:49.859 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:34:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:49 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/766283118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T05:34:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:49 smithi032 ceph-mon[32351]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-16T05:34:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:49 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1929324824' entity='client.admin' 2024-04-16T05:34:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:49 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1707361406' entity='client.admin' 2024-04-16T05:34:50.004 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:49 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/766283118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T05:34:50.005 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:49 smithi134 ceph-mon[36153]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-16T05:34:50.005 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:49 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1929324824' entity='client.admin' 2024-04-16T05:34:50.005 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:49 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1707361406' entity='client.admin' 2024-04-16T05:34:50.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:49 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/766283118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T05:34:50.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:49 smithi082 ceph-mon[36195]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-16T05:34:50.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:49 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1929324824' entity='client.admin' 2024-04-16T05:34:50.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:49 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1707361406' entity='client.admin' 2024-04-16T05:34:50.270 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:50.106+0000 7f970cd9f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:34:50.270 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:50.208+0000 7f970cd9f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:34:50.330 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.005+0000 7f2964640200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:34:50.331 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.097+0000 7f2964640200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:34:50.331 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.238+0000 7f2964640200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:34:50.331 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.331+0000 7f2964640200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:34:50.335 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.009+0000 7f5e0d396200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T05:34:50.335 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.102+0000 7f5e0d396200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T05:34:50.335 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.243+0000 7f5e0d396200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:34:50.539 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:50.356+0000 7f970cd9f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T05:34:50.539 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:50.455+0000 7f970cd9f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:34:50.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.414+0000 7f2964640200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:34:50.649 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.336+0000 7f5e0d396200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T05:34:50.650 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.418+0000 7f5e0d396200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:34:50.817 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:50.540+0000 7f970cd9f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T05:34:50.965 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.677+0000 7f2964640200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:34:50.965 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.788+0000 7f2964640200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:34:50.966 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.873+0000 7f2964640200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:34:50.966 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:50.966+0000 7f2964640200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:34:50.972 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.682+0000 7f5e0d396200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:34:50.972 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.793+0000 7f5e0d396200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:34:50.972 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.879+0000 7f5e0d396200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:34:50.972 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:50.973+0000 7f5e0d396200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:34:51.066 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:50.816+0000 7f970cd9f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T05:34:51.067 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:50.930+0000 7f970cd9f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T05:34:51.067 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:51.020+0000 7f970cd9f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T05:34:51.381 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:51.124+0000 7f970cd9f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T05:34:51.381 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:51.310+0000 7f970cd9f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:34:51.384 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:51 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:51.141+0000 7f2964640200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:34:51.399 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:51 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:51.147+0000 7f5e0d396200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T05:34:51.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 -- 172.21.15.32:0/4151717172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006b120 msgr2=0x7ff69010f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:51.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 --2- 172.21.15.32:0/4151717172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006b120 0x7ff69010f060 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009990 tx=0x7ff67c009ca0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 -- 172.21.15.32:0/4151717172 shutdown_connections 2024-04-16T05:34:51.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 --2- 172.21.15.32:0/4151717172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff69010f5a0 0x7ff690111990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 --2- 172.21.15.32:0/4151717172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69006a800 0x7ff69006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 --2- 172.21.15.32:0/4151717172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006b120 0x7ff69010f060 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 -- 172.21.15.32:0/4151717172 >> 172.21.15.32:0/4151717172 conn(0x7ff6900789a0 msgr2=0x7ff690078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 -- 172.21.15.32:0/4151717172 shutdown_connections 2024-04-16T05:34:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.610+0000 7ff696b80700 1 -- 172.21.15.32:0/4151717172 wait complete. 2024-04-16T05:34:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff696b80700 1 Processor -- start 2024-04-16T05:34:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff696b80700 1 -- start start 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff696b80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006a800 0x7ff69019b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff696b80700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69006b120 0x7ff69019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff696b80700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff69010f5a0 0x7ff69019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff696b80700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff690115000 con 0x7ff69006a800 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff696b80700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff690114e80 con 0x7ff69010f5a0 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff696b80700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff690115180 con 0x7ff69006b120 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69491c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006a800 0x7ff69019b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69491c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006a800 0x7ff69019b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40244/0 (socket says 172.21.15.32:40244) 2024-04-16T05:34:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69491c700 1 -- 172.21.15.32:0/3532185591 learned_addr learned my addr 172.21.15.32:0/3532185591 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff68ffff700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69006b120 0x7ff69019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69511d700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff69010f5a0 0x7ff69019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69491c700 1 -- 172.21.15.32:0/3532185591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69006b120 msgr2=0x7ff69019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69491c700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69006b120 0x7ff69019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69491c700 1 -- 172.21.15.32:0/3532185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff69010f5a0 msgr2=0x7ff69019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69491c700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff69010f5a0 0x7ff69019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.611+0000 7ff69491c700 1 -- 172.21.15.32:0/3532185591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6901a0500 con 0x7ff69006a800 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.612+0000 7ff69511d700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff69010f5a0 0x7ff69019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.612+0000 7ff69491c700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006a800 0x7ff69019b4f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff68400d950 tx=0x7ff68400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.612+0000 7ff68dffb700 1 -- 172.21.15.32:0/3532185591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684016070 con 0x7ff69006a800 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.612+0000 7ff68dffb700 1 -- 172.21.15.32:0/3532185591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff684009e20 con 0x7ff69006a800 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.612+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6901a07f0 con 0x7ff69006a800 2024-04-16T05:34:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.612+0000 7ff68dffb700 1 -- 172.21.15.32:0/3532185591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68400f620 con 0x7ff69006a800 2024-04-16T05:34:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.612+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6901a8070 con 0x7ff69006a800 2024-04-16T05:34:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.613+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6900697e0 con 0x7ff69006a800 2024-04-16T05:34:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.614+0000 7ff68dffb700 1 -- 172.21.15.32:0/3532185591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7ff684019070 con 0x7ff69006a800 2024-04-16T05:34:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.615+0000 7ff68dffb700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff6800a3c90 0x7ff6800a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.615+0000 7ff68ffff700 1 -- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff6800a3c90 msgr2=0x7ff6800a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.615+0000 7ff68dffb700 1 -- 172.21.15.32:0/3532185591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff6840d0360 con 0x7ff69006a800 2024-04-16T05:34:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.615+0000 7ff68ffff700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff6800a3c90 0x7ff6800a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:34:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.620+0000 7ff68dffb700 1 -- 172.21.15.32:0/3532185591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff68409c9e0 con 0x7ff69006a800 2024-04-16T05:34:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.816+0000 7ff68ffff700 1 -- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff6800a3c90 msgr2=0x7ff6800a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.816+0000 7ff68ffff700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff6800a3c90 0x7ff6800a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:34:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.910+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff69002dfe0 con 0x7ff69006a800 2024-04-16T05:34:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.910+0000 7ff68dffb700 1 -- 172.21.15.32:0/3532185591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7ff68409c2e0 con 0x7ff69006a800 2024-04-16T05:34:51.909 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T05:34:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff6800a3c90 msgr2=0x7ff6800a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:34:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff6800a3c90 0x7ff6800a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006a800 msgr2=0x7ff69019b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006a800 0x7ff69019b4f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff68400d950 tx=0x7ff68400dc60 comp rx=0 tx=0).stop 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 shutdown_connections 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7ff6800a3c90 0x7ff6800a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff69010f5a0 0x7ff69019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff69006a800 0x7ff69019b4f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 --2- 172.21.15.32:0/3532185591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69006b120 0x7ff69019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 >> 172.21.15.32:0/3532185591 conn(0x7ff6900789a0 msgr2=0x7ff690110a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.912+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 shutdown_connections 2024-04-16T05:34:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:51.913+0000 7ff696b80700 1 -- 172.21.15.32:0/3532185591 wait complete. 2024-04-16T05:34:52.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:51 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3532185591' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:34:52.218 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:52.090+0000 7f2964640200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:34:52.230 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:51 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3532185591' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:34:52.230 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:52.102+0000 7f5e0d396200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:34:52.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:51 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3532185591' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T05:34:52.548 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:52.218+0000 7f2964640200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:34:52.559 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:52.231+0000 7f5e0d396200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:34:52.617 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:52.324+0000 7f970cd9f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T05:34:52.617 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:52.460+0000 7f970cd9f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T05:34:52.650 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-16T05:34:52.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:52.653 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-16T05:34:52.671 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41942sec preferred_lft 41942sec 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-16T05:34:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-16T05:34:52.672 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T05:34:52.673 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:34:52.673 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:34:52.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-04-16T05:34:52.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-04-16T05:34:52.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-04-16T05:34:52.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T05:34:52.673 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:34:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:34:52.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:52.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:52.830 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T05:34:52.834 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:52.549+0000 7f2964640200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:34:52.834 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:52.638+0000 7f2964640200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:34:52.834 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:52.836+0000 7f2964640200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:34:52.841 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:52.561+0000 7f5e0d396200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:34:52.841 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:52.648+0000 7f5e0d396200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:34:52.857 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-16T05:34:52.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:52.859 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-16T05:34:52.909 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:52.811+0000 7f970cd9f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T05:34:52.909 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:52.910+0000 7f970cd9f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T05:34:52.916 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-16T05:34:52.916 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T05:34:52.916 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-16T05:34:52.916 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-16T05:34:52.916 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T05:34:52.916 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:34:52.916 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:34:52.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-16T05:34:52.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-16T05:34:52.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-16T05:34:52.917 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:34:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:34:53.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:53.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:53.058 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-16T05:34:53.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:53.074 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-04-16T05:34:53.128 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:34:53.129 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:34:53.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T05:34:53.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-16T05:34:53.129 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:34:53.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:53.022+0000 7f2964640200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:34:53.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:52.842+0000 7f5e0d396200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:34:53.150 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:53.024+0000 7f5e0d396200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:34:53.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:34:53.230 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:53.122+0000 7f970cd9f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:34:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:53.255 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-16T05:34:53.256 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:34:53.256 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:34:53.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-16T05:34:53.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T05:34:53.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-16T05:34:53.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-16T05:34:53.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-16T05:34:53.256 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:34:53.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:34:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:53.389 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T05:34:53.389 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-16T05:34:53.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-16T05:34:53.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T05:34:53.390 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-16T05:34:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:34:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:53.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T05:34:53.475 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-16T05:34:53.482 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:53.324+0000 7f970cd9f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T05:34:53.493 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-16T05:34:53.493 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T05:34:53.493 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-16T05:34:53.549 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-16T05:34:53.549 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-16T05:34:53.549 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/96f1f0e6-c969-48f2-b2af-33d422a19791 2024-04-16T05:34:53.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:53.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/96f1f0e6-c969-48f2-b2af-33d422a19791 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T05:34:53.557 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:53.304+0000 7f5e0d396200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:34:53.568 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:53.308+0000 7f2964640200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:34:53.636 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T05:34:53.636 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-16T05:34:53.755 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T05:34:53.755 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-16T05:34:53.755 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.82:3300,172.21.15.134:3300" 2024-04-16T05:34:53.756 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@578203a8-fbb1-11ee-bc8f-c7b262605968.cephfs=/volumes/qa/sv_1/96f1f0e6-c969-48f2-b2af-33d422a19791 2024-04-16T05:34:53.756 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.32:3300/172.21.15.82:3300/172.21.15.134:3300" will pass to kernel 2024-04-16T05:34:53.757 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-16T05:34:53.757 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-16T05:34:53.757 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-16T05:34:53.757 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-16T05:34:53.757 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-16T05:34:53.759 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-16T05:34:53.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:34:53.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-16T05:34:53.796 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T05:34:53.884 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:53.570+0000 7f2964640200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:34:53.884 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:53.809+0000 7f2964640200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:34:53.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T05:34:53.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:53.895 INFO:teuthology.orchestra.run.smithi032.stdout:client15450 2024-04-16T05:34:53.895 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:34:53.895 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:34:53.899 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:53.558+0000 7f5e0d396200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:34:53.900 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:53.796+0000 7f5e0d396200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:34:53.912 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T05:34:53.912 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:53.638+0000 7f970cd9f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T05:34:53.912 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:53.912+0000 7f970cd9f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T05:34:54.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T05:34:54.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:54.009 INFO:teuthology.orchestra.run.smithi032.stdout:client15450 2024-04-16T05:34:54.009 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T05:34:54.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T05:34:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:54.067 INFO:teuthology.orchestra.run.smithi032.stdout:client15450 2024-04-16T05:34:54.068 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968.client15450/status') 2024-04-16T05:34:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T05:34:54.131 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.15450 (3)192.168.0.1:0/3707257285 2024-04-16T05:34:54.131 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2024-04-16T05:34:54.131 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2024-04-16T05:34:54.131 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2024-04-16T05:34:54.131 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 2.0608e-05 s, 3.4 MB/s 2024-04-16T05:34:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T05:34:54.138 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T05:34:54.147 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T05:34:54.147 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:34:54.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-16T05:34:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.257+0000 7f15b08f8640 1 -- 172.21.15.32:0/1303643694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 msgr2=0x7f15ac101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.257+0000 7f15b08f8640 1 --2- 172.21.15.32:0/1303643694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 0x7f15ac101350 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f159c009900 tx=0x7f159c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 -- 172.21.15.32:0/1303643694 shutdown_connections 2024-04-16T05:34:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 --2- 172.21.15.32:0/1303643694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15ac102aa0 0x7f15ac1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 --2- 172.21.15.32:0/1303643694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ac1020e0 0x7f15ac102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 --2- 172.21.15.32:0/1303643694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 0x7f15ac101350 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 -- 172.21.15.32:0/1303643694 >> 172.21.15.32:0/1303643694 conn(0x7f15ac0fc4d0 msgr2=0x7f15ac0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 -- 172.21.15.32:0/1303643694 shutdown_connections 2024-04-16T05:34:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 -- 172.21.15.32:0/1303643694 wait complete. 2024-04-16T05:34:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 Processor -- start 2024-04-16T05:34:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 -- start start 2024-04-16T05:34:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 0x7f15ac06a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ac1020e0 0x7f15ac19d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15ac102aa0 0x7f15ac19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10cbe0 con 0x7f15ac100f10 2024-04-16T05:34:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10ca60 con 0x7f15ac102aa0 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.258+0000 7f15b08f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10cd60 con 0x7f15ac1020e0 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aa59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 0x7f15ac06a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15ac102aa0 0x7f15ac19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aa59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 0x7f15ac06a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40258/0 (socket says 172.21.15.32:40258) 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aa59c640 1 -- 172.21.15.32:0/2631895531 learned_addr learned my addr 172.21.15.32:0/2631895531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15a3fff640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ac1020e0 0x7f15ac19d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aad9d640 1 -- 172.21.15.32:0/2631895531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ac1020e0 msgr2=0x7f15ac19d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aad9d640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ac1020e0 0x7f15ac19d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aad9d640 1 -- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 msgr2=0x7f15ac06a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aad9d640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 0x7f15ac06a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aad9d640 1 -- 172.21.15.32:0/2631895531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac19ff70 con 0x7f15ac102aa0 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aa59c640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 0x7f15ac06a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15aad9d640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15ac102aa0 0x7f15ac19f6b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009fd0 tx=0x7f15a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:54.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15a37fe640 1 -- 172.21.15.32:0/2631895531 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a400ce20 con 0x7f15ac102aa0 2024-04-16T05:34:54.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15a37fe640 1 -- 172.21.15.32:0/2631895531 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15a4005020 con 0x7f15ac102aa0 2024-04-16T05:34:54.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac1a0260 con 0x7f15ac102aa0 2024-04-16T05:34:54.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15a37fe640 1 -- 172.21.15.32:0/2631895531 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4010720 con 0x7f15ac102aa0 2024-04-16T05:34:54.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.259+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1a0760 con 0x7f15ac102aa0 2024-04-16T05:34:54.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.260+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac1082a0 con 0x7f15ac102aa0 2024-04-16T05:34:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.262+0000 7f15a37fe640 1 -- 172.21.15.32:0/2631895531 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f15a4005540 con 0x7f15ac102aa0 2024-04-16T05:34:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.262+0000 7f15a37fe640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f15980a3ca0 0x7f15980a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.262+0000 7f15aa59c640 1 -- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f15980a3ca0 msgr2=0x7f15980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.262+0000 7f15a37fe640 1 -- 172.21.15.32:0/2631895531 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f15a40d12e0 con 0x7f15ac102aa0 2024-04-16T05:34:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.262+0000 7f15aa59c640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f15980a3ca0 0x7f15980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:34:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.266+0000 7f15a37fe640 1 -- 172.21.15.32:0/2631895531 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15a409d960 con 0x7f15ac102aa0 2024-04-16T05:34:54.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.463+0000 7f15aa59c640 1 -- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f15980a3ca0 msgr2=0x7f15980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:54.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.463+0000 7f15aa59c640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f15980a3ca0 0x7f15980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T05:34:54.482 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:54 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:54.169+0000 7f970cd9f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T05:34:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.494+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f15ac02d7a0 con 0x7f15ac102aa0 2024-04-16T05:34:54.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.549+0000 7f15a37fe640 1 -- 172.21.15.32:0/2631895531 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f15a409d260 con 0x7f15ac102aa0 2024-04-16T05:34:54.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f15980a3ca0 msgr2=0x7f15980a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:34:54.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f15980a3ca0 0x7f15980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15ac102aa0 msgr2=0x7f15ac19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15ac102aa0 0x7f15ac19f6b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009fd0 tx=0x7f15a400ef10 comp rx=0 tx=0).stop 2024-04-16T05:34:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 shutdown_connections 2024-04-16T05:34:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f15980a3ca0 0x7f15980a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15ac102aa0 0x7f15ac19f6b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ac1020e0 0x7f15ac19d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 --2- 172.21.15.32:0/2631895531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ac100f10 0x7f15ac06a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.552+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 >> 172.21.15.32:0/2631895531 conn(0x7f15ac0fc4d0 msgr2=0x7f15ac105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.553+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 shutdown_connections 2024-04-16T05:34:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.553+0000 7f15b08f8640 1 -- 172.21.15.32:0/2631895531 wait complete. 2024-04-16T05:34:54.562 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-16T05:34:54.573 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-16T05:34:54.617 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 -- 172.21.15.32:0/190678697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00102090 msgr2=0x7fbf00102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 --2- 172.21.15.32:0/190678697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00102090 0x7fbf00102510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 -- 172.21.15.32:0/190678697 shutdown_connections 2024-04-16T05:34:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 --2- 172.21.15.32:0/190678697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00102a50 0x7fbf00109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 --2- 172.21.15.32:0/190678697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00102090 0x7fbf00102510 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 --2- 172.21.15.32:0/190678697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf00100f40 0x7fbf00101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 -- 172.21.15.32:0/190678697 >> 172.21.15.32:0/190678697 conn(0x7fbf000fc4b0 msgr2=0x7fbf000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 -- 172.21.15.32:0/190678697 shutdown_connections 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 -- 172.21.15.32:0/190678697 wait complete. 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 Processor -- start 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.727+0000 7fbf07c95640 1 -- start start 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf07c95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100f40 0x7fbf00198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf07c95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00102090 0x7fbf001988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf07c95640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf00102a50 0x7fbf0019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf07c95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010cbf0 con 0x7fbf00100f40 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf07c95640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010ca70 con 0x7fbf00102a50 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf07c95640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010cd70 con 0x7fbf00102090 2024-04-16T05:34:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100f40 0x7fbf00198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100f40 0x7fbf00198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40270/0 (socket says 172.21.15.32:40270) 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 -- 172.21.15.32:0/1511698800 learned_addr learned my addr 172.21.15.32:0/1511698800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05230640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00102090 0x7fbf001988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf06232640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf00102a50 0x7fbf0019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 -- 172.21.15.32:0/1511698800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00102090 msgr2=0x7fbf001988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00102090 0x7fbf001988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 -- 172.21.15.32:0/1511698800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf00102a50 msgr2=0x7fbf0019f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf00102a50 0x7fbf0019f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 -- 172.21.15.32:0/1511698800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0019ffb0 con 0x7fbf00100f40 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf05a31640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100f40 0x7fbf00198390 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e9f0 tx=0x7fbefc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbef2ffd640 1 -- 172.21.15.32:0/1511698800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc00cd80 con 0x7fbf00100f40 2024-04-16T05:34:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbef2ffd640 1 -- 172.21.15.32:0/1511698800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbefc010910 con 0x7fbf00100f40 2024-04-16T05:34:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf001a02a0 con 0x7fbf00100f40 2024-04-16T05:34:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.728+0000 7fbef2ffd640 1 -- 172.21.15.32:0/1511698800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0188f0 con 0x7fbf00100f40 2024-04-16T05:34:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.729+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf001a07c0 con 0x7fbf00100f40 2024-04-16T05:34:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.730+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf001a0a20 con 0x7fbf00100f40 2024-04-16T05:34:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.731+0000 7fbef2ffd640 1 -- 172.21.15.32:0/1511698800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fbefc010ac0 con 0x7fbf00100f40 2024-04-16T05:34:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.731+0000 7fbef2ffd640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbeec0a3c00 0x7fbeec0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.731+0000 7fbf05230640 1 -- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbeec0a3c00 msgr2=0x7fbeec0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.732+0000 7fbef2ffd640 1 -- 172.21.15.32:0/1511698800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbefc0d0b50 con 0x7fbf00100f40 2024-04-16T05:34:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.732+0000 7fbf05230640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbeec0a3c00 0x7fbeec0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:34:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.736+0000 7fbef2ffd640 1 -- 172.21.15.32:0/1511698800 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbefc09cb80 con 0x7fbf00100f40 2024-04-16T05:34:54.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:54 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2631895531' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T05:34:54.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:54 smithi134 ceph-mon[36153]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T05:34:54.884 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:54 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:54.610+0000 7f2964640200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:34:54.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:54 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2631895531' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T05:34:54.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:54 smithi082 ceph-mon[36195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T05:34:54.902 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:54 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:54.597+0000 7f5e0d396200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:34:54.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.927+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf001a1170 con 0x7fbf00100f40 2024-04-16T05:34:54.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.929+0000 7fbef2ffd640 1 -- 172.21.15.32:0/1511698800 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24618 (secure 0 0 0) 0x7fbefc0a0e80 con 0x7fbf00100f40 2024-04-16T05:34:54.928 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:54.928 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:33:34.757328+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbeec0a3c00 msgr2=0x7fbeec0a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbeec0a3c00 0x7fbeec0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100f40 msgr2=0x7fbf00198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100f40 0x7fbf00198390 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e9f0 tx=0x7fbefc00eeb0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 shutdown_connections 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fbeec0a3c00 0x7fbeec0a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf00102a50 0x7fbf0019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf00102090 0x7fbf001988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 --2- 172.21.15.32:0/1511698800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100f40 0x7fbf00198390 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 >> 172.21.15.32:0/1511698800 conn(0x7fbf000fc4b0 msgr2=0x7fbf000fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 shutdown_connections 2024-04-16T05:34:54.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:54.931+0000 7fbf07c95640 1 -- 172.21.15.32:0/1511698800 wait complete. 2024-04-16T05:34:54.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:54 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2631895531' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T05:34:54.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:54 smithi032 ceph-mon[32351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T05:34:54.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:54.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:55.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.054+0000 7f9724de2640 1 -- 172.21.15.32:0/2603502493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 msgr2=0x7f97201036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.054+0000 7f9724de2640 1 --2- 172.21.15.32:0/2603502493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 0x7f97201036c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9714009900 tx=0x7f971402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:55.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 -- 172.21.15.32:0/2603502493 shutdown_connections 2024-04-16T05:34:55.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 --2- 172.21.15.32:0/2603502493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9720103c00 0x7f972010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 --2- 172.21.15.32:0/2603502493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 0x7f97201036c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 --2- 172.21.15.32:0/2603502493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9720102050 0x7f9720102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 -- 172.21.15.32:0/2603502493 >> 172.21.15.32:0/2603502493 conn(0x7f97200fd5e0 msgr2=0x7f97200ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:55.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 -- 172.21.15.32:0/2603502493 shutdown_connections 2024-04-16T05:34:55.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 -- 172.21.15.32:0/2603502493 wait complete. 2024-04-16T05:34:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 Processor -- start 2024-04-16T05:34:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 -- start start 2024-04-16T05:34:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9720102050 0x7f9720075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 0x7f97200743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9720103c00 0x7f9720074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.055+0000 7f9724de2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f972010dce0 con 0x7f9720103240 2024-04-16T05:34:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f9724de2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f972010db60 con 0x7f9720102050 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f9724de2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f972010de60 con 0x7f9720103c00 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 0x7f97200743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 0x7f97200743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40290/0 (socket says 172.21.15.32:40290) 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971dd9b640 1 -- 172.21.15.32:0/3848336385 learned_addr learned my addr 172.21.15.32:0/3848336385 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971e59c640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9720102050 0x7f9720075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971ed9d640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9720103c00 0x7f9720074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971e59c640 1 -- 172.21.15.32:0/3848336385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9720103c00 msgr2=0x7f9720074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971e59c640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9720103c00 0x7f9720074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971e59c640 1 -- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 msgr2=0x7f97200743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971e59c640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 0x7f97200743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971e59c640 1 -- 172.21.15.32:0/3848336385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97200751c0 con 0x7f9720102050 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971dd9b640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 0x7f97200743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f971e59c640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9720102050 0x7f9720075db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f971000e9c0 tx=0x7f971000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f970f7fe640 1 -- 172.21.15.32:0/3848336385 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971000ccc0 con 0x7f9720102050 2024-04-16T05:34:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97201a2490 con 0x7f9720102050 2024-04-16T05:34:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f970f7fe640 1 -- 172.21.15.32:0/3848336385 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97100035f0 con 0x7f9720102050 2024-04-16T05:34:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f970f7fe640 1 -- 172.21.15.32:0/3848336385 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971000ce60 con 0x7f9720102050 2024-04-16T05:34:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.056+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97201a2960 con 0x7f9720102050 2024-04-16T05:34:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.057+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9720068590 con 0x7f9720102050 2024-04-16T05:34:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.058+0000 7f970f7fe640 1 -- 172.21.15.32:0/3848336385 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f97100037a0 con 0x7f9720102050 2024-04-16T05:34:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.059+0000 7f970f7fe640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f97080a3c00 0x7f97080a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.059+0000 7f971dd9b640 1 -- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f97080a3c00 msgr2=0x7f97080a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.059+0000 7f970f7fe640 1 -- 172.21.15.32:0/3848336385 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f97100d0ed0 con 0x7f9720102050 2024-04-16T05:34:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.059+0000 7f971dd9b640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f97080a3c00 0x7f97080a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:34:55.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.063+0000 7f970f7fe640 1 -- 172.21.15.32:0/3848336385 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f971009d4f0 con 0x7f9720102050 2024-04-16T05:34:55.232 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:55.041+0000 7f970cd9f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T05:34:55.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.252+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9720062c00 con 0x7f9720102050 2024-04-16T05:34:55.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.252+0000 7f970f7fe640 1 -- 172.21.15.32:0/3848336385 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f971009cdf0 con 0x7f9720102050 2024-04-16T05:34:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f97080a3c00 msgr2=0x7f97080a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f97080a3c00 0x7f97080a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9720102050 msgr2=0x7f9720075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9720102050 0x7f9720075db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f971000e9c0 tx=0x7f971000ee80 comp rx=0 tx=0).stop 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 shutdown_connections 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f97080a3c00 0x7f97080a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9720103c00 0x7f9720074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720103240 0x7f97200743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 --2- 172.21.15.32:0/3848336385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9720102050 0x7f9720075db0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 >> 172.21.15.32:0/3848336385 conn(0x7f97200fd5e0 msgr2=0x7f97201065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:55.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 shutdown_connections 2024-04-16T05:34:55.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.254+0000 7f9724de2640 1 -- 172.21.15.32:0/3848336385 wait complete. 2024-04-16T05:34:55.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:34:55.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.374+0000 7fc09f17c640 1 -- 172.21.15.32:0/1192803127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098102b20 msgr2=0x7fc098104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.374+0000 7fc09f17c640 1 --2- 172.21.15.32:0/1192803127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098102b20 0x7fc098104f10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009900 tx=0x7fc08c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:55.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.374+0000 7fc09f17c640 1 -- 172.21.15.32:0/1192803127 shutdown_connections 2024-04-16T05:34:55.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.374+0000 7fc09f17c640 1 --2- 172.21.15.32:0/1192803127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc098107d80 0x7fc09810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.374+0000 7fc09f17c640 1 --2- 172.21.15.32:0/1192803127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc098105450 0x7fc098107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.374+0000 7fc09f17c640 1 --2- 172.21.15.32:0/1192803127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098102b20 0x7fc098104f10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.374+0000 7fc09f17c640 1 -- 172.21.15.32:0/1192803127 >> 172.21.15.32:0/1192803127 conn(0x7fc0980fc4b0 msgr2=0x7fc0980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 -- 172.21.15.32:0/1192803127 shutdown_connections 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 -- 172.21.15.32:0/1192803127 wait complete. 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 Processor -- start 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 -- start start 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc098102b20 0x7fc0981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc098105450 0x7fc0981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098107d80 0x7fc09819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810dcd0 con 0x7fc098107d80 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810db50 con 0x7fc098102b20 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09f17c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810de50 con 0x7fc098105450 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09d719640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098107d80 0x7fc09819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09cf18640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc098102b20 0x7fc0981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09d719640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098107d80 0x7fc09819f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40304/0 (socket says 172.21.15.32:40304) 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09cf18640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc098102b20 0x7fc0981983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.32:60764/0 (socket says 172.21.15.32:60764) 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.375+0000 7fc09d719640 1 -- 172.21.15.32:0/3871571320 learned_addr learned my addr 172.21.15.32:0/3871571320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc097fff640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc098105450 0x7fc0981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09cf18640 1 -- 172.21.15.32:0/3871571320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc098105450 msgr2=0x7fc0981988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09cf18640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc098105450 0x7fc0981988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09cf18640 1 -- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098107d80 msgr2=0x7fc09819f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09cf18640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098107d80 0x7fc09819f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09cf18640 1 -- 172.21.15.32:0/3871571320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09819ffd0 con 0x7fc098102b20 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09d719640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098107d80 0x7fc09819f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09cf18640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc098102b20 0x7fc0981983b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009230 tx=0x7fc08c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc095ffb640 1 -- 172.21.15.32:0/3871571320 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c041070 con 0x7fc098102b20 2024-04-16T05:34:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc095ffb640 1 -- 172.21.15.32:0/3871571320 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc08c004450 con 0x7fc098102b20 2024-04-16T05:34:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0981a0260 con 0x7fc098102b20 2024-04-16T05:34:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc095ffb640 1 -- 172.21.15.32:0/3871571320 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c003df0 con 0x7fc098102b20 2024-04-16T05:34:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.376+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0981a0750 con 0x7fc098102b20 2024-04-16T05:34:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.377+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0980fdfa0 con 0x7fc098102b20 2024-04-16T05:34:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.379+0000 7fc095ffb640 1 -- 172.21.15.32:0/3871571320 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fc08c044070 con 0x7fc098102b20 2024-04-16T05:34:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.379+0000 7fc095ffb640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0880a3c00 0x7fc0880a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.379+0000 7fc097fff640 1 -- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0880a3c00 msgr2=0x7fc0880a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.379+0000 7fc095ffb640 1 -- 172.21.15.32:0/3871571320 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc08c0f2200 con 0x7fc098102b20 2024-04-16T05:34:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.379+0000 7fc097fff640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0880a3c00 0x7fc0880a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:34:55.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.383+0000 7fc095ffb640 1 -- 172.21.15.32:0/3871571320 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc08c0be1d0 con 0x7fc098102b20 2024-04-16T05:34:55.480 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:55.168+0000 7f5e0d396200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:34:55.480 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:55.264+0000 7f5e0d396200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:34:55.501 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:55.186+0000 7f2964640200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:34:55.501 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:55.284+0000 7f2964640200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1511698800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3848336385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: Standby manager daemon y restarted 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: Standby manager daemon y started 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:34:55.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: Standby manager daemon z restarted 2024-04-16T05:34:55.552 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: Standby manager daemon z started 2024-04-16T05:34:55.552 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:34:55.552 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:34:55.552 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:34:55.552 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-mon[32351]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:34:55.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.564+0000 7fc095ffb640 1 -- 172.21.15.32:0/3871571320 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fc08c0035f0 con 0x7fc098102b20 2024-04-16T05:34:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.573+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc09802d7b0 con 0x7fc098102b20 2024-04-16T05:34:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.574+0000 7fc095ffb640 1 -- 172.21.15.32:0/3871571320 <== mon.2 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fc08c0df370 con 0x7fc098102b20 2024-04-16T05:34:55.573 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0880a3c00 msgr2=0x7fc0880a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0880a3c00 0x7fc0880a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc098102b20 msgr2=0x7fc0981983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc098102b20 0x7fc0981983b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009230 tx=0x7fc08c02c650 comp rx=0 tx=0).stop 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 shutdown_connections 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7fc0880a3c00 0x7fc0880a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc098107d80 0x7fc09819f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc098105450 0x7fc0981988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 --2- 172.21.15.32:0/3871571320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc098102b20 0x7fc0981983b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 >> 172.21.15.32:0/3871571320 conn(0x7fc0980fc4b0 msgr2=0x7fc098103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 shutdown_connections 2024-04-16T05:34:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.578+0000 7fc09f17c640 1 -- 172.21.15.32:0/3871571320 wait complete. 2024-04-16T05:34:55.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:34:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.702+0000 7f78c0077640 1 -- 172.21.15.32:0/3063352146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102a50 msgr2=0x7f78b8109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.702+0000 7f78c0077640 1 --2- 172.21.15.32:0/3063352146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102a50 0x7f78b8109370 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009ca0 tx=0x7f78b002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.702+0000 7f78c0077640 1 -- 172.21.15.32:0/3063352146 shutdown_connections 2024-04-16T05:34:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.702+0000 7f78c0077640 1 --2- 172.21.15.32:0/3063352146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102a50 0x7f78b8109370 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.702+0000 7f78c0077640 1 --2- 172.21.15.32:0/3063352146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8102090 0x7f78b8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.702+0000 7f78c0077640 1 --2- 172.21.15.32:0/3063352146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78b8100f40 0x7f78b8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.702+0000 7f78c0077640 1 -- 172.21.15.32:0/3063352146 >> 172.21.15.32:0/3063352146 conn(0x7f78b80fc4b0 msgr2=0x7f78b80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 -- 172.21.15.32:0/3063352146 shutdown_connections 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 -- 172.21.15.32:0/3063352146 wait complete. 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 Processor -- start 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 -- start start 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78b8100f40 0x7f78b8069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102090 0x7f78b819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8102a50 0x7f78b819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78b810cbf0 con 0x7f78b8102090 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f78b810ca70 con 0x7f78b8102a50 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78c0077640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78b810cd70 con 0x7f78b8100f40 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78bd612640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102090 0x7f78b819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78bd612640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102090 0x7f78b819d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40318/0 (socket says 172.21.15.32:40318) 2024-04-16T05:34:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.703+0000 7f78bd612640 1 -- 172.21.15.32:0/2069138994 learned_addr learned my addr 172.21.15.32:0/2069138994 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78be614640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8102a50 0x7f78b819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78bde13640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78b8100f40 0x7f78b8069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78bd612640 1 -- 172.21.15.32:0/2069138994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78b8100f40 msgr2=0x7f78b8069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78bd612640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78b8100f40 0x7f78b8069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78bd612640 1 -- 172.21.15.32:0/2069138994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8102a50 msgr2=0x7f78b819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78bd612640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8102a50 0x7f78b819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78bd612640 1 -- 172.21.15.32:0/2069138994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b819ffe0 con 0x7f78b8102090 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78bd612640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102090 0x7f78b819d0b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f78a800e9c0 tx=0x7f78a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78aeffd640 1 -- 172.21.15.32:0/2069138994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a800cd80 con 0x7f78b8102090 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b81a0270 con 0x7f78b8102090 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78aeffd640 1 -- 172.21.15.32:0/2069138994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78a80035f0 con 0x7f78b8102090 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78aeffd640 1 -- 172.21.15.32:0/2069138994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8010690 con 0x7f78b8102090 2024-04-16T05:34:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.704+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b81a07c0 con 0x7f78b8102090 2024-04-16T05:34:55.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.705+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b8108250 con 0x7f78b8102090 2024-04-16T05:34:55.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.706+0000 7f78aeffd640 1 -- 172.21.15.32:0/2069138994 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f78a80037a0 con 0x7f78b8102090 2024-04-16T05:34:55.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.707+0000 7f78aeffd640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f78a40a3c00 0x7f78a40a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:55.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.707+0000 7f78bde13640 1 -- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f78a40a3c00 msgr2=0x7f78a40a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2875882657 2024-04-16T05:34:55.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.707+0000 7f78aeffd640 1 -- 172.21.15.32:0/2069138994 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f78a80d0c50 con 0x7f78b8102090 2024-04-16T05:34:55.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.707+0000 7f78bde13640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f78a40a3c00 0x7f78a40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T05:34:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.711+0000 7f78aeffd640 1 -- 172.21.15.32:0/2069138994 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78a80d6020 con 0x7f78b8102090 2024-04-16T05:34:55.884 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z[38040]: 2024-04-16T05:34:55.503+0000 7f2964640200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:34:55.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-16T05:34:55.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T05:34:55.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1511698800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3848336385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: Standby manager daemon y restarted 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: Standby manager daemon y started 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: Standby manager daemon z restarted 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: Standby manager daemon z started 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:34:55.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:55 smithi134 ceph-mon[36153]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:34:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.898+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78b802d620 con 0x7f78b8102090 2024-04-16T05:34:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.899+0000 7f78aeffd640 1 -- 172.21.15.32:0/2069138994 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f78a809d2d0 con 0x7f78b8102090 2024-04-16T05:34:55.897 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:34:55.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.900+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f78a40a3c00 msgr2=0x7f78a40a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.900+0000 7f78c0077640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f78a40a3c00 0x7f78a40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102090 msgr2=0x7f78b819d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102090 0x7f78b819d0b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f78a800e9c0 tx=0x7f78a800ee80 comp rx=0 tx=0).stop 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 shutdown_connections 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:6800/2875882657,v1:172.21.15.32:6801/2875882657] conn(0x7f78a40a3c00 0x7f78a40a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8102a50 0x7f78b819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b8102090 0x7f78b819d0b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 --2- 172.21.15.32:0/2069138994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78b8100f40 0x7f78b8069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 >> 172.21.15.32:0/2069138994 conn(0x7f78b80fc4b0 msgr2=0x7f78b8105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 shutdown_connections 2024-04-16T05:34:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:55.901+0000 7f78c0077640 1 -- 172.21.15.32:0/2069138994 wait complete. 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y[37575]: 2024-04-16T05:34:55.481+0000 7f5e0d396200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1511698800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3848336385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: Standby manager daemon y restarted 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: Standby manager daemon y started 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:34:55.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T05:34:55.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.82:0/1401963687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:34:55.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: Standby manager daemon z restarted 2024-04-16T05:34:55.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: Standby manager daemon z started 2024-04-16T05:34:55.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T05:34:55.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T05:34:55.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T05:34:55.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:55 smithi082 ceph-mon[36195]: from='mgr.? 172.21.15.134:0/1247221493' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T05:34:55.909 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:55.657+0000 7f970cd9f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T05:34:55.910 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:55 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:55.765+0000 7f970cd9f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T05:34:55.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:34:56.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 -- 172.21.15.32:0/1186784299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103c00 msgr2=0x7fd26410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 --2- 172.21.15.32:0/1186784299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103c00 0x7fd26410a490 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009900 tx=0x7fd25c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:56.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 -- 172.21.15.32:0/1186784299 shutdown_connections 2024-04-16T05:34:56.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 --2- 172.21.15.32:0/1186784299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103c00 0x7fd26410a490 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 --2- 172.21.15.32:0/1186784299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264103240 0x7fd2641036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 --2- 172.21.15.32:0/1186784299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd264102050 0x7fd264102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 -- 172.21.15.32:0/1186784299 >> 172.21.15.32:0/1186784299 conn(0x7fd2640fd5e0 msgr2=0x7fd2640ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:56.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 -- 172.21.15.32:0/1186784299 shutdown_connections 2024-04-16T05:34:56.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.028+0000 7fd26b11b640 1 -- 172.21.15.32:0/1186784299 wait complete. 2024-04-16T05:34:56.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.029+0000 7fd26b11b640 1 Processor -- start 2024-04-16T05:34:56.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.029+0000 7fd26b11b640 1 -- start start 2024-04-16T05:34:56.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd26b11b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd264102050 0x7fd2641982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd26b11b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103240 0x7fd264198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd26b11b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264103c00 0x7fd26419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd26b11b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd26410dce0 con 0x7fd264103240 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd26b11b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd26410db60 con 0x7fd264103c00 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd26b11b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd26410de60 con 0x7fd264102050 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103240 0x7fd264198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103240 0x7fd264198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40372/0 (socket says 172.21.15.32:40372) 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 -- 172.21.15.32:0/93259076 learned_addr learned my addr 172.21.15.32:0/93259076 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd268eb7640 1 --2- 172.21.15.32:0/93259076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd264102050 0x7fd2641982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 -- 172.21.15.32:0/93259076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd264102050 msgr2=0x7fd2641982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 --2- 172.21.15.32:0/93259076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd264102050 0x7fd2641982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 -- 172.21.15.32:0/93259076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264103c00 msgr2=0x7fd26419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 --2- 172.21.15.32:0/93259076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264103c00 0x7fd26419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 -- 172.21.15.32:0/93259076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26419ff00 con 0x7fd264103240 2024-04-16T05:34:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.030+0000 7fd25bfff640 1 --2- 172.21.15.32:0/93259076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103240 0x7fd264198820 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd254009b60 tx=0x7fd254007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:56.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.031+0000 7fd259ffb640 1 -- 172.21.15.32:0/93259076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254009d70 con 0x7fd264103240 2024-04-16T05:34:56.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.031+0000 7fd259ffb640 1 -- 172.21.15.32:0/93259076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd254007c40 con 0x7fd264103240 2024-04-16T05:34:56.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.031+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd264074290 con 0x7fd264103240 2024-04-16T05:34:56.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.031+0000 7fd259ffb640 1 -- 172.21.15.32:0/93259076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254005280 con 0x7fd264103240 2024-04-16T05:34:56.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.031+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2640747e0 con 0x7fd264103240 2024-04-16T05:34:56.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.032+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd264109370 con 0x7fd264103240 2024-04-16T05:34:56.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.034+0000 7fd259ffb640 1 -- 172.21.15.32:0/93259076 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fd25400f030 con 0x7fd264103240 2024-04-16T05:34:56.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.035+0000 7fd259ffb640 1 -- 172.21.15.32:0/93259076 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fd2540d37d0 con 0x7fd264103240 2024-04-16T05:34:56.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.039+0000 7fd259ffb640 1 -- 172.21.15.32:0/93259076 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2540a00f0 con 0x7fd264103240 2024-04-16T05:34:56.162 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.000+0000 7f970cd9f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T05:34:56.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.306+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd264075ee0 con 0x7fd264103240 2024-04-16T05:34:56.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.307+0000 7fd259ffb640 1 -- 172.21.15.32:0/93259076 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fd25409f9f0 con 0x7fd264103240 2024-04-16T05:34:56.305 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103240 msgr2=0x7fd264198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 --2- 172.21.15.32:0/93259076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103240 0x7fd264198820 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd254009b60 tx=0x7fd254007590 comp rx=0 tx=0).stop 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 shutdown_connections 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 --2- 172.21.15.32:0/93259076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264103c00 0x7fd26419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 --2- 172.21.15.32:0/93259076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd264103240 0x7fd264198820 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 --2- 172.21.15.32:0/93259076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd264102050 0x7fd2641982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 >> 172.21.15.32:0/93259076 conn(0x7fd2640fd5e0 msgr2=0x7fd264106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 shutdown_connections 2024-04-16T05:34:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.309+0000 7fd26b11b640 1 -- 172.21.15.32:0/93259076 wait complete. 2024-04-16T05:34:56.319 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 -- 172.21.15.32:0/2304011218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c102ac0 msgr2=0x7f893c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 --2- 172.21.15.32:0/2304011218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c102ac0 0x7f893c104eb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8938009900 tx=0x7f893802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 -- 172.21.15.32:0/2304011218 shutdown_connections 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 --2- 172.21.15.32:0/2304011218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c107d20 0x7f893c10a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 --2- 172.21.15.32:0/2304011218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f893c1053f0 0x7f893c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 --2- 172.21.15.32:0/2304011218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c102ac0 0x7f893c104eb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 -- 172.21.15.32:0/2304011218 >> 172.21.15.32:0/2304011218 conn(0x7f893c0fc4b0 msgr2=0x7f893c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 -- 172.21.15.32:0/2304011218 shutdown_connections 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.434+0000 7f8943d59640 1 -- 172.21.15.32:0/2304011218 wait complete. 2024-04-16T05:34:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f8943d59640 1 Processor -- start 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f8943d59640 1 -- start start 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f8943d59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f893c102ac0 0x7f893c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f8943d59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c1053f0 0x7f893c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f8943d59640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c107d20 0x7f893c19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f8943d59640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f893c10dd30 con 0x7f893c107d20 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f8943d59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f893c10dbb0 con 0x7f893c102ac0 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f8943d59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f893c10deb0 con 0x7f893c1053f0 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f89422f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c107d20 0x7f893c19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f89422f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c107d20 0x7f893c19f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40496/0 (socket says 172.21.15.32:40496) 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f89422f6640 1 -- 172.21.15.32:0/2446813729 learned_addr learned my addr 172.21.15.32:0/2446813729 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.435+0000 7f89412f4640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c1053f0 0x7f893c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f8941af5640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f893c102ac0 0x7f893c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f89422f6640 1 -- 172.21.15.32:0/2446813729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c1053f0 msgr2=0x7f893c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f89422f6640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c1053f0 0x7f893c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f89422f6640 1 -- 172.21.15.32:0/2446813729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f893c102ac0 msgr2=0x7f893c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f89422f6640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f893c102ac0 0x7f893c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f89422f6640 1 -- 172.21.15.32:0/2446813729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c1a0020 con 0x7f893c107d20 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f89422f6640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c107d20 0x7f893c19f9b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f893400cad0 tx=0x7f893400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f8932ffd640 1 -- 172.21.15.32:0/2446813729 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893400ad60 con 0x7f893c107d20 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c1a0310 con 0x7f893c107d20 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f8932ffd640 1 -- 172.21.15.32:0/2446813729 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8934005020 con 0x7f893c107d20 2024-04-16T05:34:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f8932ffd640 1 -- 172.21.15.32:0/2446813729 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893400f6b0 con 0x7f893c107d20 2024-04-16T05:34:56.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.436+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c1a0860 con 0x7f893c107d20 2024-04-16T05:34:56.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.437+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c0fdfa0 con 0x7f893c107d20 2024-04-16T05:34:56.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.439+0000 7f8932ffd640 1 -- 172.21.15.32:0/2446813729 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f8934005560 con 0x7f893c107d20 2024-04-16T05:34:56.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.439+0000 7f8932ffd640 1 -- 172.21.15.32:0/2446813729 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f89340d05c0 con 0x7f893c107d20 2024-04-16T05:34:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.443+0000 7f8932ffd640 1 -- 172.21.15.32:0/2446813729 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8934014030 con 0x7f893c107d20 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966afa9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966afa9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966afa9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966afa9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966afa9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966dfaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966dfaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966dfaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.483 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966dfaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.484 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:34:56.257+0000 7f966dfaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:34:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.644+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f893c062c00 con 0x7f893c107d20 2024-04-16T05:34:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.647+0000 7f8932ffd640 1 -- 172.21.15.32:0/2446813729 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24995 (secure 0 0 0) 0x7f893409cfc0 con 0x7f893c107d20 2024-04-16T05:34:56.645 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:56.646 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:34:56.002698+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1356969979":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6800/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/558122612":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2362160984":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1729232468":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:56.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c107d20 msgr2=0x7f893c19f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c107d20 0x7f893c19f9b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f893400cad0 tx=0x7f893400ced0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 shutdown_connections 2024-04-16T05:34:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c107d20 0x7f893c19f9b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c1053f0 0x7f893c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 --2- 172.21.15.32:0/2446813729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f893c102ac0 0x7f893c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 >> 172.21.15.32:0/2446813729 conn(0x7f893c0fc4b0 msgr2=0x7f893c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 shutdown_connections 2024-04-16T05:34:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.649+0000 7f8943d59640 1 -- 172.21.15.32:0/2446813729 wait complete. 2024-04-16T05:34:56.659 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:56.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2072783862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a981053f0 msgr2=0x7f0a981077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 --2- 172.21.15.32:0/2072783862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a981053f0 0x7f0a981077e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009900 tx=0x7f0a8402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2072783862 shutdown_connections 2024-04-16T05:34:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 --2- 172.21.15.32:0/2072783862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98107d20 0x7f0a9810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 --2- 172.21.15.32:0/2072783862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a981053f0 0x7f0a981077e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 --2- 172.21.15.32:0/2072783862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a98102ac0 0x7f0a98104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2072783862 >> 172.21.15.32:0/2072783862 conn(0x7f0a980fc4b0 msgr2=0x7f0a980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2072783862 shutdown_connections 2024-04-16T05:34:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.771+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2072783862 wait complete. 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9ec28640 1 Processor -- start 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9ec28640 1 -- start start 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9ec28640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a98102ac0 0x7f0a981983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9ec28640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a981053f0 0x7f0a98198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9ec28640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98107d20 0x7f0a9819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9ec28640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810dcd0 con 0x7f0a98102ac0 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9ec28640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810db50 con 0x7f0a981053f0 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9ec28640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810de50 con 0x7f0a98107d20 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9c9c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a98102ac0 0x7f0a981983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9c9c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a98102ac0 0x7f0a981983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40516/0 (socket says 172.21.15.32:40516) 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9c9c4640 1 -- 172.21.15.32:0/2810413533 learned_addr learned my addr 172.21.15.32:0/2810413533 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a97fff640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a981053f0 0x7f0a98198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9d1c5640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98107d20 0x7f0a9819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9c9c4640 1 -- 172.21.15.32:0/2810413533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98107d20 msgr2=0x7f0a9819f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9c9c4640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98107d20 0x7f0a9819f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9c9c4640 1 -- 172.21.15.32:0/2810413533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a981053f0 msgr2=0x7f0a98198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9c9c4640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a981053f0 0x7f0a98198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.772+0000 7f0a9c9c4640 1 -- 172.21.15.32:0/2810413533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a981a0000 con 0x7f0a98102ac0 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.773+0000 7f0a9c9c4640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a98102ac0 0x7f0a981983e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9f0 tx=0x7f0a8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.773+0000 7f0a95ffb640 1 -- 172.21.15.32:0/2810413533 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00cd80 con 0x7f0a98102ac0 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.773+0000 7f0a95ffb640 1 -- 172.21.15.32:0/2810413533 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c010910 con 0x7f0a98102ac0 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.773+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a981a02f0 con 0x7f0a98102ac0 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.773+0000 7f0a95ffb640 1 -- 172.21.15.32:0/2810413533 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c018990 con 0x7f0a98102ac0 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.773+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a981a0860 con 0x7f0a98102ac0 2024-04-16T05:34:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.774+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a980fdfa0 con 0x7f0a98102ac0 2024-04-16T05:34:56.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.776+0000 7f0a95ffb640 1 -- 172.21.15.32:0/2810413533 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f0a8c010ac0 con 0x7f0a98102ac0 2024-04-16T05:34:56.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.777+0000 7f0a95ffb640 1 -- 172.21.15.32:0/2810413533 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f0a8c0d09b0 con 0x7f0a98102ac0 2024-04-16T05:34:56.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.780+0000 7f0a95ffb640 1 -- 172.21.15.32:0/2810413533 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a8c09cc80 con 0x7f0a98102ac0 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3871571320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2069138994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: Active manager daemon x restarted 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: Activating manager daemon x 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: mgrmap e21: x(active, starting, since 0.0118045s), standbys: z, y 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:34:56.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:34:56.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: Manager daemon x is now available 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:34:56.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:34:56.888 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:56 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/93259076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-16T05:34:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3871571320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2069138994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: Active manager daemon x restarted 2024-04-16T05:34:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: Activating manager daemon x 2024-04-16T05:34:56.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: mgrmap e21: x(active, starting, since 0.0118045s), standbys: z, y 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:34:56.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:34:56.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:34:56.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:34:56.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:34:56.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:34:56.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:34:56.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:34:56.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:34:56.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: Manager daemon x is now available 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:34:56.903 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:56 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/93259076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.976+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a98062c00 con 0x7f0a98102ac0 2024-04-16T05:34:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.976+0000 7f0a95ffb640 1 -- 172.21.15.32:0/2810413533 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f0a8c0a0f80 con 0x7f0a98102ac0 2024-04-16T05:34:56.975 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:34:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.978+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a98102ac0 msgr2=0x7f0a981983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.978+0000 7f0a9ec28640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a98102ac0 0x7f0a981983e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9f0 tx=0x7f0a8c00eeb0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.978+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 shutdown_connections 2024-04-16T05:34:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.978+0000 7f0a9ec28640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98107d20 0x7f0a9819f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.978+0000 7f0a9ec28640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a981053f0 0x7f0a98198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.978+0000 7f0a9ec28640 1 --2- 172.21.15.32:0/2810413533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a98102ac0 0x7f0a981983e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.978+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 >> 172.21.15.32:0/2810413533 conn(0x7f0a980fc4b0 msgr2=0x7f0a98104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.978+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 shutdown_connections 2024-04-16T05:34:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:56.979+0000 7f0a9ec28640 1 -- 172.21.15.32:0/2810413533 wait complete. 2024-04-16T05:34:56.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-16T05:34:56.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3871571320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2069138994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: Active manager daemon x restarted 2024-04-16T05:34:56.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: Activating manager daemon x 2024-04-16T05:34:56.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: mgrmap e21: x(active, starting, since 0.0118045s), standbys: z, y 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:34:56.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T05:34:56.985 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: Manager daemon x is now available 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T05:34:56.986 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:56 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/93259076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:56.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:34:57.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 -- 172.21.15.32:0/2088210848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102ae0 msgr2=0x7f70c0104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 --2- 172.21.15.32:0/2088210848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102ae0 0x7f70c0104ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009900 tx=0x7f70b002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:57.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 -- 172.21.15.32:0/2088210848 shutdown_connections 2024-04-16T05:34:57.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 --2- 172.21.15.32:0/2088210848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70c0107d40 0x7f70c010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 --2- 172.21.15.32:0/2088210848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70c0105410 0x7f70c0107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 --2- 172.21.15.32:0/2088210848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102ae0 0x7f70c0104ed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 -- 172.21.15.32:0/2088210848 >> 172.21.15.32:0/2088210848 conn(0x7f70c00fc4f0 msgr2=0x7f70c00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:57.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 -- 172.21.15.32:0/2088210848 shutdown_connections 2024-04-16T05:34:57.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.119+0000 7f70c49a8640 1 -- 172.21.15.32:0/2088210848 wait complete. 2024-04-16T05:34:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.120+0000 7f70c49a8640 1 Processor -- start 2024-04-16T05:34:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.120+0000 7f70c49a8640 1 -- start start 2024-04-16T05:34:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.120+0000 7f70c49a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70c0102ae0 0x7f70c0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.120+0000 7f70c49a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70c0105410 0x7f70c0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.120+0000 7f70c49a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0107d40 0x7f70c019f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.120+0000 7f70c49a8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010dcf0 con 0x7f70c0102ae0 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.120+0000 7f70c49a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010db70 con 0x7f70c0105410 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.120+0000 7f70c49a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010de70 con 0x7f70c0107d40 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70c0102ae0 0x7f70c0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70c0102ae0 0x7f70c0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40538/0 (socket says 172.21.15.32:40538) 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 -- 172.21.15.32:0/53217879 learned_addr learned my addr 172.21.15.32:0/53217879 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70bed9d640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0107d40 0x7f70c019f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70b7fff640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70c0105410 0x7f70c0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 -- 172.21.15.32:0/53217879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0107d40 msgr2=0x7f70c019f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0107d40 0x7f70c019f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 -- 172.21.15.32:0/53217879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70c0105410 msgr2=0x7f70c0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70c0105410 0x7f70c0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 -- 172.21.15.32:0/53217879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c01a0020 con 0x7f70c0102ae0 2024-04-16T05:34:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.121+0000 7f70be59c640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70c0102ae0 0x7f70c0198400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f70b0005ab0 tx=0x7f70b00049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.122+0000 7f70b77fe640 1 -- 172.21.15.32:0/53217879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0041070 con 0x7f70c0102ae0 2024-04-16T05:34:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.122+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c01a02b0 con 0x7f70c0102ae0 2024-04-16T05:34:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.122+0000 7f70b77fe640 1 -- 172.21.15.32:0/53217879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b0004450 con 0x7f70c0102ae0 2024-04-16T05:34:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.122+0000 7f70b77fe640 1 -- 172.21.15.32:0/53217879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0003d90 con 0x7f70c0102ae0 2024-04-16T05:34:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.122+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01a07f0 con 0x7f70c0102ae0 2024-04-16T05:34:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.123+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c00fdfe0 con 0x7f70c0102ae0 2024-04-16T05:34:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.124+0000 7f70b77fe640 1 -- 172.21.15.32:0/53217879 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f70b0044070 con 0x7f70c0102ae0 2024-04-16T05:34:57.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.125+0000 7f70b77fe640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f70ac0a39b0 0x7f70ac0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.125+0000 7f70b77fe640 1 -- 172.21.15.32:0/53217879 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f70b00f20a0 con 0x7f70c0102ae0 2024-04-16T05:34:57.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.125+0000 7f70b7fff640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f70ac0a39b0 0x7f70ac0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.126+0000 7f70b7fff640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f70ac0a39b0 0x7f70ac0a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f70a80095f0 tx=0x7f70a8005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:57.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.133+0000 7f70b77fe640 1 -- 172.21.15.32:0/53217879 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70b00be910 con 0x7f70c0102ae0 2024-04-16T05:34:57.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.320+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70c002d7b0 con 0x7f70c0102ae0 2024-04-16T05:34:57.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.320+0000 7f70b77fe640 1 -- 172.21.15.32:0/53217879 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f70b00be210 con 0x7f70c0102ae0 2024-04-16T05:34:57.319 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f70ac0a39b0 msgr2=0x7f70ac0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f70ac0a39b0 0x7f70ac0a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f70a80095f0 tx=0x7f70a8005f90 comp rx=0 tx=0).stop 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70c0102ae0 msgr2=0x7f70c0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70c0102ae0 0x7f70c0198400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f70b0005ab0 tx=0x7f70b00049c0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 shutdown_connections 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0107d40 0x7f70c019f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70c0105410 0x7f70c0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f70ac0a39b0 0x7f70ac0a5e70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 --2- 172.21.15.32:0/53217879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70c0102ae0 0x7f70c0198400 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 >> 172.21.15.32:0/53217879 conn(0x7f70c00fc4f0 msgr2=0x7f70c0103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 shutdown_connections 2024-04-16T05:34:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.322+0000 7f70c49a8640 1 -- 172.21.15.32:0/53217879 wait complete. 2024-04-16T05:34:57.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:34:57.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.444+0000 7fb6236e0640 1 -- 172.21.15.32:0/3083912417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c105490 msgr2=0x7fb61c107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.444+0000 7fb6236e0640 1 --2- 172.21.15.32:0/3083912417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c105490 0x7fb61c107880 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb610009900 tx=0x7fb61002bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.444+0000 7fb6236e0640 1 -- 172.21.15.32:0/3083912417 shutdown_connections 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.444+0000 7fb6236e0640 1 --2- 172.21.15.32:0/3083912417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c107dc0 0x7fb61c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.444+0000 7fb6236e0640 1 --2- 172.21.15.32:0/3083912417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c105490 0x7fb61c107880 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.444+0000 7fb6236e0640 1 --2- 172.21.15.32:0/3083912417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb61c102b60 0x7fb61c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.444+0000 7fb6236e0640 1 -- 172.21.15.32:0/3083912417 >> 172.21.15.32:0/3083912417 conn(0x7fb61c0fc4b0 msgr2=0x7fb61c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 -- 172.21.15.32:0/3083912417 shutdown_connections 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 -- 172.21.15.32:0/3083912417 wait complete. 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 Processor -- start 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 -- start start 2024-04-16T05:34:57.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c102b60 0x7fb61c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb61c105490 0x7fb61c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c107dc0 0x7fb61c19f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c10dcf0 con 0x7fb61c102b60 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c10db70 con 0x7fb61c105490 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb6236e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c10de70 con 0x7fb61c107dc0 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb62147c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c102b60 0x7fb61c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb62147c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c102b60 0x7fb61c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40556/0 (socket says 172.21.15.32:40556) 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb62147c640 1 -- 172.21.15.32:0/385068334 learned_addr learned my addr 172.21.15.32:0/385068334 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.445+0000 7fb621c7d640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c107dc0 0x7fb61c19f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb620c7b640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb61c105490 0x7fb61c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb62147c640 1 -- 172.21.15.32:0/385068334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c107dc0 msgr2=0x7fb61c19f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb62147c640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c107dc0 0x7fb61c19f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb62147c640 1 -- 172.21.15.32:0/385068334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb61c105490 msgr2=0x7fb61c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb62147c640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb61c105490 0x7fb61c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb62147c640 1 -- 172.21.15.32:0/385068334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c19ffd0 con 0x7fb61c102b60 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb621c7d640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c107dc0 0x7fb61c19f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb62147c640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c102b60 0x7fb61c1983b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb61800ea60 tx=0x7fb61800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb60e7fc640 1 -- 172.21.15.32:0/385068334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61800cde0 con 0x7fb61c102b60 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb60e7fc640 1 -- 172.21.15.32:0/385068334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6180035f0 con 0x7fb61c102b60 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61c1a02c0 con 0x7fb61c102b60 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb60e7fc640 1 -- 172.21.15.32:0/385068334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618010670 con 0x7fb61c102b60 2024-04-16T05:34:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.446+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61c1a07e0 con 0x7fb61c102b60 2024-04-16T05:34:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.447+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb61c0fdfa0 con 0x7fb61c102b60 2024-04-16T05:34:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.449+0000 7fb60e7fc640 1 -- 172.21.15.32:0/385068334 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fb6180037a0 con 0x7fb61c102b60 2024-04-16T05:34:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.450+0000 7fb60e7fc640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6080a39b0 0x7fb6080a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.450+0000 7fb620c7b640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6080a39b0 0x7fb6080a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.450+0000 7fb60e7fc640 1 -- 172.21.15.32:0/385068334 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fb6180d0710 con 0x7fb61c102b60 2024-04-16T05:34:57.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.450+0000 7fb620c7b640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6080a39b0 0x7fb6080a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb610000c00 tx=0x7fb610005fd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:57.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.454+0000 7fb60e7fc640 1 -- 172.21.15.32:0/385068334 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb61809cf80 con 0x7fb61c102b60 2024-04-16T05:34:57.599 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2446813729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:57.600 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2810413533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:57.600 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:57 smithi032 ceph-mon[32351]: mgrmap e22: x(active, since 1.0186s), standbys: z, y 2024-04-16T05:34:57.600 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/53217879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:57.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.643+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb61c062c00 con 0x7fb61c102b60 2024-04-16T05:34:57.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.644+0000 7fb60e7fc640 1 -- 172.21.15.32:0/385068334 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fb61809c880 con 0x7fb61c102b60 2024-04-16T05:34:57.642 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:34:57.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6080a39b0 msgr2=0x7fb6080a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6080a39b0 0x7fb6080a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb610000c00 tx=0x7fb610005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c102b60 msgr2=0x7fb61c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c102b60 0x7fb61c1983b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb61800ea60 tx=0x7fb61800ef20 comp rx=0 tx=0).stop 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 shutdown_connections 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c107dc0 0x7fb61c19f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb61c105490 0x7fb61c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6080a39b0 0x7fb6080a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 --2- 172.21.15.32:0/385068334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c102b60 0x7fb61c1983b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 >> 172.21.15.32:0/385068334 conn(0x7fb61c0fc4b0 msgr2=0x7fb61c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 shutdown_connections 2024-04-16T05:34:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.646+0000 7fb6236e0640 1 -- 172.21.15.32:0/385068334 wait complete. 2024-04-16T05:34:57.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 -- 172.21.15.32:0/2940912520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f28100330 msgr2=0x7f4f28100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/2940912520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f28100330 0x7f4f28100750 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009930 tx=0x7f4f1802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 -- 172.21.15.32:0/2940912520 shutdown_connections 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/2940912520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f28103d60 0x7f4f2810a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/2940912520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f281033a0 0x7f4f28103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/2940912520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f28100330 0x7f4f28100750 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 -- 172.21.15.32:0/2940912520 >> 172.21.15.32:0/2940912520 conn(0x7f4f280fbc40 msgr2=0x7f4f280fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 -- 172.21.15.32:0/2940912520 shutdown_connections 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.767+0000 7f4f2dde1640 1 -- 172.21.15.32:0/2940912520 wait complete. 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f2dde1640 1 Processor -- start 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f2dde1640 1 -- start start 2024-04-16T05:34:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f2dde1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f28100330 0x7f4f281983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f2dde1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f281033a0 0x7f4f281988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f2dde1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f28103d60 0x7f4f2819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f2dde1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2810dcb0 con 0x7f4f281033a0 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f2dde1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2810db30 con 0x7f4f28100330 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f2dde1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2810de30 con 0x7f4f28103d60 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f26ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f281033a0 0x7f4f281988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f26ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f281033a0 0x7f4f281988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40572/0 (socket says 172.21.15.32:40572) 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f26ffd640 1 -- 172.21.15.32:0/3625758070 learned_addr learned my addr 172.21.15.32:0/3625758070 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f27fff640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f28103d60 0x7f4f2819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f277fe640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f28100330 0x7f4f281983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f26ffd640 1 -- 172.21.15.32:0/3625758070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f28103d60 msgr2=0x7f4f2819f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f26ffd640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f28103d60 0x7f4f2819f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f26ffd640 1 -- 172.21.15.32:0/3625758070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f28100330 msgr2=0x7f4f281983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f26ffd640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f28100330 0x7f4f281983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.768+0000 7f4f26ffd640 1 -- 172.21.15.32:0/3625758070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2819ffd0 con 0x7f4f281033a0 2024-04-16T05:34:57.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.769+0000 7f4f26ffd640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f281033a0 0x7f4f281988f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00d950 tx=0x7f4f1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:57.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.769+0000 7f4f24ff9640 1 -- 172.21.15.32:0/3625758070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c017070 con 0x7f4f281033a0 2024-04-16T05:34:57.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.769+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f281a02c0 con 0x7f4f281033a0 2024-04-16T05:34:57.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.769+0000 7f4f24ff9640 1 -- 172.21.15.32:0/3625758070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c009c30 con 0x7f4f281033a0 2024-04-16T05:34:57.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.769+0000 7f4f24ff9640 1 -- 172.21.15.32:0/3625758070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c00f6b0 con 0x7f4f281033a0 2024-04-16T05:34:57.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.769+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f281a0860 con 0x7f4f281033a0 2024-04-16T05:34:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.770+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f281095a0 con 0x7f4f281033a0 2024-04-16T05:34:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.771+0000 7f4f24ff9640 1 -- 172.21.15.32:0/3625758070 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f4f1c0109d0 con 0x7f4f281033a0 2024-04-16T05:34:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.772+0000 7f4f24ff9640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4f100a39b0 0x7f4f100a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.772+0000 7f4f277fe640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4f100a39b0 0x7f4f100a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.772+0000 7f4f24ff9640 1 -- 172.21.15.32:0/3625758070 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4f1c0cffe0 con 0x7f4f281033a0 2024-04-16T05:34:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.772+0000 7f4f277fe640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4f100a39b0 0x7f4f100a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4f1802c7b0 tx=0x7f4f18005fd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:57.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.777+0000 7f4f24ff9640 1 -- 172.21.15.32:0/3625758070 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f1c009de0 con 0x7f4f281033a0 2024-04-16T05:34:57.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:57 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2446813729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:57.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:57 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2810413533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:57.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:57 smithi134 ceph-mon[36153]: mgrmap e22: x(active, since 1.0186s), standbys: z, y 2024-04-16T05:34:57.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:57 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/53217879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:57.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:57 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2446813729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:34:57.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:57 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2810413533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:34:57.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:57 smithi082 ceph-mon[36195]: mgrmap e22: x(active, since 1.0186s), standbys: z, y 2024-04-16T05:34:57.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:57 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/53217879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:34:57.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.971+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4f28062c00 con 0x7f4f281033a0 2024-04-16T05:34:57.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.972+0000 7f4f24ff9640 1 -- 172.21.15.32:0/3625758070 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f4f1c09c200 con 0x7f4f281033a0 2024-04-16T05:34:57.971 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:34:57.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4f100a39b0 msgr2=0x7f4f100a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4f100a39b0 0x7f4f100a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4f1802c7b0 tx=0x7f4f18005fd0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f281033a0 msgr2=0x7f4f281988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:57.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f281033a0 0x7f4f281988f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00d950 tx=0x7f4f1c00de10 comp rx=0 tx=0).stop 2024-04-16T05:34:57.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 shutdown_connections 2024-04-16T05:34:57.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f28103d60 0x7f4f2819f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4f100a39b0 0x7f4f100a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f281033a0 0x7f4f281988f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 --2- 172.21.15.32:0/3625758070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f28100330 0x7f4f281983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:57.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 >> 172.21.15.32:0/3625758070 conn(0x7f4f280fbc40 msgr2=0x7f4f28106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:57.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.974+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 shutdown_connections 2024-04-16T05:34:57.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:57.975+0000 7f4f2dde1640 1 -- 172.21.15.32:0/3625758070 wait complete. 2024-04-16T05:34:57.985 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-16T05:34:57.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:34:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 -- 172.21.15.32:0/913474506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50100f80 msgr2=0x7f5a501013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 --2- 172.21.15.32:0/913474506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50100f80 0x7f5a501013a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009900 tx=0x7f5a4c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 -- 172.21.15.32:0/913474506 shutdown_connections 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 --2- 172.21.15.32:0/913474506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a50102b30 0x7f5a501093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 --2- 172.21.15.32:0/913474506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a50102170 0x7f5a501025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 --2- 172.21.15.32:0/913474506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50100f80 0x7f5a501013a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 -- 172.21.15.32:0/913474506 >> 172.21.15.32:0/913474506 conn(0x7f5a500fc4f0 msgr2=0x7f5a500fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 -- 172.21.15.32:0/913474506 shutdown_connections 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.110+0000 7f5a57782640 1 -- 172.21.15.32:0/913474506 wait complete. 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a57782640 1 Processor -- start 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a57782640 1 -- start start 2024-04-16T05:34:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a57782640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a50100f80 0x7f5a501984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a57782640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a50102170 0x7f5a501989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a57782640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50102b30 0x7f5a5019f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a57782640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5010cc60 con 0x7f5a50102170 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a57782640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5010cae0 con 0x7f5a50102b30 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a57782640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5010cde0 con 0x7f5a50100f80 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a54d1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a50102170 0x7f5a501989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a5551e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a50100f80 0x7f5a501984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a5551e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a50100f80 0x7f5a501984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50570/0 (socket says 172.21.15.32:50570) 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a54d1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a50102170 0x7f5a501989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40610/0 (socket says 172.21.15.32:40610) 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a5551e640 1 -- 172.21.15.32:0/1910072636 learned_addr learned my addr 172.21.15.32:0/1910072636 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a55d1f640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50102b30 0x7f5a5019f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a5551e640 1 -- 172.21.15.32:0/1910072636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50102b30 msgr2=0x7f5a5019f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a5551e640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50102b30 0x7f5a5019f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a5551e640 1 -- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a50102170 msgr2=0x7f5a501989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a5551e640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a50102170 0x7f5a501989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a5551e640 1 -- 172.21.15.32:0/1910072636 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a501a0050 con 0x7f5a50100f80 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.111+0000 7f5a55d1f640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50102b30 0x7f5a5019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.112+0000 7f5a5551e640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a50100f80 0x7f5a501984a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c000c00 tx=0x7f5a4c004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.112+0000 7f5a467fc640 1 -- 172.21.15.32:0/1910072636 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c043020 con 0x7f5a50100f80 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.112+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a501a02e0 con 0x7f5a50100f80 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.112+0000 7f5a467fc640 1 -- 172.21.15.32:0/1910072636 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a4c0054e0 con 0x7f5a50100f80 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.112+0000 7f5a467fc640 1 -- 172.21.15.32:0/1910072636 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c003bd0 con 0x7f5a50100f80 2024-04-16T05:34:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.112+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a501a0850 con 0x7f5a50100f80 2024-04-16T05:34:58.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.113+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a501a0aa0 con 0x7f5a50100f80 2024-04-16T05:34:58.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.114+0000 7f5a467fc640 1 -- 172.21.15.32:0/1910072636 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5a4c046020 con 0x7f5a50100f80 2024-04-16T05:34:58.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.115+0000 7f5a467fc640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a3c0a3a30 0x7f5a3c0a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.115+0000 7f5a467fc640 1 -- 172.21.15.32:0/1910072636 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f5a4c0f2730 con 0x7f5a50100f80 2024-04-16T05:34:58.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.115+0000 7f5a54d1d640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a3c0a3a30 0x7f5a3c0a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.115+0000 7f5a54d1d640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a3c0a3a30 0x7f5a3c0a5ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5a400095f0 tx=0x7f5a40005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:58.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.118+0000 7f5a467fc640 1 -- 172.21.15.32:0/1910072636 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a4c04d020 con 0x7f5a50100f80 2024-04-16T05:34:58.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.365+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a501a0c70 con 0x7f5a50100f80 2024-04-16T05:34:58.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.366+0000 7f5a467fc640 1 -- 172.21.15.32:0/1910072636 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f5a4c0bef60 con 0x7f5a50100f80 2024-04-16T05:34:58.364 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:58.365 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:34:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a3c0a3a30 msgr2=0x7f5a3c0a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a3c0a3a30 0x7f5a3c0a5ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5a400095f0 tx=0x7f5a40005f90 comp rx=0 tx=0).stop 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a50100f80 msgr2=0x7f5a501984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a50100f80 0x7f5a501984a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c000c00 tx=0x7f5a4c004970 comp rx=0 tx=0).stop 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 shutdown_connections 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a50102b30 0x7f5a5019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a3c0a3a30 0x7f5a3c0a5ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a50102170 0x7f5a501989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 --2- 172.21.15.32:0/1910072636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a50100f80 0x7f5a501984a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 >> 172.21.15.32:0/1910072636 conn(0x7f5a500fc4f0 msgr2=0x7f5a500fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 shutdown_connections 2024-04-16T05:34:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.368+0000 7f5a57782640 1 -- 172.21.15.32:0/1910072636 wait complete. 2024-04-16T05:34:58.378 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-16T05:34:58.378 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-16T05:34:58.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 -- 172.21.15.32:0/3700873882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 msgr2=0x7f9744101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 --2- 172.21.15.32:0/3700873882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 0x7f9744101360 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9734009900 tx=0x7f973402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 -- 172.21.15.32:0/3700873882 shutdown_connections 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 --2- 172.21.15.32:0/3700873882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9744102a50 0x7f9744109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 --2- 172.21.15.32:0/3700873882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9744102090 0x7f9744102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 --2- 172.21.15.32:0/3700873882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 0x7f9744101360 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 -- 172.21.15.32:0/3700873882 >> 172.21.15.32:0/3700873882 conn(0x7f97440fc4b0 msgr2=0x7f97440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 -- 172.21.15.32:0/3700873882 shutdown_connections 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.487+0000 7f9748841640 1 -- 172.21.15.32:0/3700873882 wait complete. 2024-04-16T05:34:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9748841640 1 Processor -- start 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9748841640 1 -- start start 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9748841640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 0x7f97441972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9748841640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9744102090 0x7f97441977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9748841640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9744102a50 0x7f974419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9748841640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f974410cbf0 con 0x7f9744100f40 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9748841640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f974410ca70 con 0x7f9744102a50 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9748841640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f974410cd70 con 0x7f9744102090 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f974259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 0x7f97441972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f974259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 0x7f97441972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35710/0 (socket says 172.21.15.32:35710) 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f974259c640 1 -- 172.21.15.32:0/1980212350 learned_addr learned my addr 172.21.15.32:0/1980212350 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f9742d9d640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9744102a50 0x7f974419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f973bfff640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9744102090 0x7f97441977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f974259c640 1 -- 172.21.15.32:0/1980212350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9744102090 msgr2=0x7f97441977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f974259c640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9744102090 0x7f97441977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f974259c640 1 -- 172.21.15.32:0/1980212350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9744102a50 msgr2=0x7f974419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f974259c640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9744102a50 0x7f974419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f974259c640 1 -- 172.21.15.32:0/1980212350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974419eec0 con 0x7f9744100f40 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.488+0000 7f973bfff640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9744102090 0x7f97441977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.489+0000 7f974259c640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 0x7f97441972a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9734009230 tx=0x7f973402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.489+0000 7f973b7fe640 1 -- 172.21.15.32:0/1980212350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734041070 con 0x7f9744100f40 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.489+0000 7f973b7fe640 1 -- 172.21.15.32:0/1980212350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9734004450 con 0x7f9744100f40 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.489+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974419f150 con 0x7f9744100f40 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.489+0000 7f973b7fe640 1 -- 172.21.15.32:0/1980212350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734003e20 con 0x7f9744100f40 2024-04-16T05:34:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.489+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974419f640 con 0x7f9744100f40 2024-04-16T05:34:58.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.490+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9744108250 con 0x7f9744100f40 2024-04-16T05:34:58.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.491+0000 7f973b7fe640 1 -- 172.21.15.32:0/1980212350 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f9734044070 con 0x7f9744100f40 2024-04-16T05:34:58.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.492+0000 7f973b7fe640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f97300a39b0 0x7f97300a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.492+0000 7f973b7fe640 1 -- 172.21.15.32:0/1980212350 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f97340f2070 con 0x7f9744100f40 2024-04-16T05:34:58.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.492+0000 7f973bfff640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f97300a39b0 0x7f97300a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.493+0000 7f973bfff640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f97300a39b0 0x7f97300a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f97441987b0 tx=0x7f972c005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:58.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.496+0000 7f973b7fe640 1 -- 172.21.15.32:0/1980212350 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97340be8a0 con 0x7f9744100f40 2024-04-16T05:34:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.578+0000 7f973b7fe640 1 -- 172.21.15.32:0/1980212350 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f97340035f0 con 0x7f9744100f40 2024-04-16T05:34:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.687+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9744062440 con 0x7f97300a39b0 2024-04-16T05:34:58.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.697+0000 7f973b7fe640 1 -- 172.21.15.32:0/1980212350 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339709 (secure 0 0 0) 0x7f9744062440 con 0x7f97300a39b0 2024-04-16T05:34:58.700 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:58.702 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-16T05:34:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f97300a39b0 msgr2=0x7f97300a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f97300a39b0 0x7f97300a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f97441987b0 tx=0x7f972c005f90 comp rx=0 tx=0).stop 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 msgr2=0x7f97441972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 0x7f97441972a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9734009230 tx=0x7f973402c650 comp rx=0 tx=0).stop 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 shutdown_connections 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9744102a50 0x7f974419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9744102090 0x7f97441977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f97300a39b0 0x7f97300a5e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 --2- 172.21.15.32:0/1980212350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9744100f40 0x7f97441972a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 >> 172.21.15.32:0/1980212350 conn(0x7f97440fc4b0 msgr2=0x7f9744105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 shutdown_connections 2024-04-16T05:34:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.700+0000 7f9748841640 1 -- 172.21.15.32:0/1980212350 wait complete. 2024-04-16T05:34:58.709 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: [16/Apr/2024:05:34:57] ENGINE Bus STARTING 2024-04-16T05:34:58.709 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: [16/Apr/2024:05:34:57] ENGINE Serving on http://172.21.15.32:8765 2024-04-16T05:34:58.709 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: [16/Apr/2024:05:34:57] ENGINE Serving on https://172.21.15.32:7150 2024-04-16T05:34:58.709 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: [16/Apr/2024:05:34:57] ENGINE Bus STARTED 2024-04-16T05:34:58.709 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/385068334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:58.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3625758070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:58.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.710 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1910072636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:58.711 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-16T05:34:58.020779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188579,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":994,"num_read_kb":660,"num_write":472,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361596,"kb_used_data":12396,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580228,"statfs":{"total":1151940427776,"available":1151570153472,"internally_reserved":0,"allocated":12693504,"data_stored":9265417,"data_compressed":450707,"data_compressed_allocated":610304,"data_compressed_original":3735552,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.987473"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.640068+0000","last_change":"2024-04-16T05:32:34.759518+0000","last_active":"2024-04-16T05:34:56.640068+0000","last_peered":"2024-04-16T05:34:56.640068+0000","last_clean":"2024-04-16T05:34:56.640068+0000","last_became_active":"2024-04-16T05:32:34.759155+0000","last_became_peered":"2024-04-16T05:32:34.759155+0000","last_unstale":"2024-04-16T05:34:56.640068+0000","last_undegraded":"2024-04-16T05:34:56.640068+0000","last_fullsized":"2024-04-16T05:34:56.640068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:39.935886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266968+0000","last_change":"2024-04-16T05:32:34.758553+0000","last_active":"2024-04-16T05:34:56.266968+0000","last_peered":"2024-04-16T05:34:56.266968+0000","last_clean":"2024-04-16T05:34:56.266968+0000","last_became_active":"2024-04-16T05:32:34.758242+0000","last_became_peered":"2024-04-16T05:32:34.758242+0000","last_unstale":"2024-04-16T05:34:56.266968+0000","last_undegraded":"2024-04-16T05:34:56.266968+0000","last_fullsized":"2024-04-16T05:34:56.266968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:09.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266926+0000","last_change":"2024-04-16T05:32:34.754815+0000","last_active":"2024-04-16T05:34:56.266926+0000","last_peered":"2024-04-16T05:34:56.266926+0000","last_clean":"2024-04-16T05:34:56.266926+0000","last_became_active":"2024-04-16T05:32:34.754488+0000","last_became_peered":"2024-04-16T05:32:34.754488+0000","last_unstale":"2024-04-16T05:34:56.266926+0000","last_undegraded":"2024-04-16T05:34:56.266926+0000","last_fullsized":"2024-04-16T05:34:56.266926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:10:32.379770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007584+0000","last_change":"2024-04-16T05:33:34.766742+0000","last_active":"2024-04-16T05:34:56.007584+0000","last_peered":"2024-04-16T05:34:56.007584+0000","last_clean":"2024-04-16T05:34:56.007584+0000","last_became_active":"2024-04-16T05:33:34.766332+0000","last_became_peered":"2024-04-16T05:33:34.766332+0000","last_unstale":"2024-04-16T05:34:56.007584+0000","last_undegraded":"2024-04-16T05:34:56.007584+0000","last_fullsized":"2024-04-16T05:34:56.007584+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:52:14.644496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008710+0000","last_change":"2024-04-16T05:32:34.755623+0000","last_active":"2024-04-16T05:34:56.008710+0000","last_peered":"2024-04-16T05:34:56.008710+0000","last_clean":"2024-04-16T05:34:56.008710+0000","last_became_active":"2024-04-16T05:32:34.754205+0000","last_became_peered":"2024-04-16T05:32:34.754205+0000","last_unstale":"2024-04-16T05:34:56.008710+0000","last_undegraded":"2024-04-16T05:34:56.008710+0000","last_fullsized":"2024-04-16T05:34:56.008710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:34:07.998734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117661+0000","last_change":"2024-04-16T05:32:34.753767+0000","last_active":"2024-04-16T05:34:56.117661+0000","last_peered":"2024-04-16T05:34:56.117661+0000","last_clean":"2024-04-16T05:34:56.117661+0000","last_became_active":"2024-04-16T05:32:34.753448+0000","last_became_peered":"2024-04-16T05:32:34.753448+0000","last_unstale":"2024-04-16T05:34:56.117661+0000","last_undegraded":"2024-04-16T05:34:56.117661+0000","last_fullsized":"2024-04-16T05:34:56.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:27:42.798615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058759+0000","last_change":"2024-04-16T05:32:34.754049+0000","last_active":"2024-04-16T05:34:56.058759+0000","last_peered":"2024-04-16T05:34:56.058759+0000","last_clean":"2024-04-16T05:34:56.058759+0000","last_became_active":"2024-04-16T05:32:34.753757+0000","last_became_peered":"2024-04-16T05:32:34.753757+0000","last_unstale":"2024-04-16T05:34:56.058759+0000","last_undegraded":"2024-04-16T05:34:56.058759+0000","last_fullsized":"2024-04-16T05:34:56.058759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:42:26.747100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105386+0000","last_change":"2024-04-16T05:32:34.762950+0000","last_active":"2024-04-16T05:34:56.105386+0000","last_peered":"2024-04-16T05:34:56.105386+0000","last_clean":"2024-04-16T05:34:56.105386+0000","last_became_active":"2024-04-16T05:32:34.762073+0000","last_became_peered":"2024-04-16T05:32:34.762073+0000","last_unstale":"2024-04-16T05:34:56.105386+0000","last_undegraded":"2024-04-16T05:34:56.105386+0000","last_fullsized":"2024-04-16T05:34:56.105386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:57:26.623655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265111+0000","last_change":"2024-04-16T05:32:34.751027+0000","last_active":"2024-04-16T05:34:56.265111+0000","last_peered":"2024-04-16T05:34:56.265111+0000","last_clean":"2024-04-16T05:34:56.265111+0000","last_became_active":"2024-04-16T05:32:34.750697+0000","last_became_peered":"2024-04-16T05:32:34.750697+0000","last_unstale":"2024-04-16T05:34:56.265111+0000","last_undegraded":"2024-04-16T05:34:56.265111+0000","last_fullsized":"2024-04-16T05:34:56.265111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:39:33.834972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.118031+0000","last_change":"2024-04-16T05:32:34.752293+0000","last_active":"2024-04-16T05:34:56.118031+0000","last_peered":"2024-04-16T05:34:56.118031+0000","last_clean":"2024-04-16T05:34:56.118031+0000","last_became_active":"2024-04-16T05:32:34.752008+0000","last_became_peered":"2024-04-16T05:32:34.752008+0000","last_unstale":"2024-04-16T05:34:56.118031+0000","last_undegraded":"2024-04-16T05:34:56.118031+0000","last_fullsized":"2024-04-16T05:34:56.118031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:02:17.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059051+0000","last_change":"2024-04-16T05:32:34.760961+0000","last_active":"2024-04-16T05:34:56.059051+0000","last_peered":"2024-04-16T05:34:56.059051+0000","last_clean":"2024-04-16T05:34:56.059051+0000","last_became_active":"2024-04-16T05:32:34.760640+0000","last_became_peered":"2024-04-16T05:32:34.760640+0000","last_unstale":"2024-04-16T05:34:56.059051+0000","last_undegraded":"2024-04-16T05:34:56.059051+0000","last_fullsized":"2024-04-16T05:34:56.059051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:47:28.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009246+0000","last_change":"2024-04-16T05:32:34.748646+0000","last_active":"2024-04-16T05:34:56.009246+0000","last_peered":"2024-04-16T05:34:56.009246+0000","last_clean":"2024-04-16T05:34:56.009246+0000","last_became_active":"2024-04-16T05:32:34.748090+0000","last_became_peered":"2024-04-16T05:32:34.748090+0000","last_unstale":"2024-04-16T05:34:56.009246+0000","last_undegraded":"2024-04-16T05:34:56.009246+0000","last_fullsized":"2024-04-16T05:34:56.009246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:00:19.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266536+0000","last_change":"2024-04-16T05:32:34.747025+0000","last_active":"2024-04-16T05:34:56.266536+0000","last_peered":"2024-04-16T05:34:56.266536+0000","last_clean":"2024-04-16T05:34:56.266536+0000","last_became_active":"2024-04-16T05:32:34.746711+0000","last_became_peered":"2024-04-16T05:32:34.746711+0000","last_unstale":"2024-04-16T05:34:56.266536+0000","last_undegraded":"2024-04-16T05:34:56.266536+0000","last_fullsized":"2024-04-16T05:34:56.266536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:40:30.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266616+0000","last_change":"2024-04-16T05:32:34.756756+0000","last_active":"2024-04-16T05:34:56.266616+0000","last_peered":"2024-04-16T05:34:56.266616+0000","last_clean":"2024-04-16T05:34:56.266616+0000","last_became_active":"2024-04-16T05:32:34.756442+0000","last_became_peered":"2024-04-16T05:32:34.756442+0000","last_unstale":"2024-04-16T05:34:56.266616+0000","last_undegraded":"2024-04-16T05:34:56.266616+0000","last_fullsized":"2024-04-16T05:34:56.266616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:33.864944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266459+0000","last_change":"2024-04-16T05:32:34.765443+0000","last_active":"2024-04-16T05:34:56.266459+0000","last_peered":"2024-04-16T05:34:56.266459+0000","last_clean":"2024-04-16T05:34:56.266459+0000","last_became_active":"2024-04-16T05:32:34.765133+0000","last_became_peered":"2024-04-16T05:32:34.765133+0000","last_unstale":"2024-04-16T05:34:56.266459+0000","last_undegraded":"2024-04-16T05:34:56.266459+0000","last_fullsized":"2024-04-16T05:34:56.266459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:31.452184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264372+0000","last_change":"2024-04-16T05:32:34.757577+0000","last_active":"2024-04-16T05:34:56.264372+0000","last_peered":"2024-04-16T05:34:56.264372+0000","last_clean":"2024-04-16T05:34:56.264372+0000","last_became_active":"2024-04-16T05:32:34.753451+0000","last_became_peered":"2024-04-16T05:32:34.753451+0000","last_unstale":"2024-04-16T05:34:56.264372+0000","last_undegraded":"2024-04-16T05:34:56.264372+0000","last_fullsized":"2024-04-16T05:34:56.264372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:56.617824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639970+0000","last_change":"2024-04-16T05:32:34.759776+0000","last_active":"2024-04-16T05:34:56.639970+0000","last_peered":"2024-04-16T05:34:56.639970+0000","last_clean":"2024-04-16T05:34:56.639970+0000","last_became_active":"2024-04-16T05:32:34.759371+0000","last_became_peered":"2024-04-16T05:32:34.759371+0000","last_unstale":"2024-04-16T05:34:56.639970+0000","last_undegraded":"2024-04-16T05:34:56.639970+0000","last_fullsized":"2024-04-16T05:34:56.639970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:36.458515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007873+0000","last_change":"2024-04-16T05:32:34.747803+0000","last_active":"2024-04-16T05:34:56.007873+0000","last_peered":"2024-04-16T05:34:56.007873+0000","last_clean":"2024-04-16T05:34:56.007873+0000","last_became_active":"2024-04-16T05:32:34.747472+0000","last_became_peered":"2024-04-16T05:32:34.747472+0000","last_unstale":"2024-04-16T05:34:56.007873+0000","last_undegraded":"2024-04-16T05:34:56.007873+0000","last_fullsized":"2024-04-16T05:34:56.007873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:41.118574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264411+0000","last_change":"2024-04-16T05:32:34.754151+0000","last_active":"2024-04-16T05:34:56.264411+0000","last_peered":"2024-04-16T05:34:56.264411+0000","last_clean":"2024-04-16T05:34:56.264411+0000","last_became_active":"2024-04-16T05:32:34.753814+0000","last_became_peered":"2024-04-16T05:32:34.753814+0000","last_unstale":"2024-04-16T05:34:56.264411+0000","last_undegraded":"2024-04-16T05:34:56.264411+0000","last_fullsized":"2024-04-16T05:34:56.264411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:13:00.207978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264276+0000","last_change":"2024-04-16T05:32:38.938563+0000","last_active":"2024-04-16T05:34:56.264276+0000","last_peered":"2024-04-16T05:34:56.264276+0000","last_clean":"2024-04-16T05:34:56.264276+0000","last_became_active":"2024-04-16T05:32:38.938097+0000","last_became_peered":"2024-04-16T05:32:38.938097+0000","last_unstale":"2024-04-16T05:34:56.264276+0000","last_undegraded":"2024-04-16T05:34:56.264276+0000","last_fullsized":"2024-04-16T05:34:56.264276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:47:52.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007623+0000","last_change":"2024-04-16T05:32:33.735694+0000","last_active":"2024-04-16T05:34:56.007623+0000","last_peered":"2024-04-16T05:34:56.007623+0000","last_clean":"2024-04-16T05:34:56.007623+0000","last_became_active":"2024-04-16T05:32:33.735414+0000","last_became_peered":"2024-04-16T05:32:33.735414+0000","last_unstale":"2024-04-16T05:34:56.007623+0000","last_undegraded":"2024-04-16T05:34:56.007623+0000","last_fullsized":"2024-04-16T05:34:56.007623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:19:06.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107049+0000","last_change":"2024-04-16T05:32:34.762315+0000","last_active":"2024-04-16T05:34:56.107049+0000","last_peered":"2024-04-16T05:34:56.107049+0000","last_clean":"2024-04-16T05:34:56.107049+0000","last_became_active":"2024-04-16T05:32:34.761969+0000","last_became_peered":"2024-04-16T05:32:34.761969+0000","last_unstale":"2024-04-16T05:34:56.107049+0000","last_undegraded":"2024-04-16T05:34:56.107049+0000","last_fullsized":"2024-04-16T05:34:56.107049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:00:45.166966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008832+0000","last_change":"2024-04-16T05:32:38.940681+0000","last_active":"2024-04-16T05:34:56.008832+0000","last_peered":"2024-04-16T05:34:56.008832+0000","last_clean":"2024-04-16T05:34:56.008832+0000","last_became_active":"2024-04-16T05:32:38.940414+0000","last_became_peered":"2024-04-16T05:32:38.940414+0000","last_unstale":"2024-04-16T05:34:56.008832+0000","last_undegraded":"2024-04-16T05:34:56.008832+0000","last_fullsized":"2024-04-16T05:34:56.008832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:46:50.061607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107418+0000","last_change":"2024-04-16T05:32:33.748333+0000","last_active":"2024-04-16T05:34:56.107418+0000","last_peered":"2024-04-16T05:34:56.107418+0000","last_clean":"2024-04-16T05:34:56.107418+0000","last_became_active":"2024-04-16T05:32:33.748022+0000","last_became_peered":"2024-04-16T05:32:33.748022+0000","last_unstale":"2024-04-16T05:34:56.107418+0000","last_undegraded":"2024-04-16T05:34:56.107418+0000","last_fullsized":"2024-04-16T05:34:56.107418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:50:27.771245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266977+0000","last_change":"2024-04-16T05:32:34.764832+0000","last_active":"2024-04-16T05:34:56.266977+0000","last_peered":"2024-04-16T05:34:56.266977+0000","last_clean":"2024-04-16T05:34:56.266977+0000","last_became_active":"2024-04-16T05:32:34.763180+0000","last_became_peered":"2024-04-16T05:32:34.763180+0000","last_unstale":"2024-04-16T05:34:56.266977+0000","last_undegraded":"2024-04-16T05:34:56.266977+0000","last_fullsized":"2024-04-16T05:34:56.266977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:37:27.804376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009531+0000","last_change":"2024-04-16T05:32:38.937226+0000","last_active":"2024-04-16T05:34:56.009531+0000","last_peered":"2024-04-16T05:34:56.009531+0000","last_clean":"2024-04-16T05:34:56.009531+0000","last_became_active":"2024-04-16T05:32:38.936770+0000","last_became_peered":"2024-04-16T05:32:38.936770+0000","last_unstale":"2024-04-16T05:34:56.009531+0000","last_undegraded":"2024-04-16T05:34:56.009531+0000","last_fullsized":"2024-04-16T05:34:56.009531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:05:28.059175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639811+0000","last_change":"2024-04-16T05:33:34.775689+0000","last_active":"2024-04-16T05:34:56.639811+0000","last_peered":"2024-04-16T05:34:56.639811+0000","last_clean":"2024-04-16T05:34:56.639811+0000","last_became_active":"2024-04-16T05:33:34.770520+0000","last_became_peered":"2024-04-16T05:33:34.770520+0000","last_unstale":"2024-04-16T05:34:56.639811+0000","last_undegraded":"2024-04-16T05:34:56.639811+0000","last_fullsized":"2024-04-16T05:34:56.639811+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:40:57.616795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264658+0000","last_change":"2024-04-16T05:32:34.758891+0000","last_active":"2024-04-16T05:34:56.264658+0000","last_peered":"2024-04-16T05:34:56.264658+0000","last_clean":"2024-04-16T05:34:56.264658+0000","last_became_active":"2024-04-16T05:32:34.758559+0000","last_became_peered":"2024-04-16T05:32:34.758559+0000","last_unstale":"2024-04-16T05:34:56.264658+0000","last_undegraded":"2024-04-16T05:34:56.264658+0000","last_fullsized":"2024-04-16T05:34:56.264658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:31.213992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008864+0000","last_change":"2024-04-16T05:32:38.937163+0000","last_active":"2024-04-16T05:34:56.008864+0000","last_peered":"2024-04-16T05:34:56.008864+0000","last_clean":"2024-04-16T05:34:56.008864+0000","last_became_active":"2024-04-16T05:32:38.936745+0000","last_became_peered":"2024-04-16T05:32:38.936745+0000","last_unstale":"2024-04-16T05:34:56.008864+0000","last_undegraded":"2024-04-16T05:34:56.008864+0000","last_fullsized":"2024-04-16T05:34:56.008864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:54:37.915041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008806+0000","last_change":"2024-04-16T05:32:33.731842+0000","last_active":"2024-04-16T05:34:56.008806+0000","last_peered":"2024-04-16T05:34:56.008806+0000","last_clean":"2024-04-16T05:34:56.008806+0000","last_became_active":"2024-04-16T05:32:33.731417+0000","last_became_peered":"2024-04-16T05:32:33.731417+0000","last_unstale":"2024-04-16T05:34:56.008806+0000","last_undegraded":"2024-04-16T05:34:56.008806+0000","last_fullsized":"2024-04-16T05:34:56.008806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:04:03.141533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114569+0000","last_change":"2024-04-16T05:32:34.755062+0000","last_active":"2024-04-16T05:34:56.114569+0000","last_peered":"2024-04-16T05:34:56.114569+0000","last_clean":"2024-04-16T05:34:56.114569+0000","last_became_active":"2024-04-16T05:32:34.754759+0000","last_became_peered":"2024-04-16T05:32:34.754759+0000","last_unstale":"2024-04-16T05:34:56.114569+0000","last_undegraded":"2024-04-16T05:34:56.114569+0000","last_fullsized":"2024-04-16T05:34:56.114569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:53:59.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264299+0000","last_change":"2024-04-16T05:32:38.935646+0000","last_active":"2024-04-16T05:34:56.264299+0000","last_peered":"2024-04-16T05:34:56.264299+0000","last_clean":"2024-04-16T05:34:56.264299+0000","last_became_active":"2024-04-16T05:32:38.935206+0000","last_became_peered":"2024-04-16T05:32:38.935206+0000","last_unstale":"2024-04-16T05:34:56.264299+0000","last_undegraded":"2024-04-16T05:34:56.264299+0000","last_fullsized":"2024-04-16T05:34:56.264299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:09.772136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114682+0000","last_change":"2024-04-16T05:32:33.739343+0000","last_active":"2024-04-16T05:34:56.114682+0000","last_peered":"2024-04-16T05:34:56.114682+0000","last_clean":"2024-04-16T05:34:56.114682+0000","last_became_active":"2024-04-16T05:32:33.738982+0000","last_became_peered":"2024-04-16T05:32:33.738982+0000","last_unstale":"2024-04-16T05:34:56.114682+0000","last_undegraded":"2024-04-16T05:34:56.114682+0000","last_fullsized":"2024-04-16T05:34:56.114682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:36.002804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008721+0000","last_change":"2024-04-16T05:32:34.749316+0000","last_active":"2024-04-16T05:34:56.008721+0000","last_peered":"2024-04-16T05:34:56.008721+0000","last_clean":"2024-04-16T05:34:56.008721+0000","last_became_active":"2024-04-16T05:32:34.748976+0000","last_became_peered":"2024-04-16T05:32:34.748976+0000","last_unstale":"2024-04-16T05:34:56.008721+0000","last_undegraded":"2024-04-16T05:34:56.008721+0000","last_fullsized":"2024-04-16T05:34:56.008721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:16:29.569402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264259+0000","last_change":"2024-04-16T05:32:38.938633+0000","last_active":"2024-04-16T05:34:56.264259+0000","last_peered":"2024-04-16T05:34:56.264259+0000","last_clean":"2024-04-16T05:34:56.264259+0000","last_became_active":"2024-04-16T05:32:38.938215+0000","last_became_peered":"2024-04-16T05:32:38.938215+0000","last_unstale":"2024-04-16T05:34:56.264259+0000","last_undegraded":"2024-04-16T05:34:56.264259+0000","last_fullsized":"2024-04-16T05:34:56.264259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:05.078109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117612+0000","last_change":"2024-04-16T05:32:33.744262+0000","last_active":"2024-04-16T05:34:56.117612+0000","last_peered":"2024-04-16T05:34:56.117612+0000","last_clean":"2024-04-16T05:34:56.117612+0000","last_became_active":"2024-04-16T05:32:33.743866+0000","last_became_peered":"2024-04-16T05:32:33.743866+0000","last_unstale":"2024-04-16T05:34:56.117612+0000","last_undegraded":"2024-04-16T05:34:56.117612+0000","last_fullsized":"2024-04-16T05:34:56.117612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:22:03.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114365+0000","last_change":"2024-04-16T05:32:34.750201+0000","last_active":"2024-04-16T05:34:56.114365+0000","last_peered":"2024-04-16T05:34:56.114365+0000","last_clean":"2024-04-16T05:34:56.114365+0000","last_became_active":"2024-04-16T05:32:34.749900+0000","last_became_peered":"2024-04-16T05:32:34.749900+0000","last_unstale":"2024-04-16T05:34:56.114365+0000","last_undegraded":"2024-04-16T05:34:56.114365+0000","last_fullsized":"2024-04-16T05:34:56.114365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:42.708584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639920+0000","last_change":"2024-04-16T05:32:38.931277+0000","last_active":"2024-04-16T05:34:56.639920+0000","last_peered":"2024-04-16T05:34:56.639920+0000","last_clean":"2024-04-16T05:34:56.639920+0000","last_became_active":"2024-04-16T05:32:38.930888+0000","last_became_peered":"2024-04-16T05:32:38.930888+0000","last_unstale":"2024-04-16T05:34:56.639920+0000","last_undegraded":"2024-04-16T05:34:56.639920+0000","last_fullsized":"2024-04-16T05:34:56.639920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:27:31.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264559+0000","last_change":"2024-04-16T05:32:33.747740+0000","last_active":"2024-04-16T05:34:56.264559+0000","last_peered":"2024-04-16T05:34:56.264559+0000","last_clean":"2024-04-16T05:34:56.264559+0000","last_became_active":"2024-04-16T05:32:33.747313+0000","last_became_peered":"2024-04-16T05:32:33.747313+0000","last_unstale":"2024-04-16T05:34:56.264559+0000","last_undegraded":"2024-04-16T05:34:56.264559+0000","last_fullsized":"2024-04-16T05:34:56.264559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:05:03.937790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105220+0000","last_change":"2024-04-16T05:32:34.760075+0000","last_active":"2024-04-16T05:34:56.105220+0000","last_peered":"2024-04-16T05:34:56.105220+0000","last_clean":"2024-04-16T05:34:56.105220+0000","last_became_active":"2024-04-16T05:32:34.759569+0000","last_became_peered":"2024-04-16T05:32:34.759569+0000","last_unstale":"2024-04-16T05:34:56.105220+0000","last_undegraded":"2024-04-16T05:34:56.105220+0000","last_fullsized":"2024-04-16T05:34:56.105220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:54.222877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266895+0000","last_change":"2024-04-16T05:32:38.928568+0000","last_active":"2024-04-16T05:34:56.266895+0000","last_peered":"2024-04-16T05:34:56.266895+0000","last_clean":"2024-04-16T05:34:56.266895+0000","last_became_active":"2024-04-16T05:32:38.928109+0000","last_became_peered":"2024-04-16T05:32:38.928109+0000","last_unstale":"2024-04-16T05:34:56.266895+0000","last_undegraded":"2024-04-16T05:34:56.266895+0000","last_fullsized":"2024-04-16T05:34:56.266895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:24:14.293528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114216+0000","last_change":"2024-04-16T05:32:33.737803+0000","last_active":"2024-04-16T05:34:56.114216+0000","last_peered":"2024-04-16T05:34:56.114216+0000","last_clean":"2024-04-16T05:34:56.114216+0000","last_became_active":"2024-04-16T05:32:33.737448+0000","last_became_peered":"2024-04-16T05:32:33.737448+0000","last_unstale":"2024-04-16T05:34:56.114216+0000","last_undegraded":"2024-04-16T05:34:56.114216+0000","last_fullsized":"2024-04-16T05:34:56.114216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:44:54.302323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007203+0000","last_change":"2024-04-16T05:32:34.754070+0000","last_active":"2024-04-16T05:34:56.007203+0000","last_peered":"2024-04-16T05:34:56.007203+0000","last_clean":"2024-04-16T05:34:56.007203+0000","last_became_active":"2024-04-16T05:32:34.753717+0000","last_became_peered":"2024-04-16T05:32:34.753717+0000","last_unstale":"2024-04-16T05:34:56.007203+0000","last_undegraded":"2024-04-16T05:34:56.007203+0000","last_fullsized":"2024-04-16T05:34:56.007203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:58:56.738245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.106064+0000","last_change":"2024-04-16T05:32:38.931594+0000","last_active":"2024-04-16T05:34:56.106064+0000","last_peered":"2024-04-16T05:34:56.106064+0000","last_clean":"2024-04-16T05:34:56.106064+0000","last_became_active":"2024-04-16T05:32:38.931188+0000","last_became_peered":"2024-04-16T05:32:38.931188+0000","last_unstale":"2024-04-16T05:34:56.106064+0000","last_undegraded":"2024-04-16T05:34:56.106064+0000","last_fullsized":"2024-04-16T05:34:56.106064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:55.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.118099+0000","last_change":"2024-04-16T05:32:33.739752+0000","last_active":"2024-04-16T05:34:56.118099+0000","last_peered":"2024-04-16T05:34:56.118099+0000","last_clean":"2024-04-16T05:34:56.118099+0000","last_became_active":"2024-04-16T05:32:33.739444+0000","last_became_peered":"2024-04-16T05:32:33.739444+0000","last_unstale":"2024-04-16T05:34:56.118099+0000","last_undegraded":"2024-04-16T05:34:56.118099+0000","last_fullsized":"2024-04-16T05:34:56.118099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:04:18.091618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265179+0000","last_change":"2024-04-16T05:32:34.759197+0000","last_active":"2024-04-16T05:34:56.265179+0000","last_peered":"2024-04-16T05:34:56.265179+0000","last_clean":"2024-04-16T05:34:56.265179+0000","last_became_active":"2024-04-16T05:32:34.758783+0000","last_became_peered":"2024-04-16T05:32:34.758783+0000","last_unstale":"2024-04-16T05:34:56.265179+0000","last_undegraded":"2024-04-16T05:34:56.265179+0000","last_fullsized":"2024-04-16T05:34:56.265179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:23:37.020254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007139+0000","last_change":"2024-04-16T05:32:38.936198+0000","last_active":"2024-04-16T05:34:56.007139+0000","last_peered":"2024-04-16T05:34:56.007139+0000","last_clean":"2024-04-16T05:34:56.007139+0000","last_became_active":"2024-04-16T05:32:38.935698+0000","last_became_peered":"2024-04-16T05:32:38.935698+0000","last_unstale":"2024-04-16T05:34:56.007139+0000","last_undegraded":"2024-04-16T05:34:56.007139+0000","last_fullsized":"2024-04-16T05:34:56.007139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:17:35.602408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058994+0000","last_change":"2024-04-16T05:32:33.737653+0000","last_active":"2024-04-16T05:34:56.058994+0000","last_peered":"2024-04-16T05:34:56.058994+0000","last_clean":"2024-04-16T05:34:56.058994+0000","last_became_active":"2024-04-16T05:32:33.737342+0000","last_became_peered":"2024-04-16T05:32:33.737342+0000","last_unstale":"2024-04-16T05:34:56.058994+0000","last_undegraded":"2024-04-16T05:34:56.058994+0000","last_fullsized":"2024-04-16T05:34:56.058994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:17:35.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266896+0000","last_change":"2024-04-16T05:32:34.765722+0000","last_active":"2024-04-16T05:34:56.266896+0000","last_peered":"2024-04-16T05:34:56.266896+0000","last_clean":"2024-04-16T05:34:56.266896+0000","last_became_active":"2024-04-16T05:32:34.765413+0000","last_became_peered":"2024-04-16T05:32:34.765413+0000","last_unstale":"2024-04-16T05:34:56.266896+0000","last_undegraded":"2024-04-16T05:34:56.266896+0000","last_fullsized":"2024-04-16T05:34:56.266896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:03.884041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009328+0000","last_change":"2024-04-16T05:32:38.936186+0000","last_active":"2024-04-16T05:34:56.009328+0000","last_peered":"2024-04-16T05:34:56.009328+0000","last_clean":"2024-04-16T05:34:56.009328+0000","last_became_active":"2024-04-16T05:32:38.935738+0000","last_became_peered":"2024-04-16T05:32:38.935738+0000","last_unstale":"2024-04-16T05:34:56.009328+0000","last_undegraded":"2024-04-16T05:34:56.009328+0000","last_fullsized":"2024-04-16T05:34:56.009328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:10.519095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114565+0000","last_change":"2024-04-16T05:32:33.742991+0000","last_active":"2024-04-16T05:34:56.114565+0000","last_peered":"2024-04-16T05:34:56.114565+0000","last_clean":"2024-04-16T05:34:56.114565+0000","last_became_active":"2024-04-16T05:32:33.742659+0000","last_became_peered":"2024-04-16T05:32:33.742659+0000","last_unstale":"2024-04-16T05:34:56.114565+0000","last_undegraded":"2024-04-16T05:34:56.114565+0000","last_fullsized":"2024-04-16T05:34:56.114565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:23.756123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264808+0000","last_change":"2024-04-16T05:32:34.759926+0000","last_active":"2024-04-16T05:34:56.264808+0000","last_peered":"2024-04-16T05:34:56.264808+0000","last_clean":"2024-04-16T05:34:56.264808+0000","last_became_active":"2024-04-16T05:32:34.759626+0000","last_became_peered":"2024-04-16T05:32:34.759626+0000","last_unstale":"2024-04-16T05:34:56.264808+0000","last_undegraded":"2024-04-16T05:34:56.264808+0000","last_fullsized":"2024-04-16T05:34:56.264808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:50.314638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007364+0000","last_change":"2024-04-16T05:32:38.942135+0000","last_active":"2024-04-16T05:34:56.007364+0000","last_peered":"2024-04-16T05:34:56.007364+0000","last_clean":"2024-04-16T05:34:56.007364+0000","last_became_active":"2024-04-16T05:32:38.941534+0000","last_became_peered":"2024-04-16T05:32:38.941534+0000","last_unstale":"2024-04-16T05:34:56.007364+0000","last_undegraded":"2024-04-16T05:34:56.007364+0000","last_fullsized":"2024-04-16T05:34:56.007364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:20.494516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":323,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.873982+0000","last_change":"2024-04-16T05:32:33.734275+0000","last_active":"2024-04-16T05:34:56.873982+0000","last_peered":"2024-04-16T05:34:56.873982+0000","last_clean":"2024-04-16T05:34:56.873982+0000","last_became_active":"2024-04-16T05:32:33.733906+0000","last_became_peered":"2024-04-16T05:32:33.733906+0000","last_unstale":"2024-04-16T05:34:56.873982+0000","last_undegraded":"2024-04-16T05:34:56.873982+0000","last_fullsized":"2024-04-16T05:34:56.873982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:39:23.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":195,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008907+0000","last_change":"2024-04-16T05:32:34.750179+0000","last_active":"2024-04-16T05:34:56.008907+0000","last_peered":"2024-04-16T05:34:56.008907+0000","last_clean":"2024-04-16T05:34:56.008907+0000","last_became_active":"2024-04-16T05:32:34.749773+0000","last_became_peered":"2024-04-16T05:32:34.749773+0000","last_unstale":"2024-04-16T05:34:56.008907+0000","last_undegraded":"2024-04-16T05:34:56.008907+0000","last_fullsized":"2024-04-16T05:34:56.008907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:37.621191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114863+0000","last_change":"2024-04-16T05:32:38.935690+0000","last_active":"2024-04-16T05:34:56.114863+0000","last_peered":"2024-04-16T05:34:56.114863+0000","last_clean":"2024-04-16T05:34:56.114863+0000","last_became_active":"2024-04-16T05:32:38.935282+0000","last_became_peered":"2024-04-16T05:32:38.935282+0000","last_unstale":"2024-04-16T05:34:56.114863+0000","last_undegraded":"2024-04-16T05:34:56.114863+0000","last_fullsized":"2024-04-16T05:34:56.114863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:25:00.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639603+0000","last_change":"2024-04-16T05:33:34.782195+0000","last_active":"2024-04-16T05:34:56.639603+0000","last_peered":"2024-04-16T05:34:56.639603+0000","last_clean":"2024-04-16T05:34:56.639603+0000","last_became_active":"2024-04-16T05:33:34.771490+0000","last_became_peered":"2024-04-16T05:33:34.771490+0000","last_unstale":"2024-04-16T05:34:56.639603+0000","last_undegraded":"2024-04-16T05:34:56.639603+0000","last_fullsized":"2024-04-16T05:34:56.639603+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:52.513976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265873+0000","last_change":"2024-04-16T05:32:34.748760+0000","last_active":"2024-04-16T05:34:56.265873+0000","last_peered":"2024-04-16T05:34:56.265873+0000","last_clean":"2024-04-16T05:34:56.265873+0000","last_became_active":"2024-04-16T05:32:34.748432+0000","last_became_peered":"2024-04-16T05:32:34.748432+0000","last_unstale":"2024-04-16T05:34:56.265873+0000","last_undegraded":"2024-04-16T05:34:56.265873+0000","last_fullsized":"2024-04-16T05:34:56.265873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:12:42.859342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639575+0000","last_change":"2024-04-16T05:32:38.929705+0000","last_active":"2024-04-16T05:34:56.639575+0000","last_peered":"2024-04-16T05:34:56.639575+0000","last_clean":"2024-04-16T05:34:56.639575+0000","last_became_active":"2024-04-16T05:32:38.929282+0000","last_became_peered":"2024-04-16T05:32:38.929282+0000","last_unstale":"2024-04-16T05:34:56.639575+0000","last_undegraded":"2024-04-16T05:34:56.639575+0000","last_fullsized":"2024-04-16T05:34:56.639575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:51:27.763600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114463+0000","last_change":"2024-04-16T05:32:33.746371+0000","last_active":"2024-04-16T05:34:56.114463+0000","last_peered":"2024-04-16T05:34:56.114463+0000","last_clean":"2024-04-16T05:34:56.114463+0000","last_became_active":"2024-04-16T05:32:33.746077+0000","last_became_peered":"2024-04-16T05:32:33.746077+0000","last_unstale":"2024-04-16T05:34:56.114463+0000","last_undegraded":"2024-04-16T05:34:56.114463+0000","last_fullsized":"2024-04-16T05:34:56.114463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:57:43.193942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059205+0000","last_change":"2024-04-16T05:32:34.753196+0000","last_active":"2024-04-16T05:34:56.059205+0000","last_peered":"2024-04-16T05:34:56.059205+0000","last_clean":"2024-04-16T05:34:56.059205+0000","last_became_active":"2024-04-16T05:32:34.752881+0000","last_became_peered":"2024-04-16T05:32:34.752881+0000","last_unstale":"2024-04-16T05:34:56.059205+0000","last_undegraded":"2024-04-16T05:34:56.059205+0000","last_fullsized":"2024-04-16T05:34:56.059205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:35:56.715394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008805+0000","last_change":"2024-04-16T05:33:34.772562+0000","last_active":"2024-04-16T05:34:56.008805+0000","last_peered":"2024-04-16T05:34:56.008805+0000","last_clean":"2024-04-16T05:34:56.008805+0000","last_became_active":"2024-04-16T05:33:34.772052+0000","last_became_peered":"2024-04-16T05:33:34.772052+0000","last_unstale":"2024-04-16T05:34:56.008805+0000","last_undegraded":"2024-04-16T05:34:56.008805+0000","last_fullsized":"2024-04-16T05:34:56.008805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:50:18.425870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.873651+0000","last_change":"2024-04-16T05:32:33.735921+0000","last_active":"2024-04-16T05:34:56.873651+0000","last_peered":"2024-04-16T05:34:56.873651+0000","last_clean":"2024-04-16T05:34:56.873651+0000","last_became_active":"2024-04-16T05:32:33.735572+0000","last_became_peered":"2024-04-16T05:32:33.735572+0000","last_unstale":"2024-04-16T05:34:56.873651+0000","last_undegraded":"2024-04-16T05:34:56.873651+0000","last_fullsized":"2024-04-16T05:34:56.873651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:46:56.512827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058872+0000","last_change":"2024-04-16T05:32:34.762172+0000","last_active":"2024-04-16T05:34:56.058872+0000","last_peered":"2024-04-16T05:34:56.058872+0000","last_clean":"2024-04-16T05:34:56.058872+0000","last_became_active":"2024-04-16T05:32:34.761914+0000","last_became_peered":"2024-04-16T05:32:34.761914+0000","last_unstale":"2024-04-16T05:34:56.058872+0000","last_undegraded":"2024-04-16T05:34:56.058872+0000","last_fullsized":"2024-04-16T05:34:56.058872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:47:52.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267525+0000","last_change":"2024-04-16T05:33:34.772546+0000","last_active":"2024-04-16T05:34:56.267525+0000","last_peered":"2024-04-16T05:34:56.267525+0000","last_clean":"2024-04-16T05:34:56.267525+0000","last_became_active":"2024-04-16T05:33:34.772100+0000","last_became_peered":"2024-04-16T05:33:34.772100+0000","last_unstale":"2024-04-16T05:34:56.267525+0000","last_undegraded":"2024-04-16T05:34:56.267525+0000","last_fullsized":"2024-04-16T05:34:56.267525+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:11:50.157129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008514+0000","last_change":"2024-04-16T05:32:33.743944+0000","last_active":"2024-04-16T05:34:56.008514+0000","last_peered":"2024-04-16T05:34:56.008514+0000","last_clean":"2024-04-16T05:34:56.008514+0000","last_became_active":"2024-04-16T05:32:33.743559+0000","last_became_peered":"2024-04-16T05:32:33.743559+0000","last_unstale":"2024-04-16T05:34:56.008514+0000","last_undegraded":"2024-04-16T05:34:56.008514+0000","last_fullsized":"2024-04-16T05:34:56.008514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:08:48.889356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114332+0000","last_change":"2024-04-16T05:32:34.756741+0000","last_active":"2024-04-16T05:34:56.114332+0000","last_peered":"2024-04-16T05:34:56.114332+0000","last_clean":"2024-04-16T05:34:56.114332+0000","last_became_active":"2024-04-16T05:32:34.756426+0000","last_became_peered":"2024-04-16T05:32:34.756426+0000","last_unstale":"2024-04-16T05:34:56.114332+0000","last_undegraded":"2024-04-16T05:34:56.114332+0000","last_fullsized":"2024-04-16T05:34:56.114332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:09:30.493857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114287+0000","last_change":"2024-04-16T05:32:38.934818+0000","last_active":"2024-04-16T05:34:56.114287+0000","last_peered":"2024-04-16T05:34:56.114287+0000","last_clean":"2024-04-16T05:34:56.114287+0000","last_became_active":"2024-04-16T05:32:38.934335+0000","last_became_peered":"2024-04-16T05:32:38.934335+0000","last_unstale":"2024-04-16T05:34:56.114287+0000","last_undegraded":"2024-04-16T05:34:56.114287+0000","last_fullsized":"2024-04-16T05:34:56.114287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:02:41.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009144+0000","last_change":"2024-04-16T05:32:33.731951+0000","last_active":"2024-04-16T05:34:56.009144+0000","last_peered":"2024-04-16T05:34:56.009144+0000","last_clean":"2024-04-16T05:34:56.009144+0000","last_became_active":"2024-04-16T05:32:33.731649+0000","last_became_peered":"2024-04-16T05:32:33.731649+0000","last_unstale":"2024-04-16T05:34:56.009144+0000","last_undegraded":"2024-04-16T05:34:56.009144+0000","last_fullsized":"2024-04-16T05:34:56.009144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:41.258012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639876+0000","last_change":"2024-04-16T05:32:34.751884+0000","last_active":"2024-04-16T05:34:56.639876+0000","last_peered":"2024-04-16T05:34:56.639876+0000","last_clean":"2024-04-16T05:34:56.639876+0000","last_became_active":"2024-04-16T05:32:34.751559+0000","last_became_peered":"2024-04-16T05:32:34.751559+0000","last_unstale":"2024-04-16T05:34:56.639876+0000","last_undegraded":"2024-04-16T05:34:56.639876+0000","last_fullsized":"2024-04-16T05:34:56.639876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:32:59.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008772+0000","last_change":"2024-04-16T05:32:39.109330+0000","last_active":"2024-04-16T05:34:56.008772+0000","last_peered":"2024-04-16T05:34:56.008772+0000","last_clean":"2024-04-16T05:34:56.008772+0000","last_became_active":"2024-04-16T05:32:39.108718+0000","last_became_peered":"2024-04-16T05:32:39.108718+0000","last_unstale":"2024-04-16T05:34:56.008772+0000","last_undegraded":"2024-04-16T05:34:56.008772+0000","last_fullsized":"2024-04-16T05:34:56.008772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:36:55.418034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007436+0000","last_change":"2024-04-16T05:32:33.741603+0000","last_active":"2024-04-16T05:34:56.007436+0000","last_peered":"2024-04-16T05:34:56.007436+0000","last_clean":"2024-04-16T05:34:56.007436+0000","last_became_active":"2024-04-16T05:32:33.741184+0000","last_became_peered":"2024-04-16T05:32:33.741184+0000","last_unstale":"2024-04-16T05:34:56.007436+0000","last_undegraded":"2024-04-16T05:34:56.007436+0000","last_fullsized":"2024-04-16T05:34:56.007436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:08:38.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117997+0000","last_change":"2024-04-16T05:32:34.747768+0000","last_active":"2024-04-16T05:34:56.117997+0000","last_peered":"2024-04-16T05:34:56.117997+0000","last_clean":"2024-04-16T05:34:56.117997+0000","last_became_active":"2024-04-16T05:32:34.747440+0000","last_became_peered":"2024-04-16T05:32:34.747440+0000","last_unstale":"2024-04-16T05:34:56.117997+0000","last_undegraded":"2024-04-16T05:34:56.117997+0000","last_fullsized":"2024-04-16T05:34:56.117997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:14:31.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266840+0000","last_change":"2024-04-16T05:32:38.938673+0000","last_active":"2024-04-16T05:34:56.266840+0000","last_peered":"2024-04-16T05:34:56.266840+0000","last_clean":"2024-04-16T05:34:56.266840+0000","last_became_active":"2024-04-16T05:32:38.938252+0000","last_became_peered":"2024-04-16T05:32:38.938252+0000","last_unstale":"2024-04-16T05:34:56.266840+0000","last_undegraded":"2024-04-16T05:34:56.266840+0000","last_fullsized":"2024-04-16T05:34:56.266840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:06.620693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058730+0000","last_change":"2024-04-16T05:32:33.750812+0000","last_active":"2024-04-16T05:34:56.058730+0000","last_peered":"2024-04-16T05:34:56.058730+0000","last_clean":"2024-04-16T05:34:56.058730+0000","last_became_active":"2024-04-16T05:32:33.750514+0000","last_became_peered":"2024-04-16T05:32:33.750514+0000","last_unstale":"2024-04-16T05:34:56.058730+0000","last_undegraded":"2024-04-16T05:34:56.058730+0000","last_fullsized":"2024-04-16T05:34:56.058730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:53:43.075914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117672+0000","last_change":"2024-04-16T05:33:34.770449+0000","last_active":"2024-04-16T05:34:56.117672+0000","last_peered":"2024-04-16T05:34:56.117672+0000","last_clean":"2024-04-16T05:34:56.117672+0000","last_became_active":"2024-04-16T05:33:34.770078+0000","last_became_peered":"2024-04-16T05:33:34.770078+0000","last_unstale":"2024-04-16T05:34:56.117672+0000","last_undegraded":"2024-04-16T05:34:56.117672+0000","last_fullsized":"2024-04-16T05:34:56.117672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:30:20.885002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267319+0000","last_change":"2024-04-16T05:32:39.109394+0000","last_active":"2024-04-16T05:34:56.267319+0000","last_peered":"2024-04-16T05:34:56.267319+0000","last_clean":"2024-04-16T05:34:56.267319+0000","last_became_active":"2024-04-16T05:32:39.108924+0000","last_became_peered":"2024-04-16T05:32:39.108924+0000","last_unstale":"2024-04-16T05:34:56.267319+0000","last_undegraded":"2024-04-16T05:34:56.267319+0000","last_fullsized":"2024-04-16T05:34:56.267319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:50:37.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007055+0000","last_change":"2024-04-16T05:33:34.766071+0000","last_active":"2024-04-16T05:34:56.007055+0000","last_peered":"2024-04-16T05:34:56.007055+0000","last_clean":"2024-04-16T05:34:56.007055+0000","last_became_active":"2024-04-16T05:33:34.765670+0000","last_became_peered":"2024-04-16T05:33:34.765670+0000","last_unstale":"2024-04-16T05:34:56.007055+0000","last_undegraded":"2024-04-16T05:34:56.007055+0000","last_fullsized":"2024-04-16T05:34:56.007055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:54:47.320828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267990+0000","last_change":"2024-04-16T05:32:33.748521+0000","last_active":"2024-04-16T05:34:56.267990+0000","last_peered":"2024-04-16T05:34:56.267990+0000","last_clean":"2024-04-16T05:34:56.267990+0000","last_became_active":"2024-04-16T05:32:33.748222+0000","last_became_peered":"2024-04-16T05:32:33.748222+0000","last_unstale":"2024-04-16T05:34:56.267990+0000","last_undegraded":"2024-04-16T05:34:56.267990+0000","last_fullsized":"2024-04-16T05:34:56.267990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:04.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058859+0000","last_change":"2024-04-16T05:32:38.935995+0000","last_active":"2024-04-16T05:34:56.058859+0000","last_peered":"2024-04-16T05:34:56.058859+0000","last_clean":"2024-04-16T05:34:56.058859+0000","last_became_active":"2024-04-16T05:32:38.935511+0000","last_became_peered":"2024-04-16T05:32:38.935511+0000","last_unstale":"2024-04-16T05:34:56.058859+0000","last_undegraded":"2024-04-16T05:34:56.058859+0000","last_fullsized":"2024-04-16T05:34:56.058859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:17:08.144537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105847+0000","last_change":"2024-04-16T05:32:34.760249+0000","last_active":"2024-04-16T05:34:56.105847+0000","last_peered":"2024-04-16T05:34:56.105847+0000","last_clean":"2024-04-16T05:34:56.105847+0000","last_became_active":"2024-04-16T05:32:34.759735+0000","last_became_peered":"2024-04-16T05:32:34.759735+0000","last_unstale":"2024-04-16T05:34:56.105847+0000","last_undegraded":"2024-04-16T05:34:56.105847+0000","last_fullsized":"2024-04-16T05:34:56.105847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:13.349558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009119+0000","last_change":"2024-04-16T05:32:33.731170+0000","last_active":"2024-04-16T05:34:56.009119+0000","last_peered":"2024-04-16T05:34:56.009119+0000","last_clean":"2024-04-16T05:34:56.009119+0000","last_became_active":"2024-04-16T05:32:33.730821+0000","last_became_peered":"2024-04-16T05:32:33.730821+0000","last_unstale":"2024-04-16T05:34:56.009119+0000","last_undegraded":"2024-04-16T05:34:56.009119+0000","last_fullsized":"2024-04-16T05:34:56.009119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:38:15.707590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":365,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058562+0000","last_change":"2024-04-16T05:31:24.926319+0000","last_active":"2024-04-16T05:34:56.058562+0000","last_peered":"2024-04-16T05:34:56.058562+0000","last_clean":"2024-04-16T05:34:56.058562+0000","last_became_active":"2024-04-16T05:31:24.925936+0000","last_became_peered":"2024-04-16T05:31:24.925936+0000","last_unstale":"2024-04-16T05:34:56.058562+0000","last_undegraded":"2024-04-16T05:34:56.058562+0000","last_fullsized":"2024-04-16T05:34:56.058562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_clean_scrub_stamp":"2024-04-16T05:27:57.876673+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:28:19.212127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114596+0000","last_change":"2024-04-16T05:32:39.110241+0000","last_active":"2024-04-16T05:34:56.114596+0000","last_peered":"2024-04-16T05:34:56.114596+0000","last_clean":"2024-04-16T05:34:56.114596+0000","last_became_active":"2024-04-16T05:32:39.109792+0000","last_became_peered":"2024-04-16T05:32:39.109792+0000","last_unstale":"2024-04-16T05:34:56.114596+0000","last_undegraded":"2024-04-16T05:34:56.114596+0000","last_fullsized":"2024-04-16T05:34:56.114596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:15:05.925370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264402+0000","last_change":"2024-04-16T05:32:34.751693+0000","last_active":"2024-04-16T05:34:56.264402+0000","last_peered":"2024-04-16T05:34:56.264402+0000","last_clean":"2024-04-16T05:34:56.264402+0000","last_became_active":"2024-04-16T05:32:34.751210+0000","last_became_peered":"2024-04-16T05:32:34.751210+0000","last_unstale":"2024-04-16T05:34:56.264402+0000","last_undegraded":"2024-04-16T05:34:56.264402+0000","last_fullsized":"2024-04-16T05:34:56.264402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:33:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058618+0000","last_change":"2024-04-16T05:32:33.738310+0000","last_active":"2024-04-16T05:34:56.058618+0000","last_peered":"2024-04-16T05:34:56.058618+0000","last_clean":"2024-04-16T05:34:56.058618+0000","last_became_active":"2024-04-16T05:32:33.738012+0000","last_became_peered":"2024-04-16T05:32:33.738012+0000","last_unstale":"2024-04-16T05:34:56.058618+0000","last_undegraded":"2024-04-16T05:34:56.058618+0000","last_fullsized":"2024-04-16T05:34:56.058618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:22:18.311458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639634+0000","last_change":"2024-04-16T05:32:38.933465+0000","last_active":"2024-04-16T05:34:56.639634+0000","last_peered":"2024-04-16T05:34:56.639634+0000","last_clean":"2024-04-16T05:34:56.639634+0000","last_became_active":"2024-04-16T05:32:38.932391+0000","last_became_peered":"2024-04-16T05:32:38.932391+0000","last_unstale":"2024-04-16T05:34:56.639634+0000","last_undegraded":"2024-04-16T05:34:56.639634+0000","last_fullsized":"2024-04-16T05:34:56.639634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:39.494556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009019+0000","last_change":"2024-04-16T05:32:33.743241+0000","last_active":"2024-04-16T05:34:56.009019+0000","last_peered":"2024-04-16T05:34:56.009019+0000","last_clean":"2024-04-16T05:34:56.009019+0000","last_became_active":"2024-04-16T05:32:33.742887+0000","last_became_peered":"2024-04-16T05:32:33.742887+0000","last_unstale":"2024-04-16T05:34:56.009019+0000","last_undegraded":"2024-04-16T05:34:56.009019+0000","last_fullsized":"2024-04-16T05:34:56.009019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:24:38.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267151+0000","last_change":"2024-04-16T05:32:34.746228+0000","last_active":"2024-04-16T05:34:56.267151+0000","last_peered":"2024-04-16T05:34:56.267151+0000","last_clean":"2024-04-16T05:34:56.267151+0000","last_became_active":"2024-04-16T05:32:34.745883+0000","last_became_peered":"2024-04-16T05:32:34.745883+0000","last_unstale":"2024-04-16T05:34:56.267151+0000","last_undegraded":"2024-04-16T05:34:56.267151+0000","last_fullsized":"2024-04-16T05:34:56.267151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:33.656358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105107+0000","last_change":"2024-04-16T05:32:39.110156+0000","last_active":"2024-04-16T05:34:56.105107+0000","last_peered":"2024-04-16T05:34:56.105107+0000","last_clean":"2024-04-16T05:34:56.105107+0000","last_became_active":"2024-04-16T05:32:39.108958+0000","last_became_peered":"2024-04-16T05:32:39.108958+0000","last_unstale":"2024-04-16T05:34:56.105107+0000","last_undegraded":"2024-04-16T05:34:56.105107+0000","last_fullsized":"2024-04-16T05:34:56.105107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:27:58.658187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'45","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.229853+0000","last_change":"2024-04-16T05:32:33.738179+0000","last_active":"2024-04-16T05:34:56.229853+0000","last_peered":"2024-04-16T05:34:56.229853+0000","last_clean":"2024-04-16T05:34:56.229853+0000","last_became_active":"2024-04-16T05:32:33.737841+0000","last_became_peered":"2024-04-16T05:32:33.737841+0000","last_unstale":"2024-04-16T05:34:56.229853+0000","last_undegraded":"2024-04-16T05:34:56.229853+0000","last_fullsized":"2024-04-16T05:34:56.229853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:13:11.210559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107138+0000","last_change":"2024-04-16T05:32:34.758142+0000","last_active":"2024-04-16T05:34:56.107138+0000","last_peered":"2024-04-16T05:34:56.107138+0000","last_clean":"2024-04-16T05:34:56.107138+0000","last_became_active":"2024-04-16T05:32:34.757825+0000","last_became_peered":"2024-04-16T05:32:34.757825+0000","last_unstale":"2024-04-16T05:34:56.107138+0000","last_undegraded":"2024-04-16T05:34:56.107138+0000","last_fullsized":"2024-04-16T05:34:56.107138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:37.575764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.263978+0000","last_change":"2024-04-16T05:32:38.933327+0000","last_active":"2024-04-16T05:34:56.263978+0000","last_peered":"2024-04-16T05:34:56.263978+0000","last_clean":"2024-04-16T05:34:56.263978+0000","last_became_active":"2024-04-16T05:32:38.932808+0000","last_became_peered":"2024-04-16T05:32:38.932808+0000","last_unstale":"2024-04-16T05:34:56.263978+0000","last_undegraded":"2024-04-16T05:34:56.263978+0000","last_fullsized":"2024-04-16T05:34:56.263978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:03.217103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009338+0000","last_change":"2024-04-16T05:32:33.742405+0000","last_active":"2024-04-16T05:34:56.009338+0000","last_peered":"2024-04-16T05:34:56.009338+0000","last_clean":"2024-04-16T05:34:56.009338+0000","last_became_active":"2024-04-16T05:32:33.742051+0000","last_became_peered":"2024-04-16T05:32:33.742051+0000","last_unstale":"2024-04-16T05:34:56.009338+0000","last_undegraded":"2024-04-16T05:34:56.009338+0000","last_fullsized":"2024-04-16T05:34:56.009338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:35:13.914483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267051+0000","last_change":"2024-04-16T05:33:34.772430+0000","last_active":"2024-04-16T05:34:56.267051+0000","last_peered":"2024-04-16T05:34:56.267051+0000","last_clean":"2024-04-16T05:34:56.267051+0000","last_became_active":"2024-04-16T05:33:34.772067+0000","last_became_peered":"2024-04-16T05:33:34.772067+0000","last_unstale":"2024-04-16T05:34:56.267051+0000","last_undegraded":"2024-04-16T05:34:56.267051+0000","last_fullsized":"2024-04-16T05:34:56.267051+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:24.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009151+0000","last_change":"2024-04-16T05:33:34.770714+0000","last_active":"2024-04-16T05:34:56.009151+0000","last_peered":"2024-04-16T05:34:56.009151+0000","last_clean":"2024-04-16T05:34:56.009151+0000","last_became_active":"2024-04-16T05:33:34.770249+0000","last_became_peered":"2024-04-16T05:33:34.770249+0000","last_unstale":"2024-04-16T05:34:56.009151+0000","last_undegraded":"2024-04-16T05:34:56.009151+0000","last_fullsized":"2024-04-16T05:34:56.009151+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:46.517492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264301+0000","last_change":"2024-04-16T05:32:33.743031+0000","last_active":"2024-04-16T05:34:56.264301+0000","last_peered":"2024-04-16T05:34:56.264301+0000","last_clean":"2024-04-16T05:34:56.264301+0000","last_became_active":"2024-04-16T05:32:33.742771+0000","last_became_peered":"2024-04-16T05:32:33.742771+0000","last_unstale":"2024-04-16T05:34:56.264301+0000","last_undegraded":"2024-04-16T05:34:56.264301+0000","last_fullsized":"2024-04-16T05:34:56.264301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:06.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008461+0000","last_change":"2024-04-16T05:32:34.749428+0000","last_active":"2024-04-16T05:34:56.008461+0000","last_peered":"2024-04-16T05:34:56.008461+0000","last_clean":"2024-04-16T05:34:56.008461+0000","last_became_active":"2024-04-16T05:32:34.749087+0000","last_became_peered":"2024-04-16T05:32:34.749087+0000","last_unstale":"2024-04-16T05:34:56.008461+0000","last_undegraded":"2024-04-16T05:34:56.008461+0000","last_fullsized":"2024-04-16T05:34:56.008461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:59:25.124558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007570+0000","last_change":"2024-04-16T05:32:38.942407+0000","last_active":"2024-04-16T05:34:56.007570+0000","last_peered":"2024-04-16T05:34:56.007570+0000","last_clean":"2024-04-16T05:34:56.007570+0000","last_became_active":"2024-04-16T05:32:38.941731+0000","last_became_peered":"2024-04-16T05:32:38.941731+0000","last_unstale":"2024-04-16T05:34:56.007570+0000","last_undegraded":"2024-04-16T05:34:56.007570+0000","last_fullsized":"2024-04-16T05:34:56.007570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:42:27.700203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105694+0000","last_change":"2024-04-16T05:32:33.741487+0000","last_active":"2024-04-16T05:34:56.105694+0000","last_peered":"2024-04-16T05:34:56.105694+0000","last_clean":"2024-04-16T05:34:56.105694+0000","last_became_active":"2024-04-16T05:32:33.741140+0000","last_became_peered":"2024-04-16T05:32:33.741140+0000","last_unstale":"2024-04-16T05:34:56.105694+0000","last_undegraded":"2024-04-16T05:34:56.105694+0000","last_fullsized":"2024-04-16T05:34:56.105694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:21:36.086164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639473+0000","last_change":"2024-04-16T05:32:34.756608+0000","last_active":"2024-04-16T05:34:56.639473+0000","last_peered":"2024-04-16T05:34:56.639473+0000","last_clean":"2024-04-16T05:34:56.639473+0000","last_became_active":"2024-04-16T05:32:34.756296+0000","last_became_peered":"2024-04-16T05:32:34.756296+0000","last_unstale":"2024-04-16T05:34:56.639473+0000","last_undegraded":"2024-04-16T05:34:56.639473+0000","last_fullsized":"2024-04-16T05:34:56.639473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:28.397333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009566+0000","last_change":"2024-04-16T05:32:38.929876+0000","last_active":"2024-04-16T05:34:56.009566+0000","last_peered":"2024-04-16T05:34:56.009566+0000","last_clean":"2024-04-16T05:34:56.009566+0000","last_became_active":"2024-04-16T05:32:38.928626+0000","last_became_peered":"2024-04-16T05:32:38.928626+0000","last_unstale":"2024-04-16T05:34:56.009566+0000","last_undegraded":"2024-04-16T05:34:56.009566+0000","last_fullsized":"2024-04-16T05:34:56.009566+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:08:09.697413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264072+0000","last_change":"2024-04-16T05:32:33.736047+0000","last_active":"2024-04-16T05:34:56.264072+0000","last_peered":"2024-04-16T05:34:56.264072+0000","last_clean":"2024-04-16T05:34:56.264072+0000","last_became_active":"2024-04-16T05:32:33.735683+0000","last_became_peered":"2024-04-16T05:32:33.735683+0000","last_unstale":"2024-04-16T05:34:56.264072+0000","last_undegraded":"2024-04-16T05:34:56.264072+0000","last_fullsized":"2024-04-16T05:34:56.264072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:58:32.969924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.106396+0000","last_change":"2024-04-16T05:32:34.763558+0000","last_active":"2024-04-16T05:34:56.106396+0000","last_peered":"2024-04-16T05:34:56.106396+0000","last_clean":"2024-04-16T05:34:56.106396+0000","last_became_active":"2024-04-16T05:32:34.763190+0000","last_became_peered":"2024-04-16T05:32:34.763190+0000","last_unstale":"2024-04-16T05:34:56.106396+0000","last_undegraded":"2024-04-16T05:34:56.106396+0000","last_fullsized":"2024-04-16T05:34:56.106396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:38:31.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009179+0000","last_change":"2024-04-16T05:32:39.109426+0000","last_active":"2024-04-16T05:34:56.009179+0000","last_peered":"2024-04-16T05:34:56.009179+0000","last_clean":"2024-04-16T05:34:56.009179+0000","last_became_active":"2024-04-16T05:32:39.108970+0000","last_became_peered":"2024-04-16T05:32:39.108970+0000","last_unstale":"2024-04-16T05:34:56.009179+0000","last_undegraded":"2024-04-16T05:34:56.009179+0000","last_fullsized":"2024-04-16T05:34:56.009179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:54.824654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008593+0000","last_change":"2024-04-16T05:32:33.734983+0000","last_active":"2024-04-16T05:34:56.008593+0000","last_peered":"2024-04-16T05:34:56.008593+0000","last_clean":"2024-04-16T05:34:56.008593+0000","last_became_active":"2024-04-16T05:32:33.734609+0000","last_became_peered":"2024-04-16T05:32:33.734609+0000","last_unstale":"2024-04-16T05:34:56.008593+0000","last_undegraded":"2024-04-16T05:34:56.008593+0000","last_fullsized":"2024-04-16T05:34:56.008593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:35:59.436499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058954+0000","last_change":"2024-04-16T05:32:34.759811+0000","last_active":"2024-04-16T05:34:56.058954+0000","last_peered":"2024-04-16T05:34:56.058954+0000","last_clean":"2024-04-16T05:34:56.058954+0000","last_became_active":"2024-04-16T05:32:34.759499+0000","last_became_peered":"2024-04-16T05:32:34.759499+0000","last_unstale":"2024-04-16T05:34:56.058954+0000","last_undegraded":"2024-04-16T05:34:56.058954+0000","last_fullsized":"2024-04-16T05:34:56.058954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:46:28.554191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264537+0000","last_change":"2024-04-16T05:32:38.929021+0000","last_active":"2024-04-16T05:34:56.264537+0000","last_peered":"2024-04-16T05:34:56.264537+0000","last_clean":"2024-04-16T05:34:56.264537+0000","last_became_active":"2024-04-16T05:32:38.928574+0000","last_became_peered":"2024-04-16T05:32:38.928574+0000","last_unstale":"2024-04-16T05:34:56.264537+0000","last_undegraded":"2024-04-16T05:34:56.264537+0000","last_fullsized":"2024-04-16T05:34:56.264537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:11:35.693364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264578+0000","last_change":"2024-04-16T05:32:33.747352+0000","last_active":"2024-04-16T05:34:56.264578+0000","last_peered":"2024-04-16T05:34:56.264578+0000","last_clean":"2024-04-16T05:34:56.264578+0000","last_became_active":"2024-04-16T05:32:33.746989+0000","last_became_peered":"2024-04-16T05:32:33.746989+0000","last_unstale":"2024-04-16T05:34:56.264578+0000","last_undegraded":"2024-04-16T05:34:56.264578+0000","last_fullsized":"2024-04-16T05:34:56.264578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:38.422274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059157+0000","last_change":"2024-04-16T05:32:34.760636+0000","last_active":"2024-04-16T05:34:56.059157+0000","last_peered":"2024-04-16T05:34:56.059157+0000","last_clean":"2024-04-16T05:34:56.059157+0000","last_became_active":"2024-04-16T05:32:34.760335+0000","last_became_peered":"2024-04-16T05:32:34.760335+0000","last_unstale":"2024-04-16T05:34:56.059157+0000","last_undegraded":"2024-04-16T05:34:56.059157+0000","last_fullsized":"2024-04-16T05:34:56.059157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:04:34.322799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266617+0000","last_change":"2024-04-16T05:33:34.769486+0000","last_active":"2024-04-16T05:34:56.266617+0000","last_peered":"2024-04-16T05:34:56.266617+0000","last_clean":"2024-04-16T05:34:56.266617+0000","last_became_active":"2024-04-16T05:33:34.769025+0000","last_became_peered":"2024-04-16T05:33:34.769025+0000","last_unstale":"2024-04-16T05:34:56.266617+0000","last_undegraded":"2024-04-16T05:34:56.266617+0000","last_fullsized":"2024-04-16T05:34:56.266617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:10.382817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'39","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.228124+0000","last_change":"2024-04-16T05:32:33.746346+0000","last_active":"2024-04-16T05:34:56.228124+0000","last_peered":"2024-04-16T05:34:56.228124+0000","last_clean":"2024-04-16T05:34:56.228124+0000","last_became_active":"2024-04-16T05:32:33.746030+0000","last_became_peered":"2024-04-16T05:32:33.746030+0000","last_unstale":"2024-04-16T05:34:56.228124+0000","last_undegraded":"2024-04-16T05:34:56.228124+0000","last_fullsized":"2024-04-16T05:34:56.228124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:54:35.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058863+0000","last_change":"2024-04-16T05:32:34.752283+0000","last_active":"2024-04-16T05:34:56.058863+0000","last_peered":"2024-04-16T05:34:56.058863+0000","last_clean":"2024-04-16T05:34:56.058863+0000","last_became_active":"2024-04-16T05:32:34.751904+0000","last_became_peered":"2024-04-16T05:32:34.751904+0000","last_unstale":"2024-04-16T05:34:56.058863+0000","last_undegraded":"2024-04-16T05:34:56.058863+0000","last_fullsized":"2024-04-16T05:34:56.058863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:15:52.688683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114065+0000","last_change":"2024-04-16T05:32:38.934194+0000","last_active":"2024-04-16T05:34:56.114065+0000","last_peered":"2024-04-16T05:34:56.114065+0000","last_clean":"2024-04-16T05:34:56.114065+0000","last_became_active":"2024-04-16T05:32:38.933636+0000","last_became_peered":"2024-04-16T05:32:38.933636+0000","last_unstale":"2024-04-16T05:34:56.114065+0000","last_undegraded":"2024-04-16T05:34:56.114065+0000","last_fullsized":"2024-04-16T05:34:56.114065+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:14:40.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267445+0000","last_change":"2024-04-16T05:32:33.744267+0000","last_active":"2024-04-16T05:34:56.267445+0000","last_peered":"2024-04-16T05:34:56.267445+0000","last_clean":"2024-04-16T05:34:56.267445+0000","last_became_active":"2024-04-16T05:32:33.743931+0000","last_became_peered":"2024-04-16T05:32:33.743931+0000","last_unstale":"2024-04-16T05:34:56.267445+0000","last_undegraded":"2024-04-16T05:34:56.267445+0000","last_fullsized":"2024-04-16T05:34:56.267445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:30.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059158+0000","last_change":"2024-04-16T05:32:34.756833+0000","last_active":"2024-04-16T05:34:56.059158+0000","last_peered":"2024-04-16T05:34:56.059158+0000","last_clean":"2024-04-16T05:34:56.059158+0000","last_became_active":"2024-04-16T05:32:34.756436+0000","last_became_peered":"2024-04-16T05:32:34.756436+0000","last_unstale":"2024-04-16T05:34:56.059158+0000","last_undegraded":"2024-04-16T05:34:56.059158+0000","last_fullsized":"2024-04-16T05:34:56.059158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:20.732946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639538+0000","last_change":"2024-04-16T05:32:34.752744+0000","last_active":"2024-04-16T05:34:56.639538+0000","last_peered":"2024-04-16T05:34:56.639538+0000","last_clean":"2024-04-16T05:34:56.639538+0000","last_became_active":"2024-04-16T05:32:34.752442+0000","last_became_peered":"2024-04-16T05:32:34.752442+0000","last_unstale":"2024-04-16T05:34:56.639538+0000","last_undegraded":"2024-04-16T05:34:56.639538+0000","last_fullsized":"2024-04-16T05:34:56.639538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:01:23.644692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264130+0000","last_change":"2024-04-16T05:32:34.755118+0000","last_active":"2024-04-16T05:34:56.264130+0000","last_peered":"2024-04-16T05:34:56.264130+0000","last_clean":"2024-04-16T05:34:56.264130+0000","last_became_active":"2024-04-16T05:32:34.754824+0000","last_became_peered":"2024-04-16T05:32:34.754824+0000","last_unstale":"2024-04-16T05:34:56.264130+0000","last_undegraded":"2024-04-16T05:34:56.264130+0000","last_fullsized":"2024-04-16T05:34:56.264130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:58:37.860305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264047+0000","last_change":"2024-04-16T05:32:34.756144+0000","last_active":"2024-04-16T05:34:56.264047+0000","last_peered":"2024-04-16T05:34:56.264047+0000","last_clean":"2024-04-16T05:34:56.264047+0000","last_became_active":"2024-04-16T05:32:34.755841+0000","last_became_peered":"2024-04-16T05:32:34.755841+0000","last_unstale":"2024-04-16T05:34:56.264047+0000","last_undegraded":"2024-04-16T05:34:56.264047+0000","last_fullsized":"2024-04-16T05:34:56.264047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:23:00.482444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266708+0000","last_change":"2024-04-16T05:32:34.766204+0000","last_active":"2024-04-16T05:34:56.266708+0000","last_peered":"2024-04-16T05:34:56.266708+0000","last_clean":"2024-04-16T05:34:56.266708+0000","last_became_active":"2024-04-16T05:32:34.765709+0000","last_became_peered":"2024-04-16T05:32:34.765709+0000","last_unstale":"2024-04-16T05:34:56.266708+0000","last_undegraded":"2024-04-16T05:34:56.266708+0000","last_fullsized":"2024-04-16T05:34:56.266708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:23:53.926477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267022+0000","last_change":"2024-04-16T05:32:34.758513+0000","last_active":"2024-04-16T05:34:56.267022+0000","last_peered":"2024-04-16T05:34:56.267022+0000","last_clean":"2024-04-16T05:34:56.267022+0000","last_became_active":"2024-04-16T05:32:34.758145+0000","last_became_peered":"2024-04-16T05:32:34.758145+0000","last_unstale":"2024-04-16T05:34:56.267022+0000","last_undegraded":"2024-04-16T05:34:56.267022+0000","last_fullsized":"2024-04-16T05:34:56.267022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:09:17.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639842+0000","last_change":"2024-04-16T05:32:34.750651+0000","last_active":"2024-04-16T05:34:56.639842+0000","last_peered":"2024-04-16T05:34:56.639842+0000","last_clean":"2024-04-16T05:34:56.639842+0000","last_became_active":"2024-04-16T05:32:34.750348+0000","last_became_peered":"2024-04-16T05:32:34.750348+0000","last_unstale":"2024-04-16T05:34:56.639842+0000","last_undegraded":"2024-04-16T05:34:56.639842+0000","last_fullsized":"2024-04-16T05:34:56.639842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:29.282449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264574+0000","last_change":"2024-04-16T05:32:34.750079+0000","last_active":"2024-04-16T05:34:56.264574+0000","last_peered":"2024-04-16T05:34:56.264574+0000","last_clean":"2024-04-16T05:34:56.264574+0000","last_became_active":"2024-04-16T05:32:34.749775+0000","last_became_peered":"2024-04-16T05:32:34.749775+0000","last_unstale":"2024-04-16T05:34:56.264574+0000","last_undegraded":"2024-04-16T05:34:56.264574+0000","last_fullsized":"2024-04-16T05:34:56.264574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:02.297258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107010+0000","last_change":"2024-04-16T05:32:34.752528+0000","last_active":"2024-04-16T05:34:56.107010+0000","last_peered":"2024-04-16T05:34:56.107010+0000","last_clean":"2024-04-16T05:34:56.107010+0000","last_became_active":"2024-04-16T05:32:34.752150+0000","last_became_peered":"2024-04-16T05:32:34.752150+0000","last_unstale":"2024-04-16T05:34:56.107010+0000","last_undegraded":"2024-04-16T05:34:56.107010+0000","last_fullsized":"2024-04-16T05:34:56.107010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:41:16.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114222+0000","last_change":"2024-04-16T05:32:34.758486+0000","last_active":"2024-04-16T05:34:56.114222+0000","last_peered":"2024-04-16T05:34:56.114222+0000","last_clean":"2024-04-16T05:34:56.114222+0000","last_became_active":"2024-04-16T05:32:34.754840+0000","last_became_peered":"2024-04-16T05:32:34.754840+0000","last_unstale":"2024-04-16T05:34:56.114222+0000","last_undegraded":"2024-04-16T05:34:56.114222+0000","last_fullsized":"2024-04-16T05:34:56.114222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:35.644337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105877+0000","last_change":"2024-04-16T05:32:34.758933+0000","last_active":"2024-04-16T05:34:56.105877+0000","last_peered":"2024-04-16T05:34:56.105877+0000","last_clean":"2024-04-16T05:34:56.105877+0000","last_became_active":"2024-04-16T05:32:34.758587+0000","last_became_peered":"2024-04-16T05:32:34.758587+0000","last_unstale":"2024-04-16T05:34:56.105877+0000","last_undegraded":"2024-04-16T05:34:56.105877+0000","last_fullsized":"2024-04-16T05:34:56.105877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:19:20.674248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264656+0000","last_change":"2024-04-16T05:32:34.750846+0000","last_active":"2024-04-16T05:34:56.264656+0000","last_peered":"2024-04-16T05:34:56.264656+0000","last_clean":"2024-04-16T05:34:56.264656+0000","last_became_active":"2024-04-16T05:32:34.750529+0000","last_became_peered":"2024-04-16T05:32:34.750529+0000","last_unstale":"2024-04-16T05:34:56.264656+0000","last_undegraded":"2024-04-16T05:34:56.264656+0000","last_fullsized":"2024-04-16T05:34:56.264656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:32.288459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266353+0000","last_change":"2024-04-16T05:32:34.752566+0000","last_active":"2024-04-16T05:34:56.266353+0000","last_peered":"2024-04-16T05:34:56.266353+0000","last_clean":"2024-04-16T05:34:56.266353+0000","last_became_active":"2024-04-16T05:32:34.752236+0000","last_became_peered":"2024-04-16T05:32:34.752236+0000","last_unstale":"2024-04-16T05:34:56.266353+0000","last_undegraded":"2024-04-16T05:34:56.266353+0000","last_fullsized":"2024-04-16T05:34:56.266353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:43:36.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639755+0000","last_change":"2024-04-16T05:32:34.750972+0000","last_active":"2024-04-16T05:34:56.639755+0000","last_peered":"2024-04-16T05:34:56.639755+0000","last_clean":"2024-04-16T05:34:56.639755+0000","last_became_active":"2024-04-16T05:32:34.750671+0000","last_became_peered":"2024-04-16T05:32:34.750671+0000","last_unstale":"2024-04-16T05:34:56.639755+0000","last_undegraded":"2024-04-16T05:34:56.639755+0000","last_fullsized":"2024-04-16T05:34:56.639755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:19:45.331816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108342,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597977,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":858,"num_read_kb":541,"num_write":289,"num_write_kb":1442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1793931,"data_compressed":80799,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":510665,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1298518,"data_compressed":67519,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":908179,"data_compressed":42764,"data_compressed_allocated":57344,"data_compressed_original":458752,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":904003,"data_compressed":42764,"data_compressed_allocated":57344,"data_compressed_original":458752,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":1107760,"data_compressed":59732,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":506043,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1073152,"data_stored":574049,"data_compressed":25249,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":507431,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":565698,"data_compressed":22271,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":637821,"data_compressed":30048,"data_compressed_allocated":40960,"data_compressed_original":196608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":511225,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1101824,"data_stored":1234025,"data_compressed":65888,"data_compressed_allocated":86016,"data_compressed_original":778240,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68096,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69944,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133608,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":202197,"data_compressed":7787,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:34:58.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:34:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 -- 172.21.15.32:0/2853903697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102090 msgr2=0x7fcb2c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 --2- 172.21.15.32:0/2853903697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102090 0x7fcb2c102510 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009900 tx=0x7fcb1402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 -- 172.21.15.32:0/2853903697 shutdown_connections 2024-04-16T05:34:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 --2- 172.21.15.32:0/2853903697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb2c102a50 0x7fcb2c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 --2- 172.21.15.32:0/2853903697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102090 0x7fcb2c102510 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 --2- 172.21.15.32:0/2853903697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c100f40 0x7fcb2c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 -- 172.21.15.32:0/2853903697 >> 172.21.15.32:0/2853903697 conn(0x7fcb2c0fc4b0 msgr2=0x7fcb2c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 -- 172.21.15.32:0/2853903697 shutdown_connections 2024-04-16T05:34:58.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.881+0000 7fcb30b96640 1 -- 172.21.15.32:0/2853903697 wait complete. 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb30b96640 1 Processor -- start 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb30b96640 1 -- start start 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb30b96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb2c100f40 0x7fcb2c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb30b96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c102090 0x7fcb2c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb30b96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102a50 0x7fcb2c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb30b96640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c10cbf0 con 0x7fcb2c102a50 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb30b96640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c10ca70 con 0x7fcb2c102090 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb30b96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c10cd70 con 0x7fcb2c100f40 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb2ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102a50 0x7fcb2c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb2ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102a50 0x7fcb2c1a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35736/0 (socket says 172.21.15.32:35736) 2024-04-16T05:34:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb29d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c102090 0x7fcb2c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb2ad9d640 1 -- 172.21.15.32:0/1947353149 learned_addr learned my addr 172.21.15.32:0/1947353149 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.882+0000 7fcb2a59c640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb2c100f40 0x7fcb2c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb29d9b640 1 -- 172.21.15.32:0/1947353149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb2c100f40 msgr2=0x7fcb2c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb29d9b640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb2c100f40 0x7fcb2c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb29d9b640 1 -- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102a50 msgr2=0x7fcb2c1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb29d9b640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102a50 0x7fcb2c1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb29d9b640 1 -- 172.21.15.32:0/1947353149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c1a21b0 con 0x7fcb2c102090 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb2ad9d640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102a50 0x7fcb2c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb29d9b640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c102090 0x7fcb2c19aad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb1402c170 tx=0x7fcb1402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb237fe640 1 -- 172.21.15.32:0/1947353149 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14041070 con 0x7fcb2c102090 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c1a2440 con 0x7fcb2c102090 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb237fe640 1 -- 172.21.15.32:0/1947353149 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14004450 con 0x7fcb2c102090 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb237fe640 1 -- 172.21.15.32:0/1947353149 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14003e50 con 0x7fcb2c102090 2024-04-16T05:34:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.883+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c1a2950 con 0x7fcb2c102090 2024-04-16T05:34:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.884+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c1a2c70 con 0x7fcb2c102090 2024-04-16T05:34:58.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: [16/Apr/2024:05:34:57] ENGINE Bus STARTING 2024-04-16T05:34:58.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: [16/Apr/2024:05:34:57] ENGINE Serving on http://172.21.15.32:8765 2024-04-16T05:34:58.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: [16/Apr/2024:05:34:57] ENGINE Serving on https://172.21.15.32:7150 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: [16/Apr/2024:05:34:57] ENGINE Bus STARTED 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/385068334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3625758070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1910072636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.885+0000 7fcb237fe640 1 -- 172.21.15.32:0/1947353149 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fcb14044070 con 0x7fcb2c102090 2024-04-16T05:34:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.886+0000 7fcb237fe640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcb180a3ec0 0x7fcb180a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.886+0000 7fcb2a59c640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcb180a3ec0 0x7fcb180a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.886+0000 7fcb237fe640 1 -- 172.21.15.32:0/1947353149 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fcb140f24f0 con 0x7fcb2c102090 2024-04-16T05:34:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.887+0000 7fcb2a59c640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcb180a3ec0 0x7fcb180a6380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009bd0 tx=0x7fcb1c005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:58.891+0000 7fcb237fe640 1 -- 172.21.15.32:0/1947353149 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb140be2b0 con 0x7fcb2c102090 2024-04-16T05:34:58.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: [16/Apr/2024:05:34:57] ENGINE Bus STARTING 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: [16/Apr/2024:05:34:57] ENGINE Serving on http://172.21.15.32:8765 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: [16/Apr/2024:05:34:57] ENGINE Serving on https://172.21.15.32:7150 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: [16/Apr/2024:05:34:57] ENGINE Bus STARTED 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/385068334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3625758070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:58.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1910072636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:34:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.089+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb2c0fe660 con 0x7fcb180a3ec0 2024-04-16T05:34:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.099+0000 7fcb237fe640 1 -- 172.21.15.32:0/1947353149 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339709 (secure 0 0 0) 0x7fcb2c0fe660 con 0x7fcb180a3ec0 2024-04-16T05:34:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-16T05:34:59.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcb180a3ec0 msgr2=0x7fcb180a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcb180a3ec0 0x7fcb180a6380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009bd0 tx=0x7fcb1c005f90 comp rx=0 tx=0).stop 2024-04-16T05:34:59.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c102090 msgr2=0x7fcb2c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c102090 0x7fcb2c19aad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb1402c170 tx=0x7fcb1402c650 comp rx=0 tx=0).stop 2024-04-16T05:34:59.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 shutdown_connections 2024-04-16T05:34:59.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcb180a3ec0 0x7fcb180a6380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c102a50 0x7fcb2c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c102090 0x7fcb2c19aad0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 --2- 172.21.15.32:0/1947353149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb2c100f40 0x7fcb2c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 >> 172.21.15.32:0/1947353149 conn(0x7fcb2c0fc4b0 msgr2=0x7fcb2c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 shutdown_connections 2024-04-16T05:34:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.102+0000 7fcb30b96640 1 -- 172.21.15.32:0/1947353149 wait complete. 2024-04-16T05:34:59.112 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-16T05:34:58.020779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188579,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":994,"num_read_kb":660,"num_write":472,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361596,"kb_used_data":12396,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580228,"statfs":{"total":1151940427776,"available":1151570153472,"internally_reserved":0,"allocated":12693504,"data_stored":9265417,"data_compressed":450707,"data_compressed_allocated":610304,"data_compressed_original":3735552,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.987473"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.640068+0000","last_change":"2024-04-16T05:32:34.759518+0000","last_active":"2024-04-16T05:34:56.640068+0000","last_peered":"2024-04-16T05:34:56.640068+0000","last_clean":"2024-04-16T05:34:56.640068+0000","last_became_active":"2024-04-16T05:32:34.759155+0000","last_became_peered":"2024-04-16T05:32:34.759155+0000","last_unstale":"2024-04-16T05:34:56.640068+0000","last_undegraded":"2024-04-16T05:34:56.640068+0000","last_fullsized":"2024-04-16T05:34:56.640068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:39.935886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266968+0000","last_change":"2024-04-16T05:32:34.758553+0000","last_active":"2024-04-16T05:34:56.266968+0000","last_peered":"2024-04-16T05:34:56.266968+0000","last_clean":"2024-04-16T05:34:56.266968+0000","last_became_active":"2024-04-16T05:32:34.758242+0000","last_became_peered":"2024-04-16T05:32:34.758242+0000","last_unstale":"2024-04-16T05:34:56.266968+0000","last_undegraded":"2024-04-16T05:34:56.266968+0000","last_fullsized":"2024-04-16T05:34:56.266968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:09.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266926+0000","last_change":"2024-04-16T05:32:34.754815+0000","last_active":"2024-04-16T05:34:56.266926+0000","last_peered":"2024-04-16T05:34:56.266926+0000","last_clean":"2024-04-16T05:34:56.266926+0000","last_became_active":"2024-04-16T05:32:34.754488+0000","last_became_peered":"2024-04-16T05:32:34.754488+0000","last_unstale":"2024-04-16T05:34:56.266926+0000","last_undegraded":"2024-04-16T05:34:56.266926+0000","last_fullsized":"2024-04-16T05:34:56.266926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:10:32.379770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007584+0000","last_change":"2024-04-16T05:33:34.766742+0000","last_active":"2024-04-16T05:34:56.007584+0000","last_peered":"2024-04-16T05:34:56.007584+0000","last_clean":"2024-04-16T05:34:56.007584+0000","last_became_active":"2024-04-16T05:33:34.766332+0000","last_became_peered":"2024-04-16T05:33:34.766332+0000","last_unstale":"2024-04-16T05:34:56.007584+0000","last_undegraded":"2024-04-16T05:34:56.007584+0000","last_fullsized":"2024-04-16T05:34:56.007584+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:52:14.644496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008710+0000","last_change":"2024-04-16T05:32:34.755623+0000","last_active":"2024-04-16T05:34:56.008710+0000","last_peered":"2024-04-16T05:34:56.008710+0000","last_clean":"2024-04-16T05:34:56.008710+0000","last_became_active":"2024-04-16T05:32:34.754205+0000","last_became_peered":"2024-04-16T05:32:34.754205+0000","last_unstale":"2024-04-16T05:34:56.008710+0000","last_undegraded":"2024-04-16T05:34:56.008710+0000","last_fullsized":"2024-04-16T05:34:56.008710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:34:07.998734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117661+0000","last_change":"2024-04-16T05:32:34.753767+0000","last_active":"2024-04-16T05:34:56.117661+0000","last_peered":"2024-04-16T05:34:56.117661+0000","last_clean":"2024-04-16T05:34:56.117661+0000","last_became_active":"2024-04-16T05:32:34.753448+0000","last_became_peered":"2024-04-16T05:32:34.753448+0000","last_unstale":"2024-04-16T05:34:56.117661+0000","last_undegraded":"2024-04-16T05:34:56.117661+0000","last_fullsized":"2024-04-16T05:34:56.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:27:42.798615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058759+0000","last_change":"2024-04-16T05:32:34.754049+0000","last_active":"2024-04-16T05:34:56.058759+0000","last_peered":"2024-04-16T05:34:56.058759+0000","last_clean":"2024-04-16T05:34:56.058759+0000","last_became_active":"2024-04-16T05:32:34.753757+0000","last_became_peered":"2024-04-16T05:32:34.753757+0000","last_unstale":"2024-04-16T05:34:56.058759+0000","last_undegraded":"2024-04-16T05:34:56.058759+0000","last_fullsized":"2024-04-16T05:34:56.058759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:42:26.747100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105386+0000","last_change":"2024-04-16T05:32:34.762950+0000","last_active":"2024-04-16T05:34:56.105386+0000","last_peered":"2024-04-16T05:34:56.105386+0000","last_clean":"2024-04-16T05:34:56.105386+0000","last_became_active":"2024-04-16T05:32:34.762073+0000","last_became_peered":"2024-04-16T05:32:34.762073+0000","last_unstale":"2024-04-16T05:34:56.105386+0000","last_undegraded":"2024-04-16T05:34:56.105386+0000","last_fullsized":"2024-04-16T05:34:56.105386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:57:26.623655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265111+0000","last_change":"2024-04-16T05:32:34.751027+0000","last_active":"2024-04-16T05:34:56.265111+0000","last_peered":"2024-04-16T05:34:56.265111+0000","last_clean":"2024-04-16T05:34:56.265111+0000","last_became_active":"2024-04-16T05:32:34.750697+0000","last_became_peered":"2024-04-16T05:32:34.750697+0000","last_unstale":"2024-04-16T05:34:56.265111+0000","last_undegraded":"2024-04-16T05:34:56.265111+0000","last_fullsized":"2024-04-16T05:34:56.265111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:39:33.834972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.118031+0000","last_change":"2024-04-16T05:32:34.752293+0000","last_active":"2024-04-16T05:34:56.118031+0000","last_peered":"2024-04-16T05:34:56.118031+0000","last_clean":"2024-04-16T05:34:56.118031+0000","last_became_active":"2024-04-16T05:32:34.752008+0000","last_became_peered":"2024-04-16T05:32:34.752008+0000","last_unstale":"2024-04-16T05:34:56.118031+0000","last_undegraded":"2024-04-16T05:34:56.118031+0000","last_fullsized":"2024-04-16T05:34:56.118031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:02:17.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059051+0000","last_change":"2024-04-16T05:32:34.760961+0000","last_active":"2024-04-16T05:34:56.059051+0000","last_peered":"2024-04-16T05:34:56.059051+0000","last_clean":"2024-04-16T05:34:56.059051+0000","last_became_active":"2024-04-16T05:32:34.760640+0000","last_became_peered":"2024-04-16T05:32:34.760640+0000","last_unstale":"2024-04-16T05:34:56.059051+0000","last_undegraded":"2024-04-16T05:34:56.059051+0000","last_fullsized":"2024-04-16T05:34:56.059051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:47:28.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009246+0000","last_change":"2024-04-16T05:32:34.748646+0000","last_active":"2024-04-16T05:34:56.009246+0000","last_peered":"2024-04-16T05:34:56.009246+0000","last_clean":"2024-04-16T05:34:56.009246+0000","last_became_active":"2024-04-16T05:32:34.748090+0000","last_became_peered":"2024-04-16T05:32:34.748090+0000","last_unstale":"2024-04-16T05:34:56.009246+0000","last_undegraded":"2024-04-16T05:34:56.009246+0000","last_fullsized":"2024-04-16T05:34:56.009246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:00:19.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266536+0000","last_change":"2024-04-16T05:32:34.747025+0000","last_active":"2024-04-16T05:34:56.266536+0000","last_peered":"2024-04-16T05:34:56.266536+0000","last_clean":"2024-04-16T05:34:56.266536+0000","last_became_active":"2024-04-16T05:32:34.746711+0000","last_became_peered":"2024-04-16T05:32:34.746711+0000","last_unstale":"2024-04-16T05:34:56.266536+0000","last_undegraded":"2024-04-16T05:34:56.266536+0000","last_fullsized":"2024-04-16T05:34:56.266536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:40:30.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266616+0000","last_change":"2024-04-16T05:32:34.756756+0000","last_active":"2024-04-16T05:34:56.266616+0000","last_peered":"2024-04-16T05:34:56.266616+0000","last_clean":"2024-04-16T05:34:56.266616+0000","last_became_active":"2024-04-16T05:32:34.756442+0000","last_became_peered":"2024-04-16T05:32:34.756442+0000","last_unstale":"2024-04-16T05:34:56.266616+0000","last_undegraded":"2024-04-16T05:34:56.266616+0000","last_fullsized":"2024-04-16T05:34:56.266616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:33.864944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266459+0000","last_change":"2024-04-16T05:32:34.765443+0000","last_active":"2024-04-16T05:34:56.266459+0000","last_peered":"2024-04-16T05:34:56.266459+0000","last_clean":"2024-04-16T05:34:56.266459+0000","last_became_active":"2024-04-16T05:32:34.765133+0000","last_became_peered":"2024-04-16T05:32:34.765133+0000","last_unstale":"2024-04-16T05:34:56.266459+0000","last_undegraded":"2024-04-16T05:34:56.266459+0000","last_fullsized":"2024-04-16T05:34:56.266459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:31.452184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264372+0000","last_change":"2024-04-16T05:32:34.757577+0000","last_active":"2024-04-16T05:34:56.264372+0000","last_peered":"2024-04-16T05:34:56.264372+0000","last_clean":"2024-04-16T05:34:56.264372+0000","last_became_active":"2024-04-16T05:32:34.753451+0000","last_became_peered":"2024-04-16T05:32:34.753451+0000","last_unstale":"2024-04-16T05:34:56.264372+0000","last_undegraded":"2024-04-16T05:34:56.264372+0000","last_fullsized":"2024-04-16T05:34:56.264372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:56.617824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639970+0000","last_change":"2024-04-16T05:32:34.759776+0000","last_active":"2024-04-16T05:34:56.639970+0000","last_peered":"2024-04-16T05:34:56.639970+0000","last_clean":"2024-04-16T05:34:56.639970+0000","last_became_active":"2024-04-16T05:32:34.759371+0000","last_became_peered":"2024-04-16T05:32:34.759371+0000","last_unstale":"2024-04-16T05:34:56.639970+0000","last_undegraded":"2024-04-16T05:34:56.639970+0000","last_fullsized":"2024-04-16T05:34:56.639970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:36.458515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007873+0000","last_change":"2024-04-16T05:32:34.747803+0000","last_active":"2024-04-16T05:34:56.007873+0000","last_peered":"2024-04-16T05:34:56.007873+0000","last_clean":"2024-04-16T05:34:56.007873+0000","last_became_active":"2024-04-16T05:32:34.747472+0000","last_became_peered":"2024-04-16T05:32:34.747472+0000","last_unstale":"2024-04-16T05:34:56.007873+0000","last_undegraded":"2024-04-16T05:34:56.007873+0000","last_fullsized":"2024-04-16T05:34:56.007873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:41.118574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264411+0000","last_change":"2024-04-16T05:32:34.754151+0000","last_active":"2024-04-16T05:34:56.264411+0000","last_peered":"2024-04-16T05:34:56.264411+0000","last_clean":"2024-04-16T05:34:56.264411+0000","last_became_active":"2024-04-16T05:32:34.753814+0000","last_became_peered":"2024-04-16T05:32:34.753814+0000","last_unstale":"2024-04-16T05:34:56.264411+0000","last_undegraded":"2024-04-16T05:34:56.264411+0000","last_fullsized":"2024-04-16T05:34:56.264411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:13:00.207978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264276+0000","last_change":"2024-04-16T05:32:38.938563+0000","last_active":"2024-04-16T05:34:56.264276+0000","last_peered":"2024-04-16T05:34:56.264276+0000","last_clean":"2024-04-16T05:34:56.264276+0000","last_became_active":"2024-04-16T05:32:38.938097+0000","last_became_peered":"2024-04-16T05:32:38.938097+0000","last_unstale":"2024-04-16T05:34:56.264276+0000","last_undegraded":"2024-04-16T05:34:56.264276+0000","last_fullsized":"2024-04-16T05:34:56.264276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:47:52.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007623+0000","last_change":"2024-04-16T05:32:33.735694+0000","last_active":"2024-04-16T05:34:56.007623+0000","last_peered":"2024-04-16T05:34:56.007623+0000","last_clean":"2024-04-16T05:34:56.007623+0000","last_became_active":"2024-04-16T05:32:33.735414+0000","last_became_peered":"2024-04-16T05:32:33.735414+0000","last_unstale":"2024-04-16T05:34:56.007623+0000","last_undegraded":"2024-04-16T05:34:56.007623+0000","last_fullsized":"2024-04-16T05:34:56.007623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:19:06.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107049+0000","last_change":"2024-04-16T05:32:34.762315+0000","last_active":"2024-04-16T05:34:56.107049+0000","last_peered":"2024-04-16T05:34:56.107049+0000","last_clean":"2024-04-16T05:34:56.107049+0000","last_became_active":"2024-04-16T05:32:34.761969+0000","last_became_peered":"2024-04-16T05:32:34.761969+0000","last_unstale":"2024-04-16T05:34:56.107049+0000","last_undegraded":"2024-04-16T05:34:56.107049+0000","last_fullsized":"2024-04-16T05:34:56.107049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:00:45.166966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008832+0000","last_change":"2024-04-16T05:32:38.940681+0000","last_active":"2024-04-16T05:34:56.008832+0000","last_peered":"2024-04-16T05:34:56.008832+0000","last_clean":"2024-04-16T05:34:56.008832+0000","last_became_active":"2024-04-16T05:32:38.940414+0000","last_became_peered":"2024-04-16T05:32:38.940414+0000","last_unstale":"2024-04-16T05:34:56.008832+0000","last_undegraded":"2024-04-16T05:34:56.008832+0000","last_fullsized":"2024-04-16T05:34:56.008832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:46:50.061607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107418+0000","last_change":"2024-04-16T05:32:33.748333+0000","last_active":"2024-04-16T05:34:56.107418+0000","last_peered":"2024-04-16T05:34:56.107418+0000","last_clean":"2024-04-16T05:34:56.107418+0000","last_became_active":"2024-04-16T05:32:33.748022+0000","last_became_peered":"2024-04-16T05:32:33.748022+0000","last_unstale":"2024-04-16T05:34:56.107418+0000","last_undegraded":"2024-04-16T05:34:56.107418+0000","last_fullsized":"2024-04-16T05:34:56.107418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:50:27.771245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266977+0000","last_change":"2024-04-16T05:32:34.764832+0000","last_active":"2024-04-16T05:34:56.266977+0000","last_peered":"2024-04-16T05:34:56.266977+0000","last_clean":"2024-04-16T05:34:56.266977+0000","last_became_active":"2024-04-16T05:32:34.763180+0000","last_became_peered":"2024-04-16T05:32:34.763180+0000","last_unstale":"2024-04-16T05:34:56.266977+0000","last_undegraded":"2024-04-16T05:34:56.266977+0000","last_fullsized":"2024-04-16T05:34:56.266977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:37:27.804376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009531+0000","last_change":"2024-04-16T05:32:38.937226+0000","last_active":"2024-04-16T05:34:56.009531+0000","last_peered":"2024-04-16T05:34:56.009531+0000","last_clean":"2024-04-16T05:34:56.009531+0000","last_became_active":"2024-04-16T05:32:38.936770+0000","last_became_peered":"2024-04-16T05:32:38.936770+0000","last_unstale":"2024-04-16T05:34:56.009531+0000","last_undegraded":"2024-04-16T05:34:56.009531+0000","last_fullsized":"2024-04-16T05:34:56.009531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:05:28.059175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639811+0000","last_change":"2024-04-16T05:33:34.775689+0000","last_active":"2024-04-16T05:34:56.639811+0000","last_peered":"2024-04-16T05:34:56.639811+0000","last_clean":"2024-04-16T05:34:56.639811+0000","last_became_active":"2024-04-16T05:33:34.770520+0000","last_became_peered":"2024-04-16T05:33:34.770520+0000","last_unstale":"2024-04-16T05:34:56.639811+0000","last_undegraded":"2024-04-16T05:34:56.639811+0000","last_fullsized":"2024-04-16T05:34:56.639811+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:40:57.616795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264658+0000","last_change":"2024-04-16T05:32:34.758891+0000","last_active":"2024-04-16T05:34:56.264658+0000","last_peered":"2024-04-16T05:34:56.264658+0000","last_clean":"2024-04-16T05:34:56.264658+0000","last_became_active":"2024-04-16T05:32:34.758559+0000","last_became_peered":"2024-04-16T05:32:34.758559+0000","last_unstale":"2024-04-16T05:34:56.264658+0000","last_undegraded":"2024-04-16T05:34:56.264658+0000","last_fullsized":"2024-04-16T05:34:56.264658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:31.213992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008864+0000","last_change":"2024-04-16T05:32:38.937163+0000","last_active":"2024-04-16T05:34:56.008864+0000","last_peered":"2024-04-16T05:34:56.008864+0000","last_clean":"2024-04-16T05:34:56.008864+0000","last_became_active":"2024-04-16T05:32:38.936745+0000","last_became_peered":"2024-04-16T05:32:38.936745+0000","last_unstale":"2024-04-16T05:34:56.008864+0000","last_undegraded":"2024-04-16T05:34:56.008864+0000","last_fullsized":"2024-04-16T05:34:56.008864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:54:37.915041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008806+0000","last_change":"2024-04-16T05:32:33.731842+0000","last_active":"2024-04-16T05:34:56.008806+0000","last_peered":"2024-04-16T05:34:56.008806+0000","last_clean":"2024-04-16T05:34:56.008806+0000","last_became_active":"2024-04-16T05:32:33.731417+0000","last_became_peered":"2024-04-16T05:32:33.731417+0000","last_unstale":"2024-04-16T05:34:56.008806+0000","last_undegraded":"2024-04-16T05:34:56.008806+0000","last_fullsized":"2024-04-16T05:34:56.008806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:04:03.141533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114569+0000","last_change":"2024-04-16T05:32:34.755062+0000","last_active":"2024-04-16T05:34:56.114569+0000","last_peered":"2024-04-16T05:34:56.114569+0000","last_clean":"2024-04-16T05:34:56.114569+0000","last_became_active":"2024-04-16T05:32:34.754759+0000","last_became_peered":"2024-04-16T05:32:34.754759+0000","last_unstale":"2024-04-16T05:34:56.114569+0000","last_undegraded":"2024-04-16T05:34:56.114569+0000","last_fullsized":"2024-04-16T05:34:56.114569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:53:59.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264299+0000","last_change":"2024-04-16T05:32:38.935646+0000","last_active":"2024-04-16T05:34:56.264299+0000","last_peered":"2024-04-16T05:34:56.264299+0000","last_clean":"2024-04-16T05:34:56.264299+0000","last_became_active":"2024-04-16T05:32:38.935206+0000","last_became_peered":"2024-04-16T05:32:38.935206+0000","last_unstale":"2024-04-16T05:34:56.264299+0000","last_undegraded":"2024-04-16T05:34:56.264299+0000","last_fullsized":"2024-04-16T05:34:56.264299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:09.772136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114682+0000","last_change":"2024-04-16T05:32:33.739343+0000","last_active":"2024-04-16T05:34:56.114682+0000","last_peered":"2024-04-16T05:34:56.114682+0000","last_clean":"2024-04-16T05:34:56.114682+0000","last_became_active":"2024-04-16T05:32:33.738982+0000","last_became_peered":"2024-04-16T05:32:33.738982+0000","last_unstale":"2024-04-16T05:34:56.114682+0000","last_undegraded":"2024-04-16T05:34:56.114682+0000","last_fullsized":"2024-04-16T05:34:56.114682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:36.002804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008721+0000","last_change":"2024-04-16T05:32:34.749316+0000","last_active":"2024-04-16T05:34:56.008721+0000","last_peered":"2024-04-16T05:34:56.008721+0000","last_clean":"2024-04-16T05:34:56.008721+0000","last_became_active":"2024-04-16T05:32:34.748976+0000","last_became_peered":"2024-04-16T05:32:34.748976+0000","last_unstale":"2024-04-16T05:34:56.008721+0000","last_undegraded":"2024-04-16T05:34:56.008721+0000","last_fullsized":"2024-04-16T05:34:56.008721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:16:29.569402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264259+0000","last_change":"2024-04-16T05:32:38.938633+0000","last_active":"2024-04-16T05:34:56.264259+0000","last_peered":"2024-04-16T05:34:56.264259+0000","last_clean":"2024-04-16T05:34:56.264259+0000","last_became_active":"2024-04-16T05:32:38.938215+0000","last_became_peered":"2024-04-16T05:32:38.938215+0000","last_unstale":"2024-04-16T05:34:56.264259+0000","last_undegraded":"2024-04-16T05:34:56.264259+0000","last_fullsized":"2024-04-16T05:34:56.264259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:05.078109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117612+0000","last_change":"2024-04-16T05:32:33.744262+0000","last_active":"2024-04-16T05:34:56.117612+0000","last_peered":"2024-04-16T05:34:56.117612+0000","last_clean":"2024-04-16T05:34:56.117612+0000","last_became_active":"2024-04-16T05:32:33.743866+0000","last_became_peered":"2024-04-16T05:32:33.743866+0000","last_unstale":"2024-04-16T05:34:56.117612+0000","last_undegraded":"2024-04-16T05:34:56.117612+0000","last_fullsized":"2024-04-16T05:34:56.117612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:22:03.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114365+0000","last_change":"2024-04-16T05:32:34.750201+0000","last_active":"2024-04-16T05:34:56.114365+0000","last_peered":"2024-04-16T05:34:56.114365+0000","last_clean":"2024-04-16T05:34:56.114365+0000","last_became_active":"2024-04-16T05:32:34.749900+0000","last_became_peered":"2024-04-16T05:32:34.749900+0000","last_unstale":"2024-04-16T05:34:56.114365+0000","last_undegraded":"2024-04-16T05:34:56.114365+0000","last_fullsized":"2024-04-16T05:34:56.114365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:42.708584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639920+0000","last_change":"2024-04-16T05:32:38.931277+0000","last_active":"2024-04-16T05:34:56.639920+0000","last_peered":"2024-04-16T05:34:56.639920+0000","last_clean":"2024-04-16T05:34:56.639920+0000","last_became_active":"2024-04-16T05:32:38.930888+0000","last_became_peered":"2024-04-16T05:32:38.930888+0000","last_unstale":"2024-04-16T05:34:56.639920+0000","last_undegraded":"2024-04-16T05:34:56.639920+0000","last_fullsized":"2024-04-16T05:34:56.639920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:27:31.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264559+0000","last_change":"2024-04-16T05:32:33.747740+0000","last_active":"2024-04-16T05:34:56.264559+0000","last_peered":"2024-04-16T05:34:56.264559+0000","last_clean":"2024-04-16T05:34:56.264559+0000","last_became_active":"2024-04-16T05:32:33.747313+0000","last_became_peered":"2024-04-16T05:32:33.747313+0000","last_unstale":"2024-04-16T05:34:56.264559+0000","last_undegraded":"2024-04-16T05:34:56.264559+0000","last_fullsized":"2024-04-16T05:34:56.264559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:05:03.937790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105220+0000","last_change":"2024-04-16T05:32:34.760075+0000","last_active":"2024-04-16T05:34:56.105220+0000","last_peered":"2024-04-16T05:34:56.105220+0000","last_clean":"2024-04-16T05:34:56.105220+0000","last_became_active":"2024-04-16T05:32:34.759569+0000","last_became_peered":"2024-04-16T05:32:34.759569+0000","last_unstale":"2024-04-16T05:34:56.105220+0000","last_undegraded":"2024-04-16T05:34:56.105220+0000","last_fullsized":"2024-04-16T05:34:56.105220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:54.222877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266895+0000","last_change":"2024-04-16T05:32:38.928568+0000","last_active":"2024-04-16T05:34:56.266895+0000","last_peered":"2024-04-16T05:34:56.266895+0000","last_clean":"2024-04-16T05:34:56.266895+0000","last_became_active":"2024-04-16T05:32:38.928109+0000","last_became_peered":"2024-04-16T05:32:38.928109+0000","last_unstale":"2024-04-16T05:34:56.266895+0000","last_undegraded":"2024-04-16T05:34:56.266895+0000","last_fullsized":"2024-04-16T05:34:56.266895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:24:14.293528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114216+0000","last_change":"2024-04-16T05:32:33.737803+0000","last_active":"2024-04-16T05:34:56.114216+0000","last_peered":"2024-04-16T05:34:56.114216+0000","last_clean":"2024-04-16T05:34:56.114216+0000","last_became_active":"2024-04-16T05:32:33.737448+0000","last_became_peered":"2024-04-16T05:32:33.737448+0000","last_unstale":"2024-04-16T05:34:56.114216+0000","last_undegraded":"2024-04-16T05:34:56.114216+0000","last_fullsized":"2024-04-16T05:34:56.114216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:44:54.302323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007203+0000","last_change":"2024-04-16T05:32:34.754070+0000","last_active":"2024-04-16T05:34:56.007203+0000","last_peered":"2024-04-16T05:34:56.007203+0000","last_clean":"2024-04-16T05:34:56.007203+0000","last_became_active":"2024-04-16T05:32:34.753717+0000","last_became_peered":"2024-04-16T05:32:34.753717+0000","last_unstale":"2024-04-16T05:34:56.007203+0000","last_undegraded":"2024-04-16T05:34:56.007203+0000","last_fullsized":"2024-04-16T05:34:56.007203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:58:56.738245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.106064+0000","last_change":"2024-04-16T05:32:38.931594+0000","last_active":"2024-04-16T05:34:56.106064+0000","last_peered":"2024-04-16T05:34:56.106064+0000","last_clean":"2024-04-16T05:34:56.106064+0000","last_became_active":"2024-04-16T05:32:38.931188+0000","last_became_peered":"2024-04-16T05:32:38.931188+0000","last_unstale":"2024-04-16T05:34:56.106064+0000","last_undegraded":"2024-04-16T05:34:56.106064+0000","last_fullsized":"2024-04-16T05:34:56.106064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:55.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.118099+0000","last_change":"2024-04-16T05:32:33.739752+0000","last_active":"2024-04-16T05:34:56.118099+0000","last_peered":"2024-04-16T05:34:56.118099+0000","last_clean":"2024-04-16T05:34:56.118099+0000","last_became_active":"2024-04-16T05:32:33.739444+0000","last_became_peered":"2024-04-16T05:32:33.739444+0000","last_unstale":"2024-04-16T05:34:56.118099+0000","last_undegraded":"2024-04-16T05:34:56.118099+0000","last_fullsized":"2024-04-16T05:34:56.118099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:04:18.091618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265179+0000","last_change":"2024-04-16T05:32:34.759197+0000","last_active":"2024-04-16T05:34:56.265179+0000","last_peered":"2024-04-16T05:34:56.265179+0000","last_clean":"2024-04-16T05:34:56.265179+0000","last_became_active":"2024-04-16T05:32:34.758783+0000","last_became_peered":"2024-04-16T05:32:34.758783+0000","last_unstale":"2024-04-16T05:34:56.265179+0000","last_undegraded":"2024-04-16T05:34:56.265179+0000","last_fullsized":"2024-04-16T05:34:56.265179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:23:37.020254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007139+0000","last_change":"2024-04-16T05:32:38.936198+0000","last_active":"2024-04-16T05:34:56.007139+0000","last_peered":"2024-04-16T05:34:56.007139+0000","last_clean":"2024-04-16T05:34:56.007139+0000","last_became_active":"2024-04-16T05:32:38.935698+0000","last_became_peered":"2024-04-16T05:32:38.935698+0000","last_unstale":"2024-04-16T05:34:56.007139+0000","last_undegraded":"2024-04-16T05:34:56.007139+0000","last_fullsized":"2024-04-16T05:34:56.007139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:17:35.602408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058994+0000","last_change":"2024-04-16T05:32:33.737653+0000","last_active":"2024-04-16T05:34:56.058994+0000","last_peered":"2024-04-16T05:34:56.058994+0000","last_clean":"2024-04-16T05:34:56.058994+0000","last_became_active":"2024-04-16T05:32:33.737342+0000","last_became_peered":"2024-04-16T05:32:33.737342+0000","last_unstale":"2024-04-16T05:34:56.058994+0000","last_undegraded":"2024-04-16T05:34:56.058994+0000","last_fullsized":"2024-04-16T05:34:56.058994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:17:35.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266896+0000","last_change":"2024-04-16T05:32:34.765722+0000","last_active":"2024-04-16T05:34:56.266896+0000","last_peered":"2024-04-16T05:34:56.266896+0000","last_clean":"2024-04-16T05:34:56.266896+0000","last_became_active":"2024-04-16T05:32:34.765413+0000","last_became_peered":"2024-04-16T05:32:34.765413+0000","last_unstale":"2024-04-16T05:34:56.266896+0000","last_undegraded":"2024-04-16T05:34:56.266896+0000","last_fullsized":"2024-04-16T05:34:56.266896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:03.884041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009328+0000","last_change":"2024-04-16T05:32:38.936186+0000","last_active":"2024-04-16T05:34:56.009328+0000","last_peered":"2024-04-16T05:34:56.009328+0000","last_clean":"2024-04-16T05:34:56.009328+0000","last_became_active":"2024-04-16T05:32:38.935738+0000","last_became_peered":"2024-04-16T05:32:38.935738+0000","last_unstale":"2024-04-16T05:34:56.009328+0000","last_undegraded":"2024-04-16T05:34:56.009328+0000","last_fullsized":"2024-04-16T05:34:56.009328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:10.519095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114565+0000","last_change":"2024-04-16T05:32:33.742991+0000","last_active":"2024-04-16T05:34:56.114565+0000","last_peered":"2024-04-16T05:34:56.114565+0000","last_clean":"2024-04-16T05:34:56.114565+0000","last_became_active":"2024-04-16T05:32:33.742659+0000","last_became_peered":"2024-04-16T05:32:33.742659+0000","last_unstale":"2024-04-16T05:34:56.114565+0000","last_undegraded":"2024-04-16T05:34:56.114565+0000","last_fullsized":"2024-04-16T05:34:56.114565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:23.756123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264808+0000","last_change":"2024-04-16T05:32:34.759926+0000","last_active":"2024-04-16T05:34:56.264808+0000","last_peered":"2024-04-16T05:34:56.264808+0000","last_clean":"2024-04-16T05:34:56.264808+0000","last_became_active":"2024-04-16T05:32:34.759626+0000","last_became_peered":"2024-04-16T05:32:34.759626+0000","last_unstale":"2024-04-16T05:34:56.264808+0000","last_undegraded":"2024-04-16T05:34:56.264808+0000","last_fullsized":"2024-04-16T05:34:56.264808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:50.314638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007364+0000","last_change":"2024-04-16T05:32:38.942135+0000","last_active":"2024-04-16T05:34:56.007364+0000","last_peered":"2024-04-16T05:34:56.007364+0000","last_clean":"2024-04-16T05:34:56.007364+0000","last_became_active":"2024-04-16T05:32:38.941534+0000","last_became_peered":"2024-04-16T05:32:38.941534+0000","last_unstale":"2024-04-16T05:34:56.007364+0000","last_undegraded":"2024-04-16T05:34:56.007364+0000","last_fullsized":"2024-04-16T05:34:56.007364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:20.494516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":323,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.873982+0000","last_change":"2024-04-16T05:32:33.734275+0000","last_active":"2024-04-16T05:34:56.873982+0000","last_peered":"2024-04-16T05:34:56.873982+0000","last_clean":"2024-04-16T05:34:56.873982+0000","last_became_active":"2024-04-16T05:32:33.733906+0000","last_became_peered":"2024-04-16T05:32:33.733906+0000","last_unstale":"2024-04-16T05:34:56.873982+0000","last_undegraded":"2024-04-16T05:34:56.873982+0000","last_fullsized":"2024-04-16T05:34:56.873982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:39:23.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":195,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008907+0000","last_change":"2024-04-16T05:32:34.750179+0000","last_active":"2024-04-16T05:34:56.008907+0000","last_peered":"2024-04-16T05:34:56.008907+0000","last_clean":"2024-04-16T05:34:56.008907+0000","last_became_active":"2024-04-16T05:32:34.749773+0000","last_became_peered":"2024-04-16T05:32:34.749773+0000","last_unstale":"2024-04-16T05:34:56.008907+0000","last_undegraded":"2024-04-16T05:34:56.008907+0000","last_fullsized":"2024-04-16T05:34:56.008907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:37.621191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114863+0000","last_change":"2024-04-16T05:32:38.935690+0000","last_active":"2024-04-16T05:34:56.114863+0000","last_peered":"2024-04-16T05:34:56.114863+0000","last_clean":"2024-04-16T05:34:56.114863+0000","last_became_active":"2024-04-16T05:32:38.935282+0000","last_became_peered":"2024-04-16T05:32:38.935282+0000","last_unstale":"2024-04-16T05:34:56.114863+0000","last_undegraded":"2024-04-16T05:34:56.114863+0000","last_fullsized":"2024-04-16T05:34:56.114863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:25:00.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639603+0000","last_change":"2024-04-16T05:33:34.782195+0000","last_active":"2024-04-16T05:34:56.639603+0000","last_peered":"2024-04-16T05:34:56.639603+0000","last_clean":"2024-04-16T05:34:56.639603+0000","last_became_active":"2024-04-16T05:33:34.771490+0000","last_became_peered":"2024-04-16T05:33:34.771490+0000","last_unstale":"2024-04-16T05:34:56.639603+0000","last_undegraded":"2024-04-16T05:34:56.639603+0000","last_fullsized":"2024-04-16T05:34:56.639603+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:52.513976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265873+0000","last_change":"2024-04-16T05:32:34.748760+0000","last_active":"2024-04-16T05:34:56.265873+0000","last_peered":"2024-04-16T05:34:56.265873+0000","last_clean":"2024-04-16T05:34:56.265873+0000","last_became_active":"2024-04-16T05:32:34.748432+0000","last_became_peered":"2024-04-16T05:32:34.748432+0000","last_unstale":"2024-04-16T05:34:56.265873+0000","last_undegraded":"2024-04-16T05:34:56.265873+0000","last_fullsized":"2024-04-16T05:34:56.265873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:12:42.859342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639575+0000","last_change":"2024-04-16T05:32:38.929705+0000","last_active":"2024-04-16T05:34:56.639575+0000","last_peered":"2024-04-16T05:34:56.639575+0000","last_clean":"2024-04-16T05:34:56.639575+0000","last_became_active":"2024-04-16T05:32:38.929282+0000","last_became_peered":"2024-04-16T05:32:38.929282+0000","last_unstale":"2024-04-16T05:34:56.639575+0000","last_undegraded":"2024-04-16T05:34:56.639575+0000","last_fullsized":"2024-04-16T05:34:56.639575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:51:27.763600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114463+0000","last_change":"2024-04-16T05:32:33.746371+0000","last_active":"2024-04-16T05:34:56.114463+0000","last_peered":"2024-04-16T05:34:56.114463+0000","last_clean":"2024-04-16T05:34:56.114463+0000","last_became_active":"2024-04-16T05:32:33.746077+0000","last_became_peered":"2024-04-16T05:32:33.746077+0000","last_unstale":"2024-04-16T05:34:56.114463+0000","last_undegraded":"2024-04-16T05:34:56.114463+0000","last_fullsized":"2024-04-16T05:34:56.114463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:57:43.193942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059205+0000","last_change":"2024-04-16T05:32:34.753196+0000","last_active":"2024-04-16T05:34:56.059205+0000","last_peered":"2024-04-16T05:34:56.059205+0000","last_clean":"2024-04-16T05:34:56.059205+0000","last_became_active":"2024-04-16T05:32:34.752881+0000","last_became_peered":"2024-04-16T05:32:34.752881+0000","last_unstale":"2024-04-16T05:34:56.059205+0000","last_undegraded":"2024-04-16T05:34:56.059205+0000","last_fullsized":"2024-04-16T05:34:56.059205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:35:56.715394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008805+0000","last_change":"2024-04-16T05:33:34.772562+0000","last_active":"2024-04-16T05:34:56.008805+0000","last_peered":"2024-04-16T05:34:56.008805+0000","last_clean":"2024-04-16T05:34:56.008805+0000","last_became_active":"2024-04-16T05:33:34.772052+0000","last_became_peered":"2024-04-16T05:33:34.772052+0000","last_unstale":"2024-04-16T05:34:56.008805+0000","last_undegraded":"2024-04-16T05:34:56.008805+0000","last_fullsized":"2024-04-16T05:34:56.008805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:50:18.425870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.873651+0000","last_change":"2024-04-16T05:32:33.735921+0000","last_active":"2024-04-16T05:34:56.873651+0000","last_peered":"2024-04-16T05:34:56.873651+0000","last_clean":"2024-04-16T05:34:56.873651+0000","last_became_active":"2024-04-16T05:32:33.735572+0000","last_became_peered":"2024-04-16T05:32:33.735572+0000","last_unstale":"2024-04-16T05:34:56.873651+0000","last_undegraded":"2024-04-16T05:34:56.873651+0000","last_fullsized":"2024-04-16T05:34:56.873651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:46:56.512827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058872+0000","last_change":"2024-04-16T05:32:34.762172+0000","last_active":"2024-04-16T05:34:56.058872+0000","last_peered":"2024-04-16T05:34:56.058872+0000","last_clean":"2024-04-16T05:34:56.058872+0000","last_became_active":"2024-04-16T05:32:34.761914+0000","last_became_peered":"2024-04-16T05:32:34.761914+0000","last_unstale":"2024-04-16T05:34:56.058872+0000","last_undegraded":"2024-04-16T05:34:56.058872+0000","last_fullsized":"2024-04-16T05:34:56.058872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:47:52.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267525+0000","last_change":"2024-04-16T05:33:34.772546+0000","last_active":"2024-04-16T05:34:56.267525+0000","last_peered":"2024-04-16T05:34:56.267525+0000","last_clean":"2024-04-16T05:34:56.267525+0000","last_became_active":"2024-04-16T05:33:34.772100+0000","last_became_peered":"2024-04-16T05:33:34.772100+0000","last_unstale":"2024-04-16T05:34:56.267525+0000","last_undegraded":"2024-04-16T05:34:56.267525+0000","last_fullsized":"2024-04-16T05:34:56.267525+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:11:50.157129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008514+0000","last_change":"2024-04-16T05:32:33.743944+0000","last_active":"2024-04-16T05:34:56.008514+0000","last_peered":"2024-04-16T05:34:56.008514+0000","last_clean":"2024-04-16T05:34:56.008514+0000","last_became_active":"2024-04-16T05:32:33.743559+0000","last_became_peered":"2024-04-16T05:32:33.743559+0000","last_unstale":"2024-04-16T05:34:56.008514+0000","last_undegraded":"2024-04-16T05:34:56.008514+0000","last_fullsized":"2024-04-16T05:34:56.008514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:08:48.889356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114332+0000","last_change":"2024-04-16T05:32:34.756741+0000","last_active":"2024-04-16T05:34:56.114332+0000","last_peered":"2024-04-16T05:34:56.114332+0000","last_clean":"2024-04-16T05:34:56.114332+0000","last_became_active":"2024-04-16T05:32:34.756426+0000","last_became_peered":"2024-04-16T05:32:34.756426+0000","last_unstale":"2024-04-16T05:34:56.114332+0000","last_undegraded":"2024-04-16T05:34:56.114332+0000","last_fullsized":"2024-04-16T05:34:56.114332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:09:30.493857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114287+0000","last_change":"2024-04-16T05:32:38.934818+0000","last_active":"2024-04-16T05:34:56.114287+0000","last_peered":"2024-04-16T05:34:56.114287+0000","last_clean":"2024-04-16T05:34:56.114287+0000","last_became_active":"2024-04-16T05:32:38.934335+0000","last_became_peered":"2024-04-16T05:32:38.934335+0000","last_unstale":"2024-04-16T05:34:56.114287+0000","last_undegraded":"2024-04-16T05:34:56.114287+0000","last_fullsized":"2024-04-16T05:34:56.114287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:02:41.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009144+0000","last_change":"2024-04-16T05:32:33.731951+0000","last_active":"2024-04-16T05:34:56.009144+0000","last_peered":"2024-04-16T05:34:56.009144+0000","last_clean":"2024-04-16T05:34:56.009144+0000","last_became_active":"2024-04-16T05:32:33.731649+0000","last_became_peered":"2024-04-16T05:32:33.731649+0000","last_unstale":"2024-04-16T05:34:56.009144+0000","last_undegraded":"2024-04-16T05:34:56.009144+0000","last_fullsized":"2024-04-16T05:34:56.009144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:41.258012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639876+0000","last_change":"2024-04-16T05:32:34.751884+0000","last_active":"2024-04-16T05:34:56.639876+0000","last_peered":"2024-04-16T05:34:56.639876+0000","last_clean":"2024-04-16T05:34:56.639876+0000","last_became_active":"2024-04-16T05:32:34.751559+0000","last_became_peered":"2024-04-16T05:32:34.751559+0000","last_unstale":"2024-04-16T05:34:56.639876+0000","last_undegraded":"2024-04-16T05:34:56.639876+0000","last_fullsized":"2024-04-16T05:34:56.639876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:32:59.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008772+0000","last_change":"2024-04-16T05:32:39.109330+0000","last_active":"2024-04-16T05:34:56.008772+0000","last_peered":"2024-04-16T05:34:56.008772+0000","last_clean":"2024-04-16T05:34:56.008772+0000","last_became_active":"2024-04-16T05:32:39.108718+0000","last_became_peered":"2024-04-16T05:32:39.108718+0000","last_unstale":"2024-04-16T05:34:56.008772+0000","last_undegraded":"2024-04-16T05:34:56.008772+0000","last_fullsized":"2024-04-16T05:34:56.008772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:36:55.418034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007436+0000","last_change":"2024-04-16T05:32:33.741603+0000","last_active":"2024-04-16T05:34:56.007436+0000","last_peered":"2024-04-16T05:34:56.007436+0000","last_clean":"2024-04-16T05:34:56.007436+0000","last_became_active":"2024-04-16T05:32:33.741184+0000","last_became_peered":"2024-04-16T05:32:33.741184+0000","last_unstale":"2024-04-16T05:34:56.007436+0000","last_undegraded":"2024-04-16T05:34:56.007436+0000","last_fullsized":"2024-04-16T05:34:56.007436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:08:38.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117997+0000","last_change":"2024-04-16T05:32:34.747768+0000","last_active":"2024-04-16T05:34:56.117997+0000","last_peered":"2024-04-16T05:34:56.117997+0000","last_clean":"2024-04-16T05:34:56.117997+0000","last_became_active":"2024-04-16T05:32:34.747440+0000","last_became_peered":"2024-04-16T05:32:34.747440+0000","last_unstale":"2024-04-16T05:34:56.117997+0000","last_undegraded":"2024-04-16T05:34:56.117997+0000","last_fullsized":"2024-04-16T05:34:56.117997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:14:31.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266840+0000","last_change":"2024-04-16T05:32:38.938673+0000","last_active":"2024-04-16T05:34:56.266840+0000","last_peered":"2024-04-16T05:34:56.266840+0000","last_clean":"2024-04-16T05:34:56.266840+0000","last_became_active":"2024-04-16T05:32:38.938252+0000","last_became_peered":"2024-04-16T05:32:38.938252+0000","last_unstale":"2024-04-16T05:34:56.266840+0000","last_undegraded":"2024-04-16T05:34:56.266840+0000","last_fullsized":"2024-04-16T05:34:56.266840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:06.620693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058730+0000","last_change":"2024-04-16T05:32:33.750812+0000","last_active":"2024-04-16T05:34:56.058730+0000","last_peered":"2024-04-16T05:34:56.058730+0000","last_clean":"2024-04-16T05:34:56.058730+0000","last_became_active":"2024-04-16T05:32:33.750514+0000","last_became_peered":"2024-04-16T05:32:33.750514+0000","last_unstale":"2024-04-16T05:34:56.058730+0000","last_undegraded":"2024-04-16T05:34:56.058730+0000","last_fullsized":"2024-04-16T05:34:56.058730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:53:43.075914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117672+0000","last_change":"2024-04-16T05:33:34.770449+0000","last_active":"2024-04-16T05:34:56.117672+0000","last_peered":"2024-04-16T05:34:56.117672+0000","last_clean":"2024-04-16T05:34:56.117672+0000","last_became_active":"2024-04-16T05:33:34.770078+0000","last_became_peered":"2024-04-16T05:33:34.770078+0000","last_unstale":"2024-04-16T05:34:56.117672+0000","last_undegraded":"2024-04-16T05:34:56.117672+0000","last_fullsized":"2024-04-16T05:34:56.117672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:30:20.885002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267319+0000","last_change":"2024-04-16T05:32:39.109394+0000","last_active":"2024-04-16T05:34:56.267319+0000","last_peered":"2024-04-16T05:34:56.267319+0000","last_clean":"2024-04-16T05:34:56.267319+0000","last_became_active":"2024-04-16T05:32:39.108924+0000","last_became_peered":"2024-04-16T05:32:39.108924+0000","last_unstale":"2024-04-16T05:34:56.267319+0000","last_undegraded":"2024-04-16T05:34:56.267319+0000","last_fullsized":"2024-04-16T05:34:56.267319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:50:37.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007055+0000","last_change":"2024-04-16T05:33:34.766071+0000","last_active":"2024-04-16T05:34:56.007055+0000","last_peered":"2024-04-16T05:34:56.007055+0000","last_clean":"2024-04-16T05:34:56.007055+0000","last_became_active":"2024-04-16T05:33:34.765670+0000","last_became_peered":"2024-04-16T05:33:34.765670+0000","last_unstale":"2024-04-16T05:34:56.007055+0000","last_undegraded":"2024-04-16T05:34:56.007055+0000","last_fullsized":"2024-04-16T05:34:56.007055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:54:47.320828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267990+0000","last_change":"2024-04-16T05:32:33.748521+0000","last_active":"2024-04-16T05:34:56.267990+0000","last_peered":"2024-04-16T05:34:56.267990+0000","last_clean":"2024-04-16T05:34:56.267990+0000","last_became_active":"2024-04-16T05:32:33.748222+0000","last_became_peered":"2024-04-16T05:32:33.748222+0000","last_unstale":"2024-04-16T05:34:56.267990+0000","last_undegraded":"2024-04-16T05:34:56.267990+0000","last_fullsized":"2024-04-16T05:34:56.267990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:04.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058859+0000","last_change":"2024-04-16T05:32:38.935995+0000","last_active":"2024-04-16T05:34:56.058859+0000","last_peered":"2024-04-16T05:34:56.058859+0000","last_clean":"2024-04-16T05:34:56.058859+0000","last_became_active":"2024-04-16T05:32:38.935511+0000","last_became_peered":"2024-04-16T05:32:38.935511+0000","last_unstale":"2024-04-16T05:34:56.058859+0000","last_undegraded":"2024-04-16T05:34:56.058859+0000","last_fullsized":"2024-04-16T05:34:56.058859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:17:08.144537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105847+0000","last_change":"2024-04-16T05:32:34.760249+0000","last_active":"2024-04-16T05:34:56.105847+0000","last_peered":"2024-04-16T05:34:56.105847+0000","last_clean":"2024-04-16T05:34:56.105847+0000","last_became_active":"2024-04-16T05:32:34.759735+0000","last_became_peered":"2024-04-16T05:32:34.759735+0000","last_unstale":"2024-04-16T05:34:56.105847+0000","last_undegraded":"2024-04-16T05:34:56.105847+0000","last_fullsized":"2024-04-16T05:34:56.105847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:13.349558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009119+0000","last_change":"2024-04-16T05:32:33.731170+0000","last_active":"2024-04-16T05:34:56.009119+0000","last_peered":"2024-04-16T05:34:56.009119+0000","last_clean":"2024-04-16T05:34:56.009119+0000","last_became_active":"2024-04-16T05:32:33.730821+0000","last_became_peered":"2024-04-16T05:32:33.730821+0000","last_unstale":"2024-04-16T05:34:56.009119+0000","last_undegraded":"2024-04-16T05:34:56.009119+0000","last_fullsized":"2024-04-16T05:34:56.009119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:38:15.707590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":365,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058562+0000","last_change":"2024-04-16T05:31:24.926319+0000","last_active":"2024-04-16T05:34:56.058562+0000","last_peered":"2024-04-16T05:34:56.058562+0000","last_clean":"2024-04-16T05:34:56.058562+0000","last_became_active":"2024-04-16T05:31:24.925936+0000","last_became_peered":"2024-04-16T05:31:24.925936+0000","last_unstale":"2024-04-16T05:34:56.058562+0000","last_undegraded":"2024-04-16T05:34:56.058562+0000","last_fullsized":"2024-04-16T05:34:56.058562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_clean_scrub_stamp":"2024-04-16T05:27:57.876673+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:28:19.212127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114596+0000","last_change":"2024-04-16T05:32:39.110241+0000","last_active":"2024-04-16T05:34:56.114596+0000","last_peered":"2024-04-16T05:34:56.114596+0000","last_clean":"2024-04-16T05:34:56.114596+0000","last_became_active":"2024-04-16T05:32:39.109792+0000","last_became_peered":"2024-04-16T05:32:39.109792+0000","last_unstale":"2024-04-16T05:34:56.114596+0000","last_undegraded":"2024-04-16T05:34:56.114596+0000","last_fullsized":"2024-04-16T05:34:56.114596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:15:05.925370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264402+0000","last_change":"2024-04-16T05:32:34.751693+0000","last_active":"2024-04-16T05:34:56.264402+0000","last_peered":"2024-04-16T05:34:56.264402+0000","last_clean":"2024-04-16T05:34:56.264402+0000","last_became_active":"2024-04-16T05:32:34.751210+0000","last_became_peered":"2024-04-16T05:32:34.751210+0000","last_unstale":"2024-04-16T05:34:56.264402+0000","last_undegraded":"2024-04-16T05:34:56.264402+0000","last_fullsized":"2024-04-16T05:34:56.264402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:33:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058618+0000","last_change":"2024-04-16T05:32:33.738310+0000","last_active":"2024-04-16T05:34:56.058618+0000","last_peered":"2024-04-16T05:34:56.058618+0000","last_clean":"2024-04-16T05:34:56.058618+0000","last_became_active":"2024-04-16T05:32:33.738012+0000","last_became_peered":"2024-04-16T05:32:33.738012+0000","last_unstale":"2024-04-16T05:34:56.058618+0000","last_undegraded":"2024-04-16T05:34:56.058618+0000","last_fullsized":"2024-04-16T05:34:56.058618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:22:18.311458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639634+0000","last_change":"2024-04-16T05:32:38.933465+0000","last_active":"2024-04-16T05:34:56.639634+0000","last_peered":"2024-04-16T05:34:56.639634+0000","last_clean":"2024-04-16T05:34:56.639634+0000","last_became_active":"2024-04-16T05:32:38.932391+0000","last_became_peered":"2024-04-16T05:32:38.932391+0000","last_unstale":"2024-04-16T05:34:56.639634+0000","last_undegraded":"2024-04-16T05:34:56.639634+0000","last_fullsized":"2024-04-16T05:34:56.639634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:39.494556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009019+0000","last_change":"2024-04-16T05:32:33.743241+0000","last_active":"2024-04-16T05:34:56.009019+0000","last_peered":"2024-04-16T05:34:56.009019+0000","last_clean":"2024-04-16T05:34:56.009019+0000","last_became_active":"2024-04-16T05:32:33.742887+0000","last_became_peered":"2024-04-16T05:32:33.742887+0000","last_unstale":"2024-04-16T05:34:56.009019+0000","last_undegraded":"2024-04-16T05:34:56.009019+0000","last_fullsized":"2024-04-16T05:34:56.009019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:24:38.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267151+0000","last_change":"2024-04-16T05:32:34.746228+0000","last_active":"2024-04-16T05:34:56.267151+0000","last_peered":"2024-04-16T05:34:56.267151+0000","last_clean":"2024-04-16T05:34:56.267151+0000","last_became_active":"2024-04-16T05:32:34.745883+0000","last_became_peered":"2024-04-16T05:32:34.745883+0000","last_unstale":"2024-04-16T05:34:56.267151+0000","last_undegraded":"2024-04-16T05:34:56.267151+0000","last_fullsized":"2024-04-16T05:34:56.267151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:33.656358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105107+0000","last_change":"2024-04-16T05:32:39.110156+0000","last_active":"2024-04-16T05:34:56.105107+0000","last_peered":"2024-04-16T05:34:56.105107+0000","last_clean":"2024-04-16T05:34:56.105107+0000","last_became_active":"2024-04-16T05:32:39.108958+0000","last_became_peered":"2024-04-16T05:32:39.108958+0000","last_unstale":"2024-04-16T05:34:56.105107+0000","last_undegraded":"2024-04-16T05:34:56.105107+0000","last_fullsized":"2024-04-16T05:34:56.105107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:27:58.658187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'45","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.229853+0000","last_change":"2024-04-16T05:32:33.738179+0000","last_active":"2024-04-16T05:34:56.229853+0000","last_peered":"2024-04-16T05:34:56.229853+0000","last_clean":"2024-04-16T05:34:56.229853+0000","last_became_active":"2024-04-16T05:32:33.737841+0000","last_became_peered":"2024-04-16T05:32:33.737841+0000","last_unstale":"2024-04-16T05:34:56.229853+0000","last_undegraded":"2024-04-16T05:34:56.229853+0000","last_fullsized":"2024-04-16T05:34:56.229853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:13:11.210559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107138+0000","last_change":"2024-04-16T05:32:34.758142+0000","last_active":"2024-04-16T05:34:56.107138+0000","last_peered":"2024-04-16T05:34:56.107138+0000","last_clean":"2024-04-16T05:34:56.107138+0000","last_became_active":"2024-04-16T05:32:34.757825+0000","last_became_peered":"2024-04-16T05:32:34.757825+0000","last_unstale":"2024-04-16T05:34:56.107138+0000","last_undegraded":"2024-04-16T05:34:56.107138+0000","last_fullsized":"2024-04-16T05:34:56.107138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:37.575764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.263978+0000","last_change":"2024-04-16T05:32:38.933327+0000","last_active":"2024-04-16T05:34:56.263978+0000","last_peered":"2024-04-16T05:34:56.263978+0000","last_clean":"2024-04-16T05:34:56.263978+0000","last_became_active":"2024-04-16T05:32:38.932808+0000","last_became_peered":"2024-04-16T05:32:38.932808+0000","last_unstale":"2024-04-16T05:34:56.263978+0000","last_undegraded":"2024-04-16T05:34:56.263978+0000","last_fullsized":"2024-04-16T05:34:56.263978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:03.217103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009338+0000","last_change":"2024-04-16T05:32:33.742405+0000","last_active":"2024-04-16T05:34:56.009338+0000","last_peered":"2024-04-16T05:34:56.009338+0000","last_clean":"2024-04-16T05:34:56.009338+0000","last_became_active":"2024-04-16T05:32:33.742051+0000","last_became_peered":"2024-04-16T05:32:33.742051+0000","last_unstale":"2024-04-16T05:34:56.009338+0000","last_undegraded":"2024-04-16T05:34:56.009338+0000","last_fullsized":"2024-04-16T05:34:56.009338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:35:13.914483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267051+0000","last_change":"2024-04-16T05:33:34.772430+0000","last_active":"2024-04-16T05:34:56.267051+0000","last_peered":"2024-04-16T05:34:56.267051+0000","last_clean":"2024-04-16T05:34:56.267051+0000","last_became_active":"2024-04-16T05:33:34.772067+0000","last_became_peered":"2024-04-16T05:33:34.772067+0000","last_unstale":"2024-04-16T05:34:56.267051+0000","last_undegraded":"2024-04-16T05:34:56.267051+0000","last_fullsized":"2024-04-16T05:34:56.267051+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:24.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009151+0000","last_change":"2024-04-16T05:33:34.770714+0000","last_active":"2024-04-16T05:34:56.009151+0000","last_peered":"2024-04-16T05:34:56.009151+0000","last_clean":"2024-04-16T05:34:56.009151+0000","last_became_active":"2024-04-16T05:33:34.770249+0000","last_became_peered":"2024-04-16T05:33:34.770249+0000","last_unstale":"2024-04-16T05:34:56.009151+0000","last_undegraded":"2024-04-16T05:34:56.009151+0000","last_fullsized":"2024-04-16T05:34:56.009151+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:46.517492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264301+0000","last_change":"2024-04-16T05:32:33.743031+0000","last_active":"2024-04-16T05:34:56.264301+0000","last_peered":"2024-04-16T05:34:56.264301+0000","last_clean":"2024-04-16T05:34:56.264301+0000","last_became_active":"2024-04-16T05:32:33.742771+0000","last_became_peered":"2024-04-16T05:32:33.742771+0000","last_unstale":"2024-04-16T05:34:56.264301+0000","last_undegraded":"2024-04-16T05:34:56.264301+0000","last_fullsized":"2024-04-16T05:34:56.264301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:06.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008461+0000","last_change":"2024-04-16T05:32:34.749428+0000","last_active":"2024-04-16T05:34:56.008461+0000","last_peered":"2024-04-16T05:34:56.008461+0000","last_clean":"2024-04-16T05:34:56.008461+0000","last_became_active":"2024-04-16T05:32:34.749087+0000","last_became_peered":"2024-04-16T05:32:34.749087+0000","last_unstale":"2024-04-16T05:34:56.008461+0000","last_undegraded":"2024-04-16T05:34:56.008461+0000","last_fullsized":"2024-04-16T05:34:56.008461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:59:25.124558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007570+0000","last_change":"2024-04-16T05:32:38.942407+0000","last_active":"2024-04-16T05:34:56.007570+0000","last_peered":"2024-04-16T05:34:56.007570+0000","last_clean":"2024-04-16T05:34:56.007570+0000","last_became_active":"2024-04-16T05:32:38.941731+0000","last_became_peered":"2024-04-16T05:32:38.941731+0000","last_unstale":"2024-04-16T05:34:56.007570+0000","last_undegraded":"2024-04-16T05:34:56.007570+0000","last_fullsized":"2024-04-16T05:34:56.007570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:42:27.700203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105694+0000","last_change":"2024-04-16T05:32:33.741487+0000","last_active":"2024-04-16T05:34:56.105694+0000","last_peered":"2024-04-16T05:34:56.105694+0000","last_clean":"2024-04-16T05:34:56.105694+0000","last_became_active":"2024-04-16T05:32:33.741140+0000","last_became_peered":"2024-04-16T05:32:33.741140+0000","last_unstale":"2024-04-16T05:34:56.105694+0000","last_undegraded":"2024-04-16T05:34:56.105694+0000","last_fullsized":"2024-04-16T05:34:56.105694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:21:36.086164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639473+0000","last_change":"2024-04-16T05:32:34.756608+0000","last_active":"2024-04-16T05:34:56.639473+0000","last_peered":"2024-04-16T05:34:56.639473+0000","last_clean":"2024-04-16T05:34:56.639473+0000","last_became_active":"2024-04-16T05:32:34.756296+0000","last_became_peered":"2024-04-16T05:32:34.756296+0000","last_unstale":"2024-04-16T05:34:56.639473+0000","last_undegraded":"2024-04-16T05:34:56.639473+0000","last_fullsized":"2024-04-16T05:34:56.639473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:28.397333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009566+0000","last_change":"2024-04-16T05:32:38.929876+0000","last_active":"2024-04-16T05:34:56.009566+0000","last_peered":"2024-04-16T05:34:56.009566+0000","last_clean":"2024-04-16T05:34:56.009566+0000","last_became_active":"2024-04-16T05:32:38.928626+0000","last_became_peered":"2024-04-16T05:32:38.928626+0000","last_unstale":"2024-04-16T05:34:56.009566+0000","last_undegraded":"2024-04-16T05:34:56.009566+0000","last_fullsized":"2024-04-16T05:34:56.009566+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:08:09.697413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264072+0000","last_change":"2024-04-16T05:32:33.736047+0000","last_active":"2024-04-16T05:34:56.264072+0000","last_peered":"2024-04-16T05:34:56.264072+0000","last_clean":"2024-04-16T05:34:56.264072+0000","last_became_active":"2024-04-16T05:32:33.735683+0000","last_became_peered":"2024-04-16T05:32:33.735683+0000","last_unstale":"2024-04-16T05:34:56.264072+0000","last_undegraded":"2024-04-16T05:34:56.264072+0000","last_fullsized":"2024-04-16T05:34:56.264072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:58:32.969924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.106396+0000","last_change":"2024-04-16T05:32:34.763558+0000","last_active":"2024-04-16T05:34:56.106396+0000","last_peered":"2024-04-16T05:34:56.106396+0000","last_clean":"2024-04-16T05:34:56.106396+0000","last_became_active":"2024-04-16T05:32:34.763190+0000","last_became_peered":"2024-04-16T05:32:34.763190+0000","last_unstale":"2024-04-16T05:34:56.106396+0000","last_undegraded":"2024-04-16T05:34:56.106396+0000","last_fullsized":"2024-04-16T05:34:56.106396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:38:31.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009179+0000","last_change":"2024-04-16T05:32:39.109426+0000","last_active":"2024-04-16T05:34:56.009179+0000","last_peered":"2024-04-16T05:34:56.009179+0000","last_clean":"2024-04-16T05:34:56.009179+0000","last_became_active":"2024-04-16T05:32:39.108970+0000","last_became_peered":"2024-04-16T05:32:39.108970+0000","last_unstale":"2024-04-16T05:34:56.009179+0000","last_undegraded":"2024-04-16T05:34:56.009179+0000","last_fullsized":"2024-04-16T05:34:56.009179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:54.824654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008593+0000","last_change":"2024-04-16T05:32:33.734983+0000","last_active":"2024-04-16T05:34:56.008593+0000","last_peered":"2024-04-16T05:34:56.008593+0000","last_clean":"2024-04-16T05:34:56.008593+0000","last_became_active":"2024-04-16T05:32:33.734609+0000","last_became_peered":"2024-04-16T05:32:33.734609+0000","last_unstale":"2024-04-16T05:34:56.008593+0000","last_undegraded":"2024-04-16T05:34:56.008593+0000","last_fullsized":"2024-04-16T05:34:56.008593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:35:59.436499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058954+0000","last_change":"2024-04-16T05:32:34.759811+0000","last_active":"2024-04-16T05:34:56.058954+0000","last_peered":"2024-04-16T05:34:56.058954+0000","last_clean":"2024-04-16T05:34:56.058954+0000","last_became_active":"2024-04-16T05:32:34.759499+0000","last_became_peered":"2024-04-16T05:32:34.759499+0000","last_unstale":"2024-04-16T05:34:56.058954+0000","last_undegraded":"2024-04-16T05:34:56.058954+0000","last_fullsized":"2024-04-16T05:34:56.058954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:46:28.554191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264537+0000","last_change":"2024-04-16T05:32:38.929021+0000","last_active":"2024-04-16T05:34:56.264537+0000","last_peered":"2024-04-16T05:34:56.264537+0000","last_clean":"2024-04-16T05:34:56.264537+0000","last_became_active":"2024-04-16T05:32:38.928574+0000","last_became_peered":"2024-04-16T05:32:38.928574+0000","last_unstale":"2024-04-16T05:34:56.264537+0000","last_undegraded":"2024-04-16T05:34:56.264537+0000","last_fullsized":"2024-04-16T05:34:56.264537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:11:35.693364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264578+0000","last_change":"2024-04-16T05:32:33.747352+0000","last_active":"2024-04-16T05:34:56.264578+0000","last_peered":"2024-04-16T05:34:56.264578+0000","last_clean":"2024-04-16T05:34:56.264578+0000","last_became_active":"2024-04-16T05:32:33.746989+0000","last_became_peered":"2024-04-16T05:32:33.746989+0000","last_unstale":"2024-04-16T05:34:56.264578+0000","last_undegraded":"2024-04-16T05:34:56.264578+0000","last_fullsized":"2024-04-16T05:34:56.264578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:38.422274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059157+0000","last_change":"2024-04-16T05:32:34.760636+0000","last_active":"2024-04-16T05:34:56.059157+0000","last_peered":"2024-04-16T05:34:56.059157+0000","last_clean":"2024-04-16T05:34:56.059157+0000","last_became_active":"2024-04-16T05:32:34.760335+0000","last_became_peered":"2024-04-16T05:32:34.760335+0000","last_unstale":"2024-04-16T05:34:56.059157+0000","last_undegraded":"2024-04-16T05:34:56.059157+0000","last_fullsized":"2024-04-16T05:34:56.059157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:04:34.322799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266617+0000","last_change":"2024-04-16T05:33:34.769486+0000","last_active":"2024-04-16T05:34:56.266617+0000","last_peered":"2024-04-16T05:34:56.266617+0000","last_clean":"2024-04-16T05:34:56.266617+0000","last_became_active":"2024-04-16T05:33:34.769025+0000","last_became_peered":"2024-04-16T05:33:34.769025+0000","last_unstale":"2024-04-16T05:34:56.266617+0000","last_undegraded":"2024-04-16T05:34:56.266617+0000","last_fullsized":"2024-04-16T05:34:56.266617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:10.382817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'39","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.228124+0000","last_change":"2024-04-16T05:32:33.746346+0000","last_active":"2024-04-16T05:34:56.228124+0000","last_peered":"2024-04-16T05:34:56.228124+0000","last_clean":"2024-04-16T05:34:56.228124+0000","last_became_active":"2024-04-16T05:32:33.746030+0000","last_became_peered":"2024-04-16T05:32:33.746030+0000","last_unstale":"2024-04-16T05:34:56.228124+0000","last_undegraded":"2024-04-16T05:34:56.228124+0000","last_fullsized":"2024-04-16T05:34:56.228124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:54:35.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058863+0000","last_change":"2024-04-16T05:32:34.752283+0000","last_active":"2024-04-16T05:34:56.058863+0000","last_peered":"2024-04-16T05:34:56.058863+0000","last_clean":"2024-04-16T05:34:56.058863+0000","last_became_active":"2024-04-16T05:32:34.751904+0000","last_became_peered":"2024-04-16T05:32:34.751904+0000","last_unstale":"2024-04-16T05:34:56.058863+0000","last_undegraded":"2024-04-16T05:34:56.058863+0000","last_fullsized":"2024-04-16T05:34:56.058863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:15:52.688683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114065+0000","last_change":"2024-04-16T05:32:38.934194+0000","last_active":"2024-04-16T05:34:56.114065+0000","last_peered":"2024-04-16T05:34:56.114065+0000","last_clean":"2024-04-16T05:34:56.114065+0000","last_became_active":"2024-04-16T05:32:38.933636+0000","last_became_peered":"2024-04-16T05:32:38.933636+0000","last_unstale":"2024-04-16T05:34:56.114065+0000","last_undegraded":"2024-04-16T05:34:56.114065+0000","last_fullsized":"2024-04-16T05:34:56.114065+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:14:40.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267445+0000","last_change":"2024-04-16T05:32:33.744267+0000","last_active":"2024-04-16T05:34:56.267445+0000","last_peered":"2024-04-16T05:34:56.267445+0000","last_clean":"2024-04-16T05:34:56.267445+0000","last_became_active":"2024-04-16T05:32:33.743931+0000","last_became_peered":"2024-04-16T05:32:33.743931+0000","last_unstale":"2024-04-16T05:34:56.267445+0000","last_undegraded":"2024-04-16T05:34:56.267445+0000","last_fullsized":"2024-04-16T05:34:56.267445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:30.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059158+0000","last_change":"2024-04-16T05:32:34.756833+0000","last_active":"2024-04-16T05:34:56.059158+0000","last_peered":"2024-04-16T05:34:56.059158+0000","last_clean":"2024-04-16T05:34:56.059158+0000","last_became_active":"2024-04-16T05:32:34.756436+0000","last_became_peered":"2024-04-16T05:32:34.756436+0000","last_unstale":"2024-04-16T05:34:56.059158+0000","last_undegraded":"2024-04-16T05:34:56.059158+0000","last_fullsized":"2024-04-16T05:34:56.059158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:20.732946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639538+0000","last_change":"2024-04-16T05:32:34.752744+0000","last_active":"2024-04-16T05:34:56.639538+0000","last_peered":"2024-04-16T05:34:56.639538+0000","last_clean":"2024-04-16T05:34:56.639538+0000","last_became_active":"2024-04-16T05:32:34.752442+0000","last_became_peered":"2024-04-16T05:32:34.752442+0000","last_unstale":"2024-04-16T05:34:56.639538+0000","last_undegraded":"2024-04-16T05:34:56.639538+0000","last_fullsized":"2024-04-16T05:34:56.639538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:01:23.644692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264130+0000","last_change":"2024-04-16T05:32:34.755118+0000","last_active":"2024-04-16T05:34:56.264130+0000","last_peered":"2024-04-16T05:34:56.264130+0000","last_clean":"2024-04-16T05:34:56.264130+0000","last_became_active":"2024-04-16T05:32:34.754824+0000","last_became_peered":"2024-04-16T05:32:34.754824+0000","last_unstale":"2024-04-16T05:34:56.264130+0000","last_undegraded":"2024-04-16T05:34:56.264130+0000","last_fullsized":"2024-04-16T05:34:56.264130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:58:37.860305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264047+0000","last_change":"2024-04-16T05:32:34.756144+0000","last_active":"2024-04-16T05:34:56.264047+0000","last_peered":"2024-04-16T05:34:56.264047+0000","last_clean":"2024-04-16T05:34:56.264047+0000","last_became_active":"2024-04-16T05:32:34.755841+0000","last_became_peered":"2024-04-16T05:32:34.755841+0000","last_unstale":"2024-04-16T05:34:56.264047+0000","last_undegraded":"2024-04-16T05:34:56.264047+0000","last_fullsized":"2024-04-16T05:34:56.264047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:23:00.482444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266708+0000","last_change":"2024-04-16T05:32:34.766204+0000","last_active":"2024-04-16T05:34:56.266708+0000","last_peered":"2024-04-16T05:34:56.266708+0000","last_clean":"2024-04-16T05:34:56.266708+0000","last_became_active":"2024-04-16T05:32:34.765709+0000","last_became_peered":"2024-04-16T05:32:34.765709+0000","last_unstale":"2024-04-16T05:34:56.266708+0000","last_undegraded":"2024-04-16T05:34:56.266708+0000","last_fullsized":"2024-04-16T05:34:56.266708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:23:53.926477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267022+0000","last_change":"2024-04-16T05:32:34.758513+0000","last_active":"2024-04-16T05:34:56.267022+0000","last_peered":"2024-04-16T05:34:56.267022+0000","last_clean":"2024-04-16T05:34:56.267022+0000","last_became_active":"2024-04-16T05:32:34.758145+0000","last_became_peered":"2024-04-16T05:32:34.758145+0000","last_unstale":"2024-04-16T05:34:56.267022+0000","last_undegraded":"2024-04-16T05:34:56.267022+0000","last_fullsized":"2024-04-16T05:34:56.267022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:09:17.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639842+0000","last_change":"2024-04-16T05:32:34.750651+0000","last_active":"2024-04-16T05:34:56.639842+0000","last_peered":"2024-04-16T05:34:56.639842+0000","last_clean":"2024-04-16T05:34:56.639842+0000","last_became_active":"2024-04-16T05:32:34.750348+0000","last_became_peered":"2024-04-16T05:32:34.750348+0000","last_unstale":"2024-04-16T05:34:56.639842+0000","last_undegraded":"2024-04-16T05:34:56.639842+0000","last_fullsized":"2024-04-16T05:34:56.639842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:29.282449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264574+0000","last_change":"2024-04-16T05:32:34.750079+0000","last_active":"2024-04-16T05:34:56.264574+0000","last_peered":"2024-04-16T05:34:56.264574+0000","last_clean":"2024-04-16T05:34:56.264574+0000","last_became_active":"2024-04-16T05:32:34.749775+0000","last_became_peered":"2024-04-16T05:32:34.749775+0000","last_unstale":"2024-04-16T05:34:56.264574+0000","last_undegraded":"2024-04-16T05:34:56.264574+0000","last_fullsized":"2024-04-16T05:34:56.264574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:02.297258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107010+0000","last_change":"2024-04-16T05:32:34.752528+0000","last_active":"2024-04-16T05:34:56.107010+0000","last_peered":"2024-04-16T05:34:56.107010+0000","last_clean":"2024-04-16T05:34:56.107010+0000","last_became_active":"2024-04-16T05:32:34.752150+0000","last_became_peered":"2024-04-16T05:32:34.752150+0000","last_unstale":"2024-04-16T05:34:56.107010+0000","last_undegraded":"2024-04-16T05:34:56.107010+0000","last_fullsized":"2024-04-16T05:34:56.107010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:41:16.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114222+0000","last_change":"2024-04-16T05:32:34.758486+0000","last_active":"2024-04-16T05:34:56.114222+0000","last_peered":"2024-04-16T05:34:56.114222+0000","last_clean":"2024-04-16T05:34:56.114222+0000","last_became_active":"2024-04-16T05:32:34.754840+0000","last_became_peered":"2024-04-16T05:32:34.754840+0000","last_unstale":"2024-04-16T05:34:56.114222+0000","last_undegraded":"2024-04-16T05:34:56.114222+0000","last_fullsized":"2024-04-16T05:34:56.114222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:35.644337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105877+0000","last_change":"2024-04-16T05:32:34.758933+0000","last_active":"2024-04-16T05:34:56.105877+0000","last_peered":"2024-04-16T05:34:56.105877+0000","last_clean":"2024-04-16T05:34:56.105877+0000","last_became_active":"2024-04-16T05:32:34.758587+0000","last_became_peered":"2024-04-16T05:32:34.758587+0000","last_unstale":"2024-04-16T05:34:56.105877+0000","last_undegraded":"2024-04-16T05:34:56.105877+0000","last_fullsized":"2024-04-16T05:34:56.105877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:19:20.674248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264656+0000","last_change":"2024-04-16T05:32:34.750846+0000","last_active":"2024-04-16T05:34:56.264656+0000","last_peered":"2024-04-16T05:34:56.264656+0000","last_clean":"2024-04-16T05:34:56.264656+0000","last_became_active":"2024-04-16T05:32:34.750529+0000","last_became_peered":"2024-04-16T05:32:34.750529+0000","last_unstale":"2024-04-16T05:34:56.264656+0000","last_undegraded":"2024-04-16T05:34:56.264656+0000","last_fullsized":"2024-04-16T05:34:56.264656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:32.288459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266353+0000","last_change":"2024-04-16T05:32:34.752566+0000","last_active":"2024-04-16T05:34:56.266353+0000","last_peered":"2024-04-16T05:34:56.266353+0000","last_clean":"2024-04-16T05:34:56.266353+0000","last_became_active":"2024-04-16T05:32:34.752236+0000","last_became_peered":"2024-04-16T05:32:34.752236+0000","last_unstale":"2024-04-16T05:34:56.266353+0000","last_undegraded":"2024-04-16T05:34:56.266353+0000","last_fullsized":"2024-04-16T05:34:56.266353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:43:36.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639755+0000","last_change":"2024-04-16T05:32:34.750972+0000","last_active":"2024-04-16T05:34:56.639755+0000","last_peered":"2024-04-16T05:34:56.639755+0000","last_clean":"2024-04-16T05:34:56.639755+0000","last_became_active":"2024-04-16T05:32:34.750671+0000","last_became_peered":"2024-04-16T05:32:34.750671+0000","last_unstale":"2024-04-16T05:34:56.639755+0000","last_undegraded":"2024-04-16T05:34:56.639755+0000","last_fullsized":"2024-04-16T05:34:56.639755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:19:45.331816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108342,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597977,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":858,"num_read_kb":541,"num_write":289,"num_write_kb":1442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1793931,"data_compressed":80799,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":510665,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1298518,"data_compressed":67519,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":908179,"data_compressed":42764,"data_compressed_allocated":57344,"data_compressed_original":458752,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":904003,"data_compressed":42764,"data_compressed_allocated":57344,"data_compressed_original":458752,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":1107760,"data_compressed":59732,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":506043,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1073152,"data_stored":574049,"data_compressed":25249,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":507431,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":565698,"data_compressed":22271,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":637821,"data_compressed":30048,"data_compressed_allocated":40960,"data_compressed_original":196608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":511225,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1101824,"data_stored":1234025,"data_compressed":65888,"data_compressed_allocated":86016,"data_compressed_original":778240,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68096,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69944,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133608,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":202197,"data_compressed":7787,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:34:59.117 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-16T05:34:59.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T05:34:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 -- 172.21.15.32:0/1857717129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105e30 msgr2=0x7f53c010c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 --2- 172.21.15.32:0/1857717129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105e30 0x7f53c010c6c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009ca0 tx=0x7f53b802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 -- 172.21.15.32:0/1857717129 shutdown_connections 2024-04-16T05:34:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 --2- 172.21.15.32:0/1857717129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105e30 0x7f53c010c6c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 --2- 172.21.15.32:0/1857717129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c0105470 0x7f53c01058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 --2- 172.21.15.32:0/1857717129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c0104280 0x7f53c01046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 -- 172.21.15.32:0/1857717129 >> 172.21.15.32:0/1857717129 conn(0x7f53c00ff7f0 msgr2=0x7f53c0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 -- 172.21.15.32:0/1857717129 shutdown_connections 2024-04-16T05:34:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.260+0000 7f53c8070640 1 -- 172.21.15.32:0/1857717129 wait complete. 2024-04-16T05:34:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c8070640 1 Processor -- start 2024-04-16T05:34:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c8070640 1 -- start start 2024-04-16T05:34:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c8070640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c0104280 0x7f53c019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c8070640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105470 0x7f53c019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c8070640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c0105e30 0x7f53c01a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c8070640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53c010ff10 con 0x7f53c0105470 2024-04-16T05:34:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c8070640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f53c010fd90 con 0x7f53c0105e30 2024-04-16T05:34:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c8070640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53c0110090 con 0x7f53c0104280 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c560b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105470 0x7f53c019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c560b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105470 0x7f53c019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35766/0 (socket says 172.21.15.32:35766) 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c560b640 1 -- 172.21.15.32:0/631502929 learned_addr learned my addr 172.21.15.32:0/631502929 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c5e0c640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c0104280 0x7f53c019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.261+0000 7f53c660d640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c0105e30 0x7f53c01a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53c560b640 1 -- 172.21.15.32:0/631502929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c0104280 msgr2=0x7f53c019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53c560b640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c0104280 0x7f53c019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53c560b640 1 -- 172.21.15.32:0/631502929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c0105e30 msgr2=0x7f53c01a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53c560b640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c0105e30 0x7f53c01a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53c560b640 1 -- 172.21.15.32:0/631502929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c01a2160 con 0x7f53c0105470 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53c560b640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105470 0x7f53c019aa80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f53b000d950 tx=0x7f53b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53b6ffd640 1 -- 172.21.15.32:0/631502929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0017030 con 0x7f53c0105470 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c01a2450 con 0x7f53c0105470 2024-04-16T05:34:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53b6ffd640 1 -- 172.21.15.32:0/631502929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b0004020 con 0x7f53c0105470 2024-04-16T05:34:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53b6ffd640 1 -- 172.21.15.32:0/631502929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00047a0 con 0x7f53c0105470 2024-04-16T05:34:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.262+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c01a29a0 con 0x7f53c0105470 2024-04-16T05:34:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.263+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c010b5a0 con 0x7f53c0105470 2024-04-16T05:34:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.264+0000 7f53b6ffd640 1 -- 172.21.15.32:0/631502929 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f53b00041d0 con 0x7f53c0105470 2024-04-16T05:34:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.265+0000 7f53b6ffd640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53ac0a3d20 0x7f53ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.265+0000 7f53c5e0c640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53ac0a3d20 0x7f53ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.265+0000 7f53b6ffd640 1 -- 172.21.15.32:0/631502929 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f53b00d0490 con 0x7f53c0105470 2024-04-16T05:34:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.266+0000 7f53c5e0c640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53ac0a3d20 0x7f53ac0a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f53c01052d0 tx=0x7f53bc005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.270+0000 7f53b6ffd640 1 -- 172.21.15.32:0/631502929 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53b009c7f0 con 0x7f53c0105470 2024-04-16T05:34:59.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.461+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53c0062440 con 0x7f53ac0a3d20 2024-04-16T05:34:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.470+0000 7f53b6ffd640 1 -- 172.21.15.32:0/631502929 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339709 (secure 0 0 0) 0x7f53c0062440 con 0x7f53ac0a3d20 2024-04-16T05:34:59.472 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:59.474 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53ac0a3d20 msgr2=0x7f53ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53ac0a3d20 0x7f53ac0a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f53c01052d0 tx=0x7f53bc005f90 comp rx=0 tx=0).stop 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105470 msgr2=0x7f53c019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105470 0x7f53c019aa80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f53b000d950 tx=0x7f53b000de10 comp rx=0 tx=0).stop 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 shutdown_connections 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c0105e30 0x7f53c01a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53ac0a3d20 0x7f53ac0a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53c0105470 0x7f53c019aa80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.473+0000 7f53c8070640 1 --2- 172.21.15.32:0/631502929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c0104280 0x7f53c019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.474+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 >> 172.21.15.32:0/631502929 conn(0x7f53c00ff7f0 msgr2=0x7f53c0108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.474+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 shutdown_connections 2024-04-16T05:34:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.474+0000 7f53c8070640 1 -- 172.21.15.32:0/631502929 wait complete. 2024-04-16T05:34:59.485 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-16T05:34:58.020779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188579,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":994,"num_read_kb":660,"num_write":472,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361596,"kb_used_data":12396,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580228,"statfs":{"total":1151940427776,"available":1151570153472,"internally_reserved":0,"allocated":12693504,"data_stored":9265417,"data_compressed":450707,"data_compressed_allocated":610304,"data_compressed_original":3735552,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.987473"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.640068+0000","last_change":"2024-04-16T05:32:34.759518+0000","last_active":"2024-04-16T05:34:56.640068+0000","last_peered":"2024-04-16T05:34:56.640068+0000","last_clean":"2024-04-16T05:34:56.640068+0000","last_became_active":"2024-04-16T05:32:34.759155+0000","last_became_peered":"2024-04-16T05:32:34.759155+0000","last_unstale":"2024-04-16T05:34:56.640068+0000","last_undegraded":"2024-04-16T05:34:56.640068+0000","last_fullsized":"2024-04-16T05:34:56.640068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:21:39.935886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266968+0000","last_change":"2024-04-16T05:32:34.758553+0000","last_active":"2024-04-16T05:34:56.266968+0000","last_peered":"2024-04-16T05:34:56.266968+0000","last_clean":"2024-04-16T05:34:56.266968+0000","last_became_active":"2024-04-16T05:32:34.758242+0000","last_became_peered":"2024-04-16T05:32:34.758242+0000","last_unstale":"2024-04-16T05:34:56.266968+0000","last_undegraded":"2024-04-16T05:34:56.266968+0000","last_fullsized":"2024-04-16T05:34:56.266968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:09.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266926+0000","last_change":"2024-04-16T05:32:34.754815+0000","last_active":"2024-04-16T05:34:56.266926+0000","last_peered":"2024-04-16T05:34:56.266926+0000","last_clean":"2024-04-16T05:34:56.266926+0000","last_became_active":"2024-04-16T05:32:34.754488+0000","last_became_peered":"2024-04-16T05:32:34.754488+0000","last_unstale":"2024-04-16T05:34:56.266926+0000","last_undegraded":"2024-04-16T05:34:56.266926+0000","last_fullsized":"2024-04-16T05:34:56.266926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:10:32.379770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007584+0000","last_change":"2024-04-16T05:33:34.766742+0000","last_active":"2024-04-16T05:34:56.007584+0000","last_peered":"2024-04-16T05:34:56.007584+0000","last_clean":"2024-04-16T05:34:56.007584+0000","last_became_active":"2024-04-16T05:33:34.766332+0000","last_became_peered":"2024-04-16T05:33:34.766332+0000","last_unstale":"2024-04-16T05:34:56.007584+0000","last_undegraded":"2024-04-16T05:34:56.007584+0000","last_fullsized":"2024-04-16T05:34:56.007584+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:52:14.644496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008710+0000","last_change":"2024-04-16T05:32:34.755623+0000","last_active":"2024-04-16T05:34:56.008710+0000","last_peered":"2024-04-16T05:34:56.008710+0000","last_clean":"2024-04-16T05:34:56.008710+0000","last_became_active":"2024-04-16T05:32:34.754205+0000","last_became_peered":"2024-04-16T05:32:34.754205+0000","last_unstale":"2024-04-16T05:34:56.008710+0000","last_undegraded":"2024-04-16T05:34:56.008710+0000","last_fullsized":"2024-04-16T05:34:56.008710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:34:07.998734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117661+0000","last_change":"2024-04-16T05:32:34.753767+0000","last_active":"2024-04-16T05:34:56.117661+0000","last_peered":"2024-04-16T05:34:56.117661+0000","last_clean":"2024-04-16T05:34:56.117661+0000","last_became_active":"2024-04-16T05:32:34.753448+0000","last_became_peered":"2024-04-16T05:32:34.753448+0000","last_unstale":"2024-04-16T05:34:56.117661+0000","last_undegraded":"2024-04-16T05:34:56.117661+0000","last_fullsized":"2024-04-16T05:34:56.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:27:42.798615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058759+0000","last_change":"2024-04-16T05:32:34.754049+0000","last_active":"2024-04-16T05:34:56.058759+0000","last_peered":"2024-04-16T05:34:56.058759+0000","last_clean":"2024-04-16T05:34:56.058759+0000","last_became_active":"2024-04-16T05:32:34.753757+0000","last_became_peered":"2024-04-16T05:32:34.753757+0000","last_unstale":"2024-04-16T05:34:56.058759+0000","last_undegraded":"2024-04-16T05:34:56.058759+0000","last_fullsized":"2024-04-16T05:34:56.058759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:42:26.747100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105386+0000","last_change":"2024-04-16T05:32:34.762950+0000","last_active":"2024-04-16T05:34:56.105386+0000","last_peered":"2024-04-16T05:34:56.105386+0000","last_clean":"2024-04-16T05:34:56.105386+0000","last_became_active":"2024-04-16T05:32:34.762073+0000","last_became_peered":"2024-04-16T05:32:34.762073+0000","last_unstale":"2024-04-16T05:34:56.105386+0000","last_undegraded":"2024-04-16T05:34:56.105386+0000","last_fullsized":"2024-04-16T05:34:56.105386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:57:26.623655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265111+0000","last_change":"2024-04-16T05:32:34.751027+0000","last_active":"2024-04-16T05:34:56.265111+0000","last_peered":"2024-04-16T05:34:56.265111+0000","last_clean":"2024-04-16T05:34:56.265111+0000","last_became_active":"2024-04-16T05:32:34.750697+0000","last_became_peered":"2024-04-16T05:32:34.750697+0000","last_unstale":"2024-04-16T05:34:56.265111+0000","last_undegraded":"2024-04-16T05:34:56.265111+0000","last_fullsized":"2024-04-16T05:34:56.265111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:39:33.834972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.118031+0000","last_change":"2024-04-16T05:32:34.752293+0000","last_active":"2024-04-16T05:34:56.118031+0000","last_peered":"2024-04-16T05:34:56.118031+0000","last_clean":"2024-04-16T05:34:56.118031+0000","last_became_active":"2024-04-16T05:32:34.752008+0000","last_became_peered":"2024-04-16T05:32:34.752008+0000","last_unstale":"2024-04-16T05:34:56.118031+0000","last_undegraded":"2024-04-16T05:34:56.118031+0000","last_fullsized":"2024-04-16T05:34:56.118031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:02:17.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059051+0000","last_change":"2024-04-16T05:32:34.760961+0000","last_active":"2024-04-16T05:34:56.059051+0000","last_peered":"2024-04-16T05:34:56.059051+0000","last_clean":"2024-04-16T05:34:56.059051+0000","last_became_active":"2024-04-16T05:32:34.760640+0000","last_became_peered":"2024-04-16T05:32:34.760640+0000","last_unstale":"2024-04-16T05:34:56.059051+0000","last_undegraded":"2024-04-16T05:34:56.059051+0000","last_fullsized":"2024-04-16T05:34:56.059051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:47:28.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009246+0000","last_change":"2024-04-16T05:32:34.748646+0000","last_active":"2024-04-16T05:34:56.009246+0000","last_peered":"2024-04-16T05:34:56.009246+0000","last_clean":"2024-04-16T05:34:56.009246+0000","last_became_active":"2024-04-16T05:32:34.748090+0000","last_became_peered":"2024-04-16T05:32:34.748090+0000","last_unstale":"2024-04-16T05:34:56.009246+0000","last_undegraded":"2024-04-16T05:34:56.009246+0000","last_fullsized":"2024-04-16T05:34:56.009246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:00:19.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266536+0000","last_change":"2024-04-16T05:32:34.747025+0000","last_active":"2024-04-16T05:34:56.266536+0000","last_peered":"2024-04-16T05:34:56.266536+0000","last_clean":"2024-04-16T05:34:56.266536+0000","last_became_active":"2024-04-16T05:32:34.746711+0000","last_became_peered":"2024-04-16T05:32:34.746711+0000","last_unstale":"2024-04-16T05:34:56.266536+0000","last_undegraded":"2024-04-16T05:34:56.266536+0000","last_fullsized":"2024-04-16T05:34:56.266536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:40:30.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266616+0000","last_change":"2024-04-16T05:32:34.756756+0000","last_active":"2024-04-16T05:34:56.266616+0000","last_peered":"2024-04-16T05:34:56.266616+0000","last_clean":"2024-04-16T05:34:56.266616+0000","last_became_active":"2024-04-16T05:32:34.756442+0000","last_became_peered":"2024-04-16T05:32:34.756442+0000","last_unstale":"2024-04-16T05:34:56.266616+0000","last_undegraded":"2024-04-16T05:34:56.266616+0000","last_fullsized":"2024-04-16T05:34:56.266616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:21:33.864944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266459+0000","last_change":"2024-04-16T05:32:34.765443+0000","last_active":"2024-04-16T05:34:56.266459+0000","last_peered":"2024-04-16T05:34:56.266459+0000","last_clean":"2024-04-16T05:34:56.266459+0000","last_became_active":"2024-04-16T05:32:34.765133+0000","last_became_peered":"2024-04-16T05:32:34.765133+0000","last_unstale":"2024-04-16T05:34:56.266459+0000","last_undegraded":"2024-04-16T05:34:56.266459+0000","last_fullsized":"2024-04-16T05:34:56.266459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:31.452184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264372+0000","last_change":"2024-04-16T05:32:34.757577+0000","last_active":"2024-04-16T05:34:56.264372+0000","last_peered":"2024-04-16T05:34:56.264372+0000","last_clean":"2024-04-16T05:34:56.264372+0000","last_became_active":"2024-04-16T05:32:34.753451+0000","last_became_peered":"2024-04-16T05:32:34.753451+0000","last_unstale":"2024-04-16T05:34:56.264372+0000","last_undegraded":"2024-04-16T05:34:56.264372+0000","last_fullsized":"2024-04-16T05:34:56.264372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:56.617824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639970+0000","last_change":"2024-04-16T05:32:34.759776+0000","last_active":"2024-04-16T05:34:56.639970+0000","last_peered":"2024-04-16T05:34:56.639970+0000","last_clean":"2024-04-16T05:34:56.639970+0000","last_became_active":"2024-04-16T05:32:34.759371+0000","last_became_peered":"2024-04-16T05:32:34.759371+0000","last_unstale":"2024-04-16T05:34:56.639970+0000","last_undegraded":"2024-04-16T05:34:56.639970+0000","last_fullsized":"2024-04-16T05:34:56.639970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:25:36.458515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007873+0000","last_change":"2024-04-16T05:32:34.747803+0000","last_active":"2024-04-16T05:34:56.007873+0000","last_peered":"2024-04-16T05:34:56.007873+0000","last_clean":"2024-04-16T05:34:56.007873+0000","last_became_active":"2024-04-16T05:32:34.747472+0000","last_became_peered":"2024-04-16T05:32:34.747472+0000","last_unstale":"2024-04-16T05:34:56.007873+0000","last_undegraded":"2024-04-16T05:34:56.007873+0000","last_fullsized":"2024-04-16T05:34:56.007873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:41.118574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264411+0000","last_change":"2024-04-16T05:32:34.754151+0000","last_active":"2024-04-16T05:34:56.264411+0000","last_peered":"2024-04-16T05:34:56.264411+0000","last_clean":"2024-04-16T05:34:56.264411+0000","last_became_active":"2024-04-16T05:32:34.753814+0000","last_became_peered":"2024-04-16T05:32:34.753814+0000","last_unstale":"2024-04-16T05:34:56.264411+0000","last_undegraded":"2024-04-16T05:34:56.264411+0000","last_fullsized":"2024-04-16T05:34:56.264411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:13:00.207978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264276+0000","last_change":"2024-04-16T05:32:38.938563+0000","last_active":"2024-04-16T05:34:56.264276+0000","last_peered":"2024-04-16T05:34:56.264276+0000","last_clean":"2024-04-16T05:34:56.264276+0000","last_became_active":"2024-04-16T05:32:38.938097+0000","last_became_peered":"2024-04-16T05:32:38.938097+0000","last_unstale":"2024-04-16T05:34:56.264276+0000","last_undegraded":"2024-04-16T05:34:56.264276+0000","last_fullsized":"2024-04-16T05:34:56.264276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:47:52.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007623+0000","last_change":"2024-04-16T05:32:33.735694+0000","last_active":"2024-04-16T05:34:56.007623+0000","last_peered":"2024-04-16T05:34:56.007623+0000","last_clean":"2024-04-16T05:34:56.007623+0000","last_became_active":"2024-04-16T05:32:33.735414+0000","last_became_peered":"2024-04-16T05:32:33.735414+0000","last_unstale":"2024-04-16T05:34:56.007623+0000","last_undegraded":"2024-04-16T05:34:56.007623+0000","last_fullsized":"2024-04-16T05:34:56.007623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:19:06.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107049+0000","last_change":"2024-04-16T05:32:34.762315+0000","last_active":"2024-04-16T05:34:56.107049+0000","last_peered":"2024-04-16T05:34:56.107049+0000","last_clean":"2024-04-16T05:34:56.107049+0000","last_became_active":"2024-04-16T05:32:34.761969+0000","last_became_peered":"2024-04-16T05:32:34.761969+0000","last_unstale":"2024-04-16T05:34:56.107049+0000","last_undegraded":"2024-04-16T05:34:56.107049+0000","last_fullsized":"2024-04-16T05:34:56.107049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:00:45.166966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008832+0000","last_change":"2024-04-16T05:32:38.940681+0000","last_active":"2024-04-16T05:34:56.008832+0000","last_peered":"2024-04-16T05:34:56.008832+0000","last_clean":"2024-04-16T05:34:56.008832+0000","last_became_active":"2024-04-16T05:32:38.940414+0000","last_became_peered":"2024-04-16T05:32:38.940414+0000","last_unstale":"2024-04-16T05:34:56.008832+0000","last_undegraded":"2024-04-16T05:34:56.008832+0000","last_fullsized":"2024-04-16T05:34:56.008832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:46:50.061607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107418+0000","last_change":"2024-04-16T05:32:33.748333+0000","last_active":"2024-04-16T05:34:56.107418+0000","last_peered":"2024-04-16T05:34:56.107418+0000","last_clean":"2024-04-16T05:34:56.107418+0000","last_became_active":"2024-04-16T05:32:33.748022+0000","last_became_peered":"2024-04-16T05:32:33.748022+0000","last_unstale":"2024-04-16T05:34:56.107418+0000","last_undegraded":"2024-04-16T05:34:56.107418+0000","last_fullsized":"2024-04-16T05:34:56.107418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:50:27.771245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266977+0000","last_change":"2024-04-16T05:32:34.764832+0000","last_active":"2024-04-16T05:34:56.266977+0000","last_peered":"2024-04-16T05:34:56.266977+0000","last_clean":"2024-04-16T05:34:56.266977+0000","last_became_active":"2024-04-16T05:32:34.763180+0000","last_became_peered":"2024-04-16T05:32:34.763180+0000","last_unstale":"2024-04-16T05:34:56.266977+0000","last_undegraded":"2024-04-16T05:34:56.266977+0000","last_fullsized":"2024-04-16T05:34:56.266977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:37:27.804376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009531+0000","last_change":"2024-04-16T05:32:38.937226+0000","last_active":"2024-04-16T05:34:56.009531+0000","last_peered":"2024-04-16T05:34:56.009531+0000","last_clean":"2024-04-16T05:34:56.009531+0000","last_became_active":"2024-04-16T05:32:38.936770+0000","last_became_peered":"2024-04-16T05:32:38.936770+0000","last_unstale":"2024-04-16T05:34:56.009531+0000","last_undegraded":"2024-04-16T05:34:56.009531+0000","last_fullsized":"2024-04-16T05:34:56.009531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:05:28.059175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639811+0000","last_change":"2024-04-16T05:33:34.775689+0000","last_active":"2024-04-16T05:34:56.639811+0000","last_peered":"2024-04-16T05:34:56.639811+0000","last_clean":"2024-04-16T05:34:56.639811+0000","last_became_active":"2024-04-16T05:33:34.770520+0000","last_became_peered":"2024-04-16T05:33:34.770520+0000","last_unstale":"2024-04-16T05:34:56.639811+0000","last_undegraded":"2024-04-16T05:34:56.639811+0000","last_fullsized":"2024-04-16T05:34:56.639811+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:40:57.616795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264658+0000","last_change":"2024-04-16T05:32:34.758891+0000","last_active":"2024-04-16T05:34:56.264658+0000","last_peered":"2024-04-16T05:34:56.264658+0000","last_clean":"2024-04-16T05:34:56.264658+0000","last_became_active":"2024-04-16T05:32:34.758559+0000","last_became_peered":"2024-04-16T05:32:34.758559+0000","last_unstale":"2024-04-16T05:34:56.264658+0000","last_undegraded":"2024-04-16T05:34:56.264658+0000","last_fullsized":"2024-04-16T05:34:56.264658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:34:31.213992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008864+0000","last_change":"2024-04-16T05:32:38.937163+0000","last_active":"2024-04-16T05:34:56.008864+0000","last_peered":"2024-04-16T05:34:56.008864+0000","last_clean":"2024-04-16T05:34:56.008864+0000","last_became_active":"2024-04-16T05:32:38.936745+0000","last_became_peered":"2024-04-16T05:32:38.936745+0000","last_unstale":"2024-04-16T05:34:56.008864+0000","last_undegraded":"2024-04-16T05:34:56.008864+0000","last_fullsized":"2024-04-16T05:34:56.008864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:54:37.915041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008806+0000","last_change":"2024-04-16T05:32:33.731842+0000","last_active":"2024-04-16T05:34:56.008806+0000","last_peered":"2024-04-16T05:34:56.008806+0000","last_clean":"2024-04-16T05:34:56.008806+0000","last_became_active":"2024-04-16T05:32:33.731417+0000","last_became_peered":"2024-04-16T05:32:33.731417+0000","last_unstale":"2024-04-16T05:34:56.008806+0000","last_undegraded":"2024-04-16T05:34:56.008806+0000","last_fullsized":"2024-04-16T05:34:56.008806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:04:03.141533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114569+0000","last_change":"2024-04-16T05:32:34.755062+0000","last_active":"2024-04-16T05:34:56.114569+0000","last_peered":"2024-04-16T05:34:56.114569+0000","last_clean":"2024-04-16T05:34:56.114569+0000","last_became_active":"2024-04-16T05:32:34.754759+0000","last_became_peered":"2024-04-16T05:32:34.754759+0000","last_unstale":"2024-04-16T05:34:56.114569+0000","last_undegraded":"2024-04-16T05:34:56.114569+0000","last_fullsized":"2024-04-16T05:34:56.114569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:53:59.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264299+0000","last_change":"2024-04-16T05:32:38.935646+0000","last_active":"2024-04-16T05:34:56.264299+0000","last_peered":"2024-04-16T05:34:56.264299+0000","last_clean":"2024-04-16T05:34:56.264299+0000","last_became_active":"2024-04-16T05:32:38.935206+0000","last_became_peered":"2024-04-16T05:32:38.935206+0000","last_unstale":"2024-04-16T05:34:56.264299+0000","last_undegraded":"2024-04-16T05:34:56.264299+0000","last_fullsized":"2024-04-16T05:34:56.264299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:09.772136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114682+0000","last_change":"2024-04-16T05:32:33.739343+0000","last_active":"2024-04-16T05:34:56.114682+0000","last_peered":"2024-04-16T05:34:56.114682+0000","last_clean":"2024-04-16T05:34:56.114682+0000","last_became_active":"2024-04-16T05:32:33.738982+0000","last_became_peered":"2024-04-16T05:32:33.738982+0000","last_unstale":"2024-04-16T05:34:56.114682+0000","last_undegraded":"2024-04-16T05:34:56.114682+0000","last_fullsized":"2024-04-16T05:34:56.114682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:36.002804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008721+0000","last_change":"2024-04-16T05:32:34.749316+0000","last_active":"2024-04-16T05:34:56.008721+0000","last_peered":"2024-04-16T05:34:56.008721+0000","last_clean":"2024-04-16T05:34:56.008721+0000","last_became_active":"2024-04-16T05:32:34.748976+0000","last_became_peered":"2024-04-16T05:32:34.748976+0000","last_unstale":"2024-04-16T05:34:56.008721+0000","last_undegraded":"2024-04-16T05:34:56.008721+0000","last_fullsized":"2024-04-16T05:34:56.008721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:16:29.569402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264259+0000","last_change":"2024-04-16T05:32:38.938633+0000","last_active":"2024-04-16T05:34:56.264259+0000","last_peered":"2024-04-16T05:34:56.264259+0000","last_clean":"2024-04-16T05:34:56.264259+0000","last_became_active":"2024-04-16T05:32:38.938215+0000","last_became_peered":"2024-04-16T05:32:38.938215+0000","last_unstale":"2024-04-16T05:34:56.264259+0000","last_undegraded":"2024-04-16T05:34:56.264259+0000","last_fullsized":"2024-04-16T05:34:56.264259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:03:05.078109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117612+0000","last_change":"2024-04-16T05:32:33.744262+0000","last_active":"2024-04-16T05:34:56.117612+0000","last_peered":"2024-04-16T05:34:56.117612+0000","last_clean":"2024-04-16T05:34:56.117612+0000","last_became_active":"2024-04-16T05:32:33.743866+0000","last_became_peered":"2024-04-16T05:32:33.743866+0000","last_unstale":"2024-04-16T05:34:56.117612+0000","last_undegraded":"2024-04-16T05:34:56.117612+0000","last_fullsized":"2024-04-16T05:34:56.117612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:22:03.121770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114365+0000","last_change":"2024-04-16T05:32:34.750201+0000","last_active":"2024-04-16T05:34:56.114365+0000","last_peered":"2024-04-16T05:34:56.114365+0000","last_clean":"2024-04-16T05:34:56.114365+0000","last_became_active":"2024-04-16T05:32:34.749900+0000","last_became_peered":"2024-04-16T05:32:34.749900+0000","last_unstale":"2024-04-16T05:34:56.114365+0000","last_undegraded":"2024-04-16T05:34:56.114365+0000","last_fullsized":"2024-04-16T05:34:56.114365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:42.708584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639920+0000","last_change":"2024-04-16T05:32:38.931277+0000","last_active":"2024-04-16T05:34:56.639920+0000","last_peered":"2024-04-16T05:34:56.639920+0000","last_clean":"2024-04-16T05:34:56.639920+0000","last_became_active":"2024-04-16T05:32:38.930888+0000","last_became_peered":"2024-04-16T05:32:38.930888+0000","last_unstale":"2024-04-16T05:34:56.639920+0000","last_undegraded":"2024-04-16T05:34:56.639920+0000","last_fullsized":"2024-04-16T05:34:56.639920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:27:31.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264559+0000","last_change":"2024-04-16T05:32:33.747740+0000","last_active":"2024-04-16T05:34:56.264559+0000","last_peered":"2024-04-16T05:34:56.264559+0000","last_clean":"2024-04-16T05:34:56.264559+0000","last_became_active":"2024-04-16T05:32:33.747313+0000","last_became_peered":"2024-04-16T05:32:33.747313+0000","last_unstale":"2024-04-16T05:34:56.264559+0000","last_undegraded":"2024-04-16T05:34:56.264559+0000","last_fullsized":"2024-04-16T05:34:56.264559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:05:03.937790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105220+0000","last_change":"2024-04-16T05:32:34.760075+0000","last_active":"2024-04-16T05:34:56.105220+0000","last_peered":"2024-04-16T05:34:56.105220+0000","last_clean":"2024-04-16T05:34:56.105220+0000","last_became_active":"2024-04-16T05:32:34.759569+0000","last_became_peered":"2024-04-16T05:32:34.759569+0000","last_unstale":"2024-04-16T05:34:56.105220+0000","last_undegraded":"2024-04-16T05:34:56.105220+0000","last_fullsized":"2024-04-16T05:34:56.105220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:12:54.222877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266895+0000","last_change":"2024-04-16T05:32:38.928568+0000","last_active":"2024-04-16T05:34:56.266895+0000","last_peered":"2024-04-16T05:34:56.266895+0000","last_clean":"2024-04-16T05:34:56.266895+0000","last_became_active":"2024-04-16T05:32:38.928109+0000","last_became_peered":"2024-04-16T05:32:38.928109+0000","last_unstale":"2024-04-16T05:34:56.266895+0000","last_undegraded":"2024-04-16T05:34:56.266895+0000","last_fullsized":"2024-04-16T05:34:56.266895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:24:14.293528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114216+0000","last_change":"2024-04-16T05:32:33.737803+0000","last_active":"2024-04-16T05:34:56.114216+0000","last_peered":"2024-04-16T05:34:56.114216+0000","last_clean":"2024-04-16T05:34:56.114216+0000","last_became_active":"2024-04-16T05:32:33.737448+0000","last_became_peered":"2024-04-16T05:32:33.737448+0000","last_unstale":"2024-04-16T05:34:56.114216+0000","last_undegraded":"2024-04-16T05:34:56.114216+0000","last_fullsized":"2024-04-16T05:34:56.114216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:44:54.302323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007203+0000","last_change":"2024-04-16T05:32:34.754070+0000","last_active":"2024-04-16T05:34:56.007203+0000","last_peered":"2024-04-16T05:34:56.007203+0000","last_clean":"2024-04-16T05:34:56.007203+0000","last_became_active":"2024-04-16T05:32:34.753717+0000","last_became_peered":"2024-04-16T05:32:34.753717+0000","last_unstale":"2024-04-16T05:34:56.007203+0000","last_undegraded":"2024-04-16T05:34:56.007203+0000","last_fullsized":"2024-04-16T05:34:56.007203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:58:56.738245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.106064+0000","last_change":"2024-04-16T05:32:38.931594+0000","last_active":"2024-04-16T05:34:56.106064+0000","last_peered":"2024-04-16T05:34:56.106064+0000","last_clean":"2024-04-16T05:34:56.106064+0000","last_became_active":"2024-04-16T05:32:38.931188+0000","last_became_peered":"2024-04-16T05:32:38.931188+0000","last_unstale":"2024-04-16T05:34:56.106064+0000","last_undegraded":"2024-04-16T05:34:56.106064+0000","last_fullsized":"2024-04-16T05:34:56.106064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:59:55.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.118099+0000","last_change":"2024-04-16T05:32:33.739752+0000","last_active":"2024-04-16T05:34:56.118099+0000","last_peered":"2024-04-16T05:34:56.118099+0000","last_clean":"2024-04-16T05:34:56.118099+0000","last_became_active":"2024-04-16T05:32:33.739444+0000","last_became_peered":"2024-04-16T05:32:33.739444+0000","last_unstale":"2024-04-16T05:34:56.118099+0000","last_undegraded":"2024-04-16T05:34:56.118099+0000","last_fullsized":"2024-04-16T05:34:56.118099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:04:18.091618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265179+0000","last_change":"2024-04-16T05:32:34.759197+0000","last_active":"2024-04-16T05:34:56.265179+0000","last_peered":"2024-04-16T05:34:56.265179+0000","last_clean":"2024-04-16T05:34:56.265179+0000","last_became_active":"2024-04-16T05:32:34.758783+0000","last_became_peered":"2024-04-16T05:32:34.758783+0000","last_unstale":"2024-04-16T05:34:56.265179+0000","last_undegraded":"2024-04-16T05:34:56.265179+0000","last_fullsized":"2024-04-16T05:34:56.265179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:23:37.020254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007139+0000","last_change":"2024-04-16T05:32:38.936198+0000","last_active":"2024-04-16T05:34:56.007139+0000","last_peered":"2024-04-16T05:34:56.007139+0000","last_clean":"2024-04-16T05:34:56.007139+0000","last_became_active":"2024-04-16T05:32:38.935698+0000","last_became_peered":"2024-04-16T05:32:38.935698+0000","last_unstale":"2024-04-16T05:34:56.007139+0000","last_undegraded":"2024-04-16T05:34:56.007139+0000","last_fullsized":"2024-04-16T05:34:56.007139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:17:35.602408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058994+0000","last_change":"2024-04-16T05:32:33.737653+0000","last_active":"2024-04-16T05:34:56.058994+0000","last_peered":"2024-04-16T05:34:56.058994+0000","last_clean":"2024-04-16T05:34:56.058994+0000","last_became_active":"2024-04-16T05:32:33.737342+0000","last_became_peered":"2024-04-16T05:32:33.737342+0000","last_unstale":"2024-04-16T05:34:56.058994+0000","last_undegraded":"2024-04-16T05:34:56.058994+0000","last_fullsized":"2024-04-16T05:34:56.058994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:17:35.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266896+0000","last_change":"2024-04-16T05:32:34.765722+0000","last_active":"2024-04-16T05:34:56.266896+0000","last_peered":"2024-04-16T05:34:56.266896+0000","last_clean":"2024-04-16T05:34:56.266896+0000","last_became_active":"2024-04-16T05:32:34.765413+0000","last_became_peered":"2024-04-16T05:32:34.765413+0000","last_unstale":"2024-04-16T05:34:56.266896+0000","last_undegraded":"2024-04-16T05:34:56.266896+0000","last_fullsized":"2024-04-16T05:34:56.266896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:05:03.884041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009328+0000","last_change":"2024-04-16T05:32:38.936186+0000","last_active":"2024-04-16T05:34:56.009328+0000","last_peered":"2024-04-16T05:34:56.009328+0000","last_clean":"2024-04-16T05:34:56.009328+0000","last_became_active":"2024-04-16T05:32:38.935738+0000","last_became_peered":"2024-04-16T05:32:38.935738+0000","last_unstale":"2024-04-16T05:34:56.009328+0000","last_undegraded":"2024-04-16T05:34:56.009328+0000","last_fullsized":"2024-04-16T05:34:56.009328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:10.519095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114565+0000","last_change":"2024-04-16T05:32:33.742991+0000","last_active":"2024-04-16T05:34:56.114565+0000","last_peered":"2024-04-16T05:34:56.114565+0000","last_clean":"2024-04-16T05:34:56.114565+0000","last_became_active":"2024-04-16T05:32:33.742659+0000","last_became_peered":"2024-04-16T05:32:33.742659+0000","last_unstale":"2024-04-16T05:34:56.114565+0000","last_undegraded":"2024-04-16T05:34:56.114565+0000","last_fullsized":"2024-04-16T05:34:56.114565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:23.756123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264808+0000","last_change":"2024-04-16T05:32:34.759926+0000","last_active":"2024-04-16T05:34:56.264808+0000","last_peered":"2024-04-16T05:34:56.264808+0000","last_clean":"2024-04-16T05:34:56.264808+0000","last_became_active":"2024-04-16T05:32:34.759626+0000","last_became_peered":"2024-04-16T05:32:34.759626+0000","last_unstale":"2024-04-16T05:34:56.264808+0000","last_undegraded":"2024-04-16T05:34:56.264808+0000","last_fullsized":"2024-04-16T05:34:56.264808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:57:50.314638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007364+0000","last_change":"2024-04-16T05:32:38.942135+0000","last_active":"2024-04-16T05:34:56.007364+0000","last_peered":"2024-04-16T05:34:56.007364+0000","last_clean":"2024-04-16T05:34:56.007364+0000","last_became_active":"2024-04-16T05:32:38.941534+0000","last_became_peered":"2024-04-16T05:32:38.941534+0000","last_unstale":"2024-04-16T05:34:56.007364+0000","last_undegraded":"2024-04-16T05:34:56.007364+0000","last_fullsized":"2024-04-16T05:34:56.007364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:02:20.494516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":323,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.873982+0000","last_change":"2024-04-16T05:32:33.734275+0000","last_active":"2024-04-16T05:34:56.873982+0000","last_peered":"2024-04-16T05:34:56.873982+0000","last_clean":"2024-04-16T05:34:56.873982+0000","last_became_active":"2024-04-16T05:32:33.733906+0000","last_became_peered":"2024-04-16T05:32:33.733906+0000","last_unstale":"2024-04-16T05:34:56.873982+0000","last_undegraded":"2024-04-16T05:34:56.873982+0000","last_fullsized":"2024-04-16T05:34:56.873982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:39:23.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":195,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008907+0000","last_change":"2024-04-16T05:32:34.750179+0000","last_active":"2024-04-16T05:34:56.008907+0000","last_peered":"2024-04-16T05:34:56.008907+0000","last_clean":"2024-04-16T05:34:56.008907+0000","last_became_active":"2024-04-16T05:32:34.749773+0000","last_became_peered":"2024-04-16T05:32:34.749773+0000","last_unstale":"2024-04-16T05:34:56.008907+0000","last_undegraded":"2024-04-16T05:34:56.008907+0000","last_fullsized":"2024-04-16T05:34:56.008907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:43:37.621191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114863+0000","last_change":"2024-04-16T05:32:38.935690+0000","last_active":"2024-04-16T05:34:56.114863+0000","last_peered":"2024-04-16T05:34:56.114863+0000","last_clean":"2024-04-16T05:34:56.114863+0000","last_became_active":"2024-04-16T05:32:38.935282+0000","last_became_peered":"2024-04-16T05:32:38.935282+0000","last_unstale":"2024-04-16T05:34:56.114863+0000","last_undegraded":"2024-04-16T05:34:56.114863+0000","last_fullsized":"2024-04-16T05:34:56.114863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:25:00.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639603+0000","last_change":"2024-04-16T05:33:34.782195+0000","last_active":"2024-04-16T05:34:56.639603+0000","last_peered":"2024-04-16T05:34:56.639603+0000","last_clean":"2024-04-16T05:34:56.639603+0000","last_became_active":"2024-04-16T05:33:34.771490+0000","last_became_peered":"2024-04-16T05:33:34.771490+0000","last_unstale":"2024-04-16T05:34:56.639603+0000","last_undegraded":"2024-04-16T05:34:56.639603+0000","last_fullsized":"2024-04-16T05:34:56.639603+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:25:52.513976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.265873+0000","last_change":"2024-04-16T05:32:34.748760+0000","last_active":"2024-04-16T05:34:56.265873+0000","last_peered":"2024-04-16T05:34:56.265873+0000","last_clean":"2024-04-16T05:34:56.265873+0000","last_became_active":"2024-04-16T05:32:34.748432+0000","last_became_peered":"2024-04-16T05:32:34.748432+0000","last_unstale":"2024-04-16T05:34:56.265873+0000","last_undegraded":"2024-04-16T05:34:56.265873+0000","last_fullsized":"2024-04-16T05:34:56.265873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:12:42.859342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639575+0000","last_change":"2024-04-16T05:32:38.929705+0000","last_active":"2024-04-16T05:34:56.639575+0000","last_peered":"2024-04-16T05:34:56.639575+0000","last_clean":"2024-04-16T05:34:56.639575+0000","last_became_active":"2024-04-16T05:32:38.929282+0000","last_became_peered":"2024-04-16T05:32:38.929282+0000","last_unstale":"2024-04-16T05:34:56.639575+0000","last_undegraded":"2024-04-16T05:34:56.639575+0000","last_fullsized":"2024-04-16T05:34:56.639575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:51:27.763600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114463+0000","last_change":"2024-04-16T05:32:33.746371+0000","last_active":"2024-04-16T05:34:56.114463+0000","last_peered":"2024-04-16T05:34:56.114463+0000","last_clean":"2024-04-16T05:34:56.114463+0000","last_became_active":"2024-04-16T05:32:33.746077+0000","last_became_peered":"2024-04-16T05:32:33.746077+0000","last_unstale":"2024-04-16T05:34:56.114463+0000","last_undegraded":"2024-04-16T05:34:56.114463+0000","last_fullsized":"2024-04-16T05:34:56.114463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:57:43.193942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059205+0000","last_change":"2024-04-16T05:32:34.753196+0000","last_active":"2024-04-16T05:34:56.059205+0000","last_peered":"2024-04-16T05:34:56.059205+0000","last_clean":"2024-04-16T05:34:56.059205+0000","last_became_active":"2024-04-16T05:32:34.752881+0000","last_became_peered":"2024-04-16T05:32:34.752881+0000","last_unstale":"2024-04-16T05:34:56.059205+0000","last_undegraded":"2024-04-16T05:34:56.059205+0000","last_fullsized":"2024-04-16T05:34:56.059205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:35:56.715394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008805+0000","last_change":"2024-04-16T05:33:34.772562+0000","last_active":"2024-04-16T05:34:56.008805+0000","last_peered":"2024-04-16T05:34:56.008805+0000","last_clean":"2024-04-16T05:34:56.008805+0000","last_became_active":"2024-04-16T05:33:34.772052+0000","last_became_peered":"2024-04-16T05:33:34.772052+0000","last_unstale":"2024-04-16T05:34:56.008805+0000","last_undegraded":"2024-04-16T05:34:56.008805+0000","last_fullsized":"2024-04-16T05:34:56.008805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:50:18.425870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.873651+0000","last_change":"2024-04-16T05:32:33.735921+0000","last_active":"2024-04-16T05:34:56.873651+0000","last_peered":"2024-04-16T05:34:56.873651+0000","last_clean":"2024-04-16T05:34:56.873651+0000","last_became_active":"2024-04-16T05:32:33.735572+0000","last_became_peered":"2024-04-16T05:32:33.735572+0000","last_unstale":"2024-04-16T05:34:56.873651+0000","last_undegraded":"2024-04-16T05:34:56.873651+0000","last_fullsized":"2024-04-16T05:34:56.873651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:46:56.512827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058872+0000","last_change":"2024-04-16T05:32:34.762172+0000","last_active":"2024-04-16T05:34:56.058872+0000","last_peered":"2024-04-16T05:34:56.058872+0000","last_clean":"2024-04-16T05:34:56.058872+0000","last_became_active":"2024-04-16T05:32:34.761914+0000","last_became_peered":"2024-04-16T05:32:34.761914+0000","last_unstale":"2024-04-16T05:34:56.058872+0000","last_undegraded":"2024-04-16T05:34:56.058872+0000","last_fullsized":"2024-04-16T05:34:56.058872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:47:52.302117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267525+0000","last_change":"2024-04-16T05:33:34.772546+0000","last_active":"2024-04-16T05:34:56.267525+0000","last_peered":"2024-04-16T05:34:56.267525+0000","last_clean":"2024-04-16T05:34:56.267525+0000","last_became_active":"2024-04-16T05:33:34.772100+0000","last_became_peered":"2024-04-16T05:33:34.772100+0000","last_unstale":"2024-04-16T05:34:56.267525+0000","last_undegraded":"2024-04-16T05:34:56.267525+0000","last_fullsized":"2024-04-16T05:34:56.267525+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:11:50.157129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008514+0000","last_change":"2024-04-16T05:32:33.743944+0000","last_active":"2024-04-16T05:34:56.008514+0000","last_peered":"2024-04-16T05:34:56.008514+0000","last_clean":"2024-04-16T05:34:56.008514+0000","last_became_active":"2024-04-16T05:32:33.743559+0000","last_became_peered":"2024-04-16T05:32:33.743559+0000","last_unstale":"2024-04-16T05:34:56.008514+0000","last_undegraded":"2024-04-16T05:34:56.008514+0000","last_fullsized":"2024-04-16T05:34:56.008514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:08:48.889356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114332+0000","last_change":"2024-04-16T05:32:34.756741+0000","last_active":"2024-04-16T05:34:56.114332+0000","last_peered":"2024-04-16T05:34:56.114332+0000","last_clean":"2024-04-16T05:34:56.114332+0000","last_became_active":"2024-04-16T05:32:34.756426+0000","last_became_peered":"2024-04-16T05:32:34.756426+0000","last_unstale":"2024-04-16T05:34:56.114332+0000","last_undegraded":"2024-04-16T05:34:56.114332+0000","last_fullsized":"2024-04-16T05:34:56.114332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:09:30.493857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114287+0000","last_change":"2024-04-16T05:32:38.934818+0000","last_active":"2024-04-16T05:34:56.114287+0000","last_peered":"2024-04-16T05:34:56.114287+0000","last_clean":"2024-04-16T05:34:56.114287+0000","last_became_active":"2024-04-16T05:32:38.934335+0000","last_became_peered":"2024-04-16T05:32:38.934335+0000","last_unstale":"2024-04-16T05:34:56.114287+0000","last_undegraded":"2024-04-16T05:34:56.114287+0000","last_fullsized":"2024-04-16T05:34:56.114287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:02:41.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009144+0000","last_change":"2024-04-16T05:32:33.731951+0000","last_active":"2024-04-16T05:34:56.009144+0000","last_peered":"2024-04-16T05:34:56.009144+0000","last_clean":"2024-04-16T05:34:56.009144+0000","last_became_active":"2024-04-16T05:32:33.731649+0000","last_became_peered":"2024-04-16T05:32:33.731649+0000","last_unstale":"2024-04-16T05:34:56.009144+0000","last_undegraded":"2024-04-16T05:34:56.009144+0000","last_fullsized":"2024-04-16T05:34:56.009144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:41.258012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639876+0000","last_change":"2024-04-16T05:32:34.751884+0000","last_active":"2024-04-16T05:34:56.639876+0000","last_peered":"2024-04-16T05:34:56.639876+0000","last_clean":"2024-04-16T05:34:56.639876+0000","last_became_active":"2024-04-16T05:32:34.751559+0000","last_became_peered":"2024-04-16T05:32:34.751559+0000","last_unstale":"2024-04-16T05:34:56.639876+0000","last_undegraded":"2024-04-16T05:34:56.639876+0000","last_fullsized":"2024-04-16T05:34:56.639876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T05:32:59.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008772+0000","last_change":"2024-04-16T05:32:39.109330+0000","last_active":"2024-04-16T05:34:56.008772+0000","last_peered":"2024-04-16T05:34:56.008772+0000","last_clean":"2024-04-16T05:34:56.008772+0000","last_became_active":"2024-04-16T05:32:39.108718+0000","last_became_peered":"2024-04-16T05:32:39.108718+0000","last_unstale":"2024-04-16T05:34:56.008772+0000","last_undegraded":"2024-04-16T05:34:56.008772+0000","last_fullsized":"2024-04-16T05:34:56.008772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:36:55.418034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007436+0000","last_change":"2024-04-16T05:32:33.741603+0000","last_active":"2024-04-16T05:34:56.007436+0000","last_peered":"2024-04-16T05:34:56.007436+0000","last_clean":"2024-04-16T05:34:56.007436+0000","last_became_active":"2024-04-16T05:32:33.741184+0000","last_became_peered":"2024-04-16T05:32:33.741184+0000","last_unstale":"2024-04-16T05:34:56.007436+0000","last_undegraded":"2024-04-16T05:34:56.007436+0000","last_fullsized":"2024-04-16T05:34:56.007436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:08:38.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117997+0000","last_change":"2024-04-16T05:32:34.747768+0000","last_active":"2024-04-16T05:34:56.117997+0000","last_peered":"2024-04-16T05:34:56.117997+0000","last_clean":"2024-04-16T05:34:56.117997+0000","last_became_active":"2024-04-16T05:32:34.747440+0000","last_became_peered":"2024-04-16T05:32:34.747440+0000","last_unstale":"2024-04-16T05:34:56.117997+0000","last_undegraded":"2024-04-16T05:34:56.117997+0000","last_fullsized":"2024-04-16T05:34:56.117997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:14:31.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266840+0000","last_change":"2024-04-16T05:32:38.938673+0000","last_active":"2024-04-16T05:34:56.266840+0000","last_peered":"2024-04-16T05:34:56.266840+0000","last_clean":"2024-04-16T05:34:56.266840+0000","last_became_active":"2024-04-16T05:32:38.938252+0000","last_became_peered":"2024-04-16T05:32:38.938252+0000","last_unstale":"2024-04-16T05:34:56.266840+0000","last_undegraded":"2024-04-16T05:34:56.266840+0000","last_fullsized":"2024-04-16T05:34:56.266840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:06.620693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058730+0000","last_change":"2024-04-16T05:32:33.750812+0000","last_active":"2024-04-16T05:34:56.058730+0000","last_peered":"2024-04-16T05:34:56.058730+0000","last_clean":"2024-04-16T05:34:56.058730+0000","last_became_active":"2024-04-16T05:32:33.750514+0000","last_became_peered":"2024-04-16T05:32:33.750514+0000","last_unstale":"2024-04-16T05:34:56.058730+0000","last_undegraded":"2024-04-16T05:34:56.058730+0000","last_fullsized":"2024-04-16T05:34:56.058730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:53:43.075914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.117672+0000","last_change":"2024-04-16T05:33:34.770449+0000","last_active":"2024-04-16T05:34:56.117672+0000","last_peered":"2024-04-16T05:34:56.117672+0000","last_clean":"2024-04-16T05:34:56.117672+0000","last_became_active":"2024-04-16T05:33:34.770078+0000","last_became_peered":"2024-04-16T05:33:34.770078+0000","last_unstale":"2024-04-16T05:34:56.117672+0000","last_undegraded":"2024-04-16T05:34:56.117672+0000","last_fullsized":"2024-04-16T05:34:56.117672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:30:20.885002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267319+0000","last_change":"2024-04-16T05:32:39.109394+0000","last_active":"2024-04-16T05:34:56.267319+0000","last_peered":"2024-04-16T05:34:56.267319+0000","last_clean":"2024-04-16T05:34:56.267319+0000","last_became_active":"2024-04-16T05:32:39.108924+0000","last_became_peered":"2024-04-16T05:32:39.108924+0000","last_unstale":"2024-04-16T05:34:56.267319+0000","last_undegraded":"2024-04-16T05:34:56.267319+0000","last_fullsized":"2024-04-16T05:34:56.267319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:50:37.748681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007055+0000","last_change":"2024-04-16T05:33:34.766071+0000","last_active":"2024-04-16T05:34:56.007055+0000","last_peered":"2024-04-16T05:34:56.007055+0000","last_clean":"2024-04-16T05:34:56.007055+0000","last_became_active":"2024-04-16T05:33:34.765670+0000","last_became_peered":"2024-04-16T05:33:34.765670+0000","last_unstale":"2024-04-16T05:34:56.007055+0000","last_undegraded":"2024-04-16T05:34:56.007055+0000","last_fullsized":"2024-04-16T05:34:56.007055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:54:47.320828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267990+0000","last_change":"2024-04-16T05:32:33.748521+0000","last_active":"2024-04-16T05:34:56.267990+0000","last_peered":"2024-04-16T05:34:56.267990+0000","last_clean":"2024-04-16T05:34:56.267990+0000","last_became_active":"2024-04-16T05:32:33.748222+0000","last_became_peered":"2024-04-16T05:32:33.748222+0000","last_unstale":"2024-04-16T05:34:56.267990+0000","last_undegraded":"2024-04-16T05:34:56.267990+0000","last_fullsized":"2024-04-16T05:34:56.267990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:04.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058859+0000","last_change":"2024-04-16T05:32:38.935995+0000","last_active":"2024-04-16T05:34:56.058859+0000","last_peered":"2024-04-16T05:34:56.058859+0000","last_clean":"2024-04-16T05:34:56.058859+0000","last_became_active":"2024-04-16T05:32:38.935511+0000","last_became_peered":"2024-04-16T05:32:38.935511+0000","last_unstale":"2024-04-16T05:34:56.058859+0000","last_undegraded":"2024-04-16T05:34:56.058859+0000","last_fullsized":"2024-04-16T05:34:56.058859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:17:08.144537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105847+0000","last_change":"2024-04-16T05:32:34.760249+0000","last_active":"2024-04-16T05:34:56.105847+0000","last_peered":"2024-04-16T05:34:56.105847+0000","last_clean":"2024-04-16T05:34:56.105847+0000","last_became_active":"2024-04-16T05:32:34.759735+0000","last_became_peered":"2024-04-16T05:32:34.759735+0000","last_unstale":"2024-04-16T05:34:56.105847+0000","last_undegraded":"2024-04-16T05:34:56.105847+0000","last_fullsized":"2024-04-16T05:34:56.105847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:41:13.349558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009119+0000","last_change":"2024-04-16T05:32:33.731170+0000","last_active":"2024-04-16T05:34:56.009119+0000","last_peered":"2024-04-16T05:34:56.009119+0000","last_clean":"2024-04-16T05:34:56.009119+0000","last_became_active":"2024-04-16T05:32:33.730821+0000","last_became_peered":"2024-04-16T05:32:33.730821+0000","last_unstale":"2024-04-16T05:34:56.009119+0000","last_undegraded":"2024-04-16T05:34:56.009119+0000","last_fullsized":"2024-04-16T05:34:56.009119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:38:15.707590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":365,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058562+0000","last_change":"2024-04-16T05:31:24.926319+0000","last_active":"2024-04-16T05:34:56.058562+0000","last_peered":"2024-04-16T05:34:56.058562+0000","last_clean":"2024-04-16T05:34:56.058562+0000","last_became_active":"2024-04-16T05:31:24.925936+0000","last_became_peered":"2024-04-16T05:31:24.925936+0000","last_unstale":"2024-04-16T05:34:56.058562+0000","last_undegraded":"2024-04-16T05:34:56.058562+0000","last_fullsized":"2024-04-16T05:34:56.058562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:27:57.876673+0000","last_clean_scrub_stamp":"2024-04-16T05:27:57.876673+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:28:19.212127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114596+0000","last_change":"2024-04-16T05:32:39.110241+0000","last_active":"2024-04-16T05:34:56.114596+0000","last_peered":"2024-04-16T05:34:56.114596+0000","last_clean":"2024-04-16T05:34:56.114596+0000","last_became_active":"2024-04-16T05:32:39.109792+0000","last_became_peered":"2024-04-16T05:32:39.109792+0000","last_unstale":"2024-04-16T05:34:56.114596+0000","last_undegraded":"2024-04-16T05:34:56.114596+0000","last_fullsized":"2024-04-16T05:34:56.114596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:15:05.925370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264402+0000","last_change":"2024-04-16T05:32:34.751693+0000","last_active":"2024-04-16T05:34:56.264402+0000","last_peered":"2024-04-16T05:34:56.264402+0000","last_clean":"2024-04-16T05:34:56.264402+0000","last_became_active":"2024-04-16T05:32:34.751210+0000","last_became_peered":"2024-04-16T05:32:34.751210+0000","last_unstale":"2024-04-16T05:34:56.264402+0000","last_undegraded":"2024-04-16T05:34:56.264402+0000","last_fullsized":"2024-04-16T05:34:56.264402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:33:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058618+0000","last_change":"2024-04-16T05:32:33.738310+0000","last_active":"2024-04-16T05:34:56.058618+0000","last_peered":"2024-04-16T05:34:56.058618+0000","last_clean":"2024-04-16T05:34:56.058618+0000","last_became_active":"2024-04-16T05:32:33.738012+0000","last_became_peered":"2024-04-16T05:32:33.738012+0000","last_unstale":"2024-04-16T05:34:56.058618+0000","last_undegraded":"2024-04-16T05:34:56.058618+0000","last_fullsized":"2024-04-16T05:34:56.058618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:22:18.311458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639634+0000","last_change":"2024-04-16T05:32:38.933465+0000","last_active":"2024-04-16T05:34:56.639634+0000","last_peered":"2024-04-16T05:34:56.639634+0000","last_clean":"2024-04-16T05:34:56.639634+0000","last_became_active":"2024-04-16T05:32:38.932391+0000","last_became_peered":"2024-04-16T05:32:38.932391+0000","last_unstale":"2024-04-16T05:34:56.639634+0000","last_undegraded":"2024-04-16T05:34:56.639634+0000","last_fullsized":"2024-04-16T05:34:56.639634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:36:39.494556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009019+0000","last_change":"2024-04-16T05:32:33.743241+0000","last_active":"2024-04-16T05:34:56.009019+0000","last_peered":"2024-04-16T05:34:56.009019+0000","last_clean":"2024-04-16T05:34:56.009019+0000","last_became_active":"2024-04-16T05:32:33.742887+0000","last_became_peered":"2024-04-16T05:32:33.742887+0000","last_unstale":"2024-04-16T05:34:56.009019+0000","last_undegraded":"2024-04-16T05:34:56.009019+0000","last_fullsized":"2024-04-16T05:34:56.009019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:24:38.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267151+0000","last_change":"2024-04-16T05:32:34.746228+0000","last_active":"2024-04-16T05:34:56.267151+0000","last_peered":"2024-04-16T05:34:56.267151+0000","last_clean":"2024-04-16T05:34:56.267151+0000","last_became_active":"2024-04-16T05:32:34.745883+0000","last_became_peered":"2024-04-16T05:32:34.745883+0000","last_unstale":"2024-04-16T05:34:56.267151+0000","last_undegraded":"2024-04-16T05:34:56.267151+0000","last_fullsized":"2024-04-16T05:34:56.267151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:33.656358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105107+0000","last_change":"2024-04-16T05:32:39.110156+0000","last_active":"2024-04-16T05:34:56.105107+0000","last_peered":"2024-04-16T05:34:56.105107+0000","last_clean":"2024-04-16T05:34:56.105107+0000","last_became_active":"2024-04-16T05:32:39.108958+0000","last_became_peered":"2024-04-16T05:32:39.108958+0000","last_unstale":"2024-04-16T05:34:56.105107+0000","last_undegraded":"2024-04-16T05:34:56.105107+0000","last_fullsized":"2024-04-16T05:34:56.105107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:27:58.658187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'45","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.229853+0000","last_change":"2024-04-16T05:32:33.738179+0000","last_active":"2024-04-16T05:34:56.229853+0000","last_peered":"2024-04-16T05:34:56.229853+0000","last_clean":"2024-04-16T05:34:56.229853+0000","last_became_active":"2024-04-16T05:32:33.737841+0000","last_became_peered":"2024-04-16T05:32:33.737841+0000","last_unstale":"2024-04-16T05:34:56.229853+0000","last_undegraded":"2024-04-16T05:34:56.229853+0000","last_fullsized":"2024-04-16T05:34:56.229853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:13:11.210559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107138+0000","last_change":"2024-04-16T05:32:34.758142+0000","last_active":"2024-04-16T05:34:56.107138+0000","last_peered":"2024-04-16T05:34:56.107138+0000","last_clean":"2024-04-16T05:34:56.107138+0000","last_became_active":"2024-04-16T05:32:34.757825+0000","last_became_peered":"2024-04-16T05:32:34.757825+0000","last_unstale":"2024-04-16T05:34:56.107138+0000","last_undegraded":"2024-04-16T05:34:56.107138+0000","last_fullsized":"2024-04-16T05:34:56.107138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:00:37.575764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.263978+0000","last_change":"2024-04-16T05:32:38.933327+0000","last_active":"2024-04-16T05:34:56.263978+0000","last_peered":"2024-04-16T05:34:56.263978+0000","last_clean":"2024-04-16T05:34:56.263978+0000","last_became_active":"2024-04-16T05:32:38.932808+0000","last_became_peered":"2024-04-16T05:32:38.932808+0000","last_unstale":"2024-04-16T05:34:56.263978+0000","last_undegraded":"2024-04-16T05:34:56.263978+0000","last_fullsized":"2024-04-16T05:34:56.263978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:30:03.217103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009338+0000","last_change":"2024-04-16T05:32:33.742405+0000","last_active":"2024-04-16T05:34:56.009338+0000","last_peered":"2024-04-16T05:34:56.009338+0000","last_clean":"2024-04-16T05:34:56.009338+0000","last_became_active":"2024-04-16T05:32:33.742051+0000","last_became_peered":"2024-04-16T05:32:33.742051+0000","last_unstale":"2024-04-16T05:34:56.009338+0000","last_undegraded":"2024-04-16T05:34:56.009338+0000","last_fullsized":"2024-04-16T05:34:56.009338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:35:13.914483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267051+0000","last_change":"2024-04-16T05:33:34.772430+0000","last_active":"2024-04-16T05:34:56.267051+0000","last_peered":"2024-04-16T05:34:56.267051+0000","last_clean":"2024-04-16T05:34:56.267051+0000","last_became_active":"2024-04-16T05:33:34.772067+0000","last_became_peered":"2024-04-16T05:33:34.772067+0000","last_unstale":"2024-04-16T05:34:56.267051+0000","last_undegraded":"2024-04-16T05:34:56.267051+0000","last_fullsized":"2024-04-16T05:34:56.267051+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:38:24.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009151+0000","last_change":"2024-04-16T05:33:34.770714+0000","last_active":"2024-04-16T05:34:56.009151+0000","last_peered":"2024-04-16T05:34:56.009151+0000","last_clean":"2024-04-16T05:34:56.009151+0000","last_became_active":"2024-04-16T05:33:34.770249+0000","last_became_peered":"2024-04-16T05:33:34.770249+0000","last_unstale":"2024-04-16T05:34:56.009151+0000","last_undegraded":"2024-04-16T05:34:56.009151+0000","last_fullsized":"2024-04-16T05:34:56.009151+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:22:46.517492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264301+0000","last_change":"2024-04-16T05:32:33.743031+0000","last_active":"2024-04-16T05:34:56.264301+0000","last_peered":"2024-04-16T05:34:56.264301+0000","last_clean":"2024-04-16T05:34:56.264301+0000","last_became_active":"2024-04-16T05:32:33.742771+0000","last_became_peered":"2024-04-16T05:32:33.742771+0000","last_unstale":"2024-04-16T05:34:56.264301+0000","last_undegraded":"2024-04-16T05:34:56.264301+0000","last_fullsized":"2024-04-16T05:34:56.264301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:47:06.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008461+0000","last_change":"2024-04-16T05:32:34.749428+0000","last_active":"2024-04-16T05:34:56.008461+0000","last_peered":"2024-04-16T05:34:56.008461+0000","last_clean":"2024-04-16T05:34:56.008461+0000","last_became_active":"2024-04-16T05:32:34.749087+0000","last_became_peered":"2024-04-16T05:32:34.749087+0000","last_unstale":"2024-04-16T05:34:56.008461+0000","last_undegraded":"2024-04-16T05:34:56.008461+0000","last_fullsized":"2024-04-16T05:34:56.008461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:59:25.124558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.007570+0000","last_change":"2024-04-16T05:32:38.942407+0000","last_active":"2024-04-16T05:34:56.007570+0000","last_peered":"2024-04-16T05:34:56.007570+0000","last_clean":"2024-04-16T05:34:56.007570+0000","last_became_active":"2024-04-16T05:32:38.941731+0000","last_became_peered":"2024-04-16T05:32:38.941731+0000","last_unstale":"2024-04-16T05:34:56.007570+0000","last_undegraded":"2024-04-16T05:34:56.007570+0000","last_fullsized":"2024-04-16T05:34:56.007570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:42:27.700203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105694+0000","last_change":"2024-04-16T05:32:33.741487+0000","last_active":"2024-04-16T05:34:56.105694+0000","last_peered":"2024-04-16T05:34:56.105694+0000","last_clean":"2024-04-16T05:34:56.105694+0000","last_became_active":"2024-04-16T05:32:33.741140+0000","last_became_peered":"2024-04-16T05:32:33.741140+0000","last_unstale":"2024-04-16T05:34:56.105694+0000","last_undegraded":"2024-04-16T05:34:56.105694+0000","last_fullsized":"2024-04-16T05:34:56.105694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:21:36.086164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639473+0000","last_change":"2024-04-16T05:32:34.756608+0000","last_active":"2024-04-16T05:34:56.639473+0000","last_peered":"2024-04-16T05:34:56.639473+0000","last_clean":"2024-04-16T05:34:56.639473+0000","last_became_active":"2024-04-16T05:32:34.756296+0000","last_became_peered":"2024-04-16T05:32:34.756296+0000","last_unstale":"2024-04-16T05:34:56.639473+0000","last_undegraded":"2024-04-16T05:34:56.639473+0000","last_fullsized":"2024-04-16T05:34:56.639473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:13:28.397333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009566+0000","last_change":"2024-04-16T05:32:38.929876+0000","last_active":"2024-04-16T05:34:56.009566+0000","last_peered":"2024-04-16T05:34:56.009566+0000","last_clean":"2024-04-16T05:34:56.009566+0000","last_became_active":"2024-04-16T05:32:38.928626+0000","last_became_peered":"2024-04-16T05:32:38.928626+0000","last_unstale":"2024-04-16T05:34:56.009566+0000","last_undegraded":"2024-04-16T05:34:56.009566+0000","last_fullsized":"2024-04-16T05:34:56.009566+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:08:09.697413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264072+0000","last_change":"2024-04-16T05:32:33.736047+0000","last_active":"2024-04-16T05:34:56.264072+0000","last_peered":"2024-04-16T05:34:56.264072+0000","last_clean":"2024-04-16T05:34:56.264072+0000","last_became_active":"2024-04-16T05:32:33.735683+0000","last_became_peered":"2024-04-16T05:32:33.735683+0000","last_unstale":"2024-04-16T05:34:56.264072+0000","last_undegraded":"2024-04-16T05:34:56.264072+0000","last_fullsized":"2024-04-16T05:34:56.264072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:58:32.969924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.106396+0000","last_change":"2024-04-16T05:32:34.763558+0000","last_active":"2024-04-16T05:34:56.106396+0000","last_peered":"2024-04-16T05:34:56.106396+0000","last_clean":"2024-04-16T05:34:56.106396+0000","last_became_active":"2024-04-16T05:32:34.763190+0000","last_became_peered":"2024-04-16T05:32:34.763190+0000","last_unstale":"2024-04-16T05:34:56.106396+0000","last_undegraded":"2024-04-16T05:34:56.106396+0000","last_fullsized":"2024-04-16T05:34:56.106396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:38:31.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.009179+0000","last_change":"2024-04-16T05:32:39.109426+0000","last_active":"2024-04-16T05:34:56.009179+0000","last_peered":"2024-04-16T05:34:56.009179+0000","last_clean":"2024-04-16T05:34:56.009179+0000","last_became_active":"2024-04-16T05:32:39.108970+0000","last_became_peered":"2024-04-16T05:32:39.108970+0000","last_unstale":"2024-04-16T05:34:56.009179+0000","last_undegraded":"2024-04-16T05:34:56.009179+0000","last_fullsized":"2024-04-16T05:34:56.009179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:54.824654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.008593+0000","last_change":"2024-04-16T05:32:33.734983+0000","last_active":"2024-04-16T05:34:56.008593+0000","last_peered":"2024-04-16T05:34:56.008593+0000","last_clean":"2024-04-16T05:34:56.008593+0000","last_became_active":"2024-04-16T05:32:33.734609+0000","last_became_peered":"2024-04-16T05:32:33.734609+0000","last_unstale":"2024-04-16T05:34:56.008593+0000","last_undegraded":"2024-04-16T05:34:56.008593+0000","last_fullsized":"2024-04-16T05:34:56.008593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:35:59.436499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058954+0000","last_change":"2024-04-16T05:32:34.759811+0000","last_active":"2024-04-16T05:34:56.058954+0000","last_peered":"2024-04-16T05:34:56.058954+0000","last_clean":"2024-04-16T05:34:56.058954+0000","last_became_active":"2024-04-16T05:32:34.759499+0000","last_became_peered":"2024-04-16T05:32:34.759499+0000","last_unstale":"2024-04-16T05:34:56.058954+0000","last_undegraded":"2024-04-16T05:34:56.058954+0000","last_fullsized":"2024-04-16T05:34:56.058954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:46:28.554191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264537+0000","last_change":"2024-04-16T05:32:38.929021+0000","last_active":"2024-04-16T05:34:56.264537+0000","last_peered":"2024-04-16T05:34:56.264537+0000","last_clean":"2024-04-16T05:34:56.264537+0000","last_became_active":"2024-04-16T05:32:38.928574+0000","last_became_peered":"2024-04-16T05:32:38.928574+0000","last_unstale":"2024-04-16T05:34:56.264537+0000","last_undegraded":"2024-04-16T05:34:56.264537+0000","last_fullsized":"2024-04-16T05:34:56.264537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:11:35.693364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264578+0000","last_change":"2024-04-16T05:32:33.747352+0000","last_active":"2024-04-16T05:34:56.264578+0000","last_peered":"2024-04-16T05:34:56.264578+0000","last_clean":"2024-04-16T05:34:56.264578+0000","last_became_active":"2024-04-16T05:32:33.746989+0000","last_became_peered":"2024-04-16T05:32:33.746989+0000","last_unstale":"2024-04-16T05:34:56.264578+0000","last_undegraded":"2024-04-16T05:34:56.264578+0000","last_fullsized":"2024-04-16T05:34:56.264578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T07:06:38.422274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059157+0000","last_change":"2024-04-16T05:32:34.760636+0000","last_active":"2024-04-16T05:34:56.059157+0000","last_peered":"2024-04-16T05:34:56.059157+0000","last_clean":"2024-04-16T05:34:56.059157+0000","last_became_active":"2024-04-16T05:32:34.760335+0000","last_became_peered":"2024-04-16T05:32:34.760335+0000","last_unstale":"2024-04-16T05:34:56.059157+0000","last_undegraded":"2024-04-16T05:34:56.059157+0000","last_fullsized":"2024-04-16T05:34:56.059157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:04:34.322799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266617+0000","last_change":"2024-04-16T05:33:34.769486+0000","last_active":"2024-04-16T05:34:56.266617+0000","last_peered":"2024-04-16T05:34:56.266617+0000","last_clean":"2024-04-16T05:34:56.266617+0000","last_became_active":"2024-04-16T05:33:34.769025+0000","last_became_peered":"2024-04-16T05:33:34.769025+0000","last_unstale":"2024-04-16T05:34:56.266617+0000","last_undegraded":"2024-04-16T05:34:56.266617+0000","last_fullsized":"2024-04-16T05:34:56.266617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:42:10.382817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'39","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.228124+0000","last_change":"2024-04-16T05:32:33.746346+0000","last_active":"2024-04-16T05:34:56.228124+0000","last_peered":"2024-04-16T05:34:56.228124+0000","last_clean":"2024-04-16T05:34:56.228124+0000","last_became_active":"2024-04-16T05:32:33.746030+0000","last_became_peered":"2024-04-16T05:32:33.746030+0000","last_unstale":"2024-04-16T05:34:56.228124+0000","last_undegraded":"2024-04-16T05:34:56.228124+0000","last_fullsized":"2024-04-16T05:34:56.228124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:54:35.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.058863+0000","last_change":"2024-04-16T05:32:34.752283+0000","last_active":"2024-04-16T05:34:56.058863+0000","last_peered":"2024-04-16T05:34:56.058863+0000","last_clean":"2024-04-16T05:34:56.058863+0000","last_became_active":"2024-04-16T05:32:34.751904+0000","last_became_peered":"2024-04-16T05:32:34.751904+0000","last_unstale":"2024-04-16T05:34:56.058863+0000","last_undegraded":"2024-04-16T05:34:56.058863+0000","last_fullsized":"2024-04-16T05:34:56.058863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T06:15:52.688683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114065+0000","last_change":"2024-04-16T05:32:38.934194+0000","last_active":"2024-04-16T05:34:56.114065+0000","last_peered":"2024-04-16T05:34:56.114065+0000","last_clean":"2024-04-16T05:34:56.114065+0000","last_became_active":"2024-04-16T05:32:38.933636+0000","last_became_peered":"2024-04-16T05:32:38.933636+0000","last_unstale":"2024-04-16T05:34:56.114065+0000","last_undegraded":"2024-04-16T05:34:56.114065+0000","last_fullsized":"2024-04-16T05:34:56.114065+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:37.898686+0000","last_clean_scrub_stamp":"2024-04-16T05:32:37.898686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:14:40.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267445+0000","last_change":"2024-04-16T05:32:33.744267+0000","last_active":"2024-04-16T05:34:56.267445+0000","last_peered":"2024-04-16T05:34:56.267445+0000","last_clean":"2024-04-16T05:34:56.267445+0000","last_became_active":"2024-04-16T05:32:33.743931+0000","last_became_peered":"2024-04-16T05:32:33.743931+0000","last_unstale":"2024-04-16T05:34:56.267445+0000","last_undegraded":"2024-04-16T05:34:56.267445+0000","last_fullsized":"2024-04-16T05:34:56.267445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:32.710126+0000","last_clean_scrub_stamp":"2024-04-16T05:32:32.710126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:59:30.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.059158+0000","last_change":"2024-04-16T05:32:34.756833+0000","last_active":"2024-04-16T05:34:56.059158+0000","last_peered":"2024-04-16T05:34:56.059158+0000","last_clean":"2024-04-16T05:34:56.059158+0000","last_became_active":"2024-04-16T05:32:34.756436+0000","last_became_peered":"2024-04-16T05:32:34.756436+0000","last_unstale":"2024-04-16T05:34:56.059158+0000","last_undegraded":"2024-04-16T05:34:56.059158+0000","last_fullsized":"2024-04-16T05:34:56.059158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:20.732946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639538+0000","last_change":"2024-04-16T05:32:34.752744+0000","last_active":"2024-04-16T05:34:56.639538+0000","last_peered":"2024-04-16T05:34:56.639538+0000","last_clean":"2024-04-16T05:34:56.639538+0000","last_became_active":"2024-04-16T05:32:34.752442+0000","last_became_peered":"2024-04-16T05:32:34.752442+0000","last_unstale":"2024-04-16T05:34:56.639538+0000","last_undegraded":"2024-04-16T05:34:56.639538+0000","last_fullsized":"2024-04-16T05:34:56.639538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:01:23.644692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264130+0000","last_change":"2024-04-16T05:32:34.755118+0000","last_active":"2024-04-16T05:34:56.264130+0000","last_peered":"2024-04-16T05:34:56.264130+0000","last_clean":"2024-04-16T05:34:56.264130+0000","last_became_active":"2024-04-16T05:32:34.754824+0000","last_became_peered":"2024-04-16T05:32:34.754824+0000","last_unstale":"2024-04-16T05:34:56.264130+0000","last_undegraded":"2024-04-16T05:34:56.264130+0000","last_fullsized":"2024-04-16T05:34:56.264130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:58:37.860305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264047+0000","last_change":"2024-04-16T05:32:34.756144+0000","last_active":"2024-04-16T05:34:56.264047+0000","last_peered":"2024-04-16T05:34:56.264047+0000","last_clean":"2024-04-16T05:34:56.264047+0000","last_became_active":"2024-04-16T05:32:34.755841+0000","last_became_peered":"2024-04-16T05:32:34.755841+0000","last_unstale":"2024-04-16T05:34:56.264047+0000","last_undegraded":"2024-04-16T05:34:56.264047+0000","last_fullsized":"2024-04-16T05:34:56.264047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:23:00.482444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266708+0000","last_change":"2024-04-16T05:32:34.766204+0000","last_active":"2024-04-16T05:34:56.266708+0000","last_peered":"2024-04-16T05:34:56.266708+0000","last_clean":"2024-04-16T05:34:56.266708+0000","last_became_active":"2024-04-16T05:32:34.765709+0000","last_became_peered":"2024-04-16T05:32:34.765709+0000","last_unstale":"2024-04-16T05:34:56.266708+0000","last_undegraded":"2024-04-16T05:34:56.266708+0000","last_fullsized":"2024-04-16T05:34:56.266708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:23:53.926477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.267022+0000","last_change":"2024-04-16T05:32:34.758513+0000","last_active":"2024-04-16T05:34:56.267022+0000","last_peered":"2024-04-16T05:34:56.267022+0000","last_clean":"2024-04-16T05:34:56.267022+0000","last_became_active":"2024-04-16T05:32:34.758145+0000","last_became_peered":"2024-04-16T05:32:34.758145+0000","last_unstale":"2024-04-16T05:34:56.267022+0000","last_undegraded":"2024-04-16T05:34:56.267022+0000","last_fullsized":"2024-04-16T05:34:56.267022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T10:09:17.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639842+0000","last_change":"2024-04-16T05:32:34.750651+0000","last_active":"2024-04-16T05:34:56.639842+0000","last_peered":"2024-04-16T05:34:56.639842+0000","last_clean":"2024-04-16T05:34:56.639842+0000","last_became_active":"2024-04-16T05:32:34.750348+0000","last_became_peered":"2024-04-16T05:32:34.750348+0000","last_unstale":"2024-04-16T05:34:56.639842+0000","last_undegraded":"2024-04-16T05:34:56.639842+0000","last_fullsized":"2024-04-16T05:34:56.639842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T14:18:29.282449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264574+0000","last_change":"2024-04-16T05:32:34.750079+0000","last_active":"2024-04-16T05:34:56.264574+0000","last_peered":"2024-04-16T05:34:56.264574+0000","last_clean":"2024-04-16T05:34:56.264574+0000","last_became_active":"2024-04-16T05:32:34.749775+0000","last_became_peered":"2024-04-16T05:32:34.749775+0000","last_unstale":"2024-04-16T05:34:56.264574+0000","last_undegraded":"2024-04-16T05:34:56.264574+0000","last_fullsized":"2024-04-16T05:34:56.264574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:31:02.297258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.107010+0000","last_change":"2024-04-16T05:32:34.752528+0000","last_active":"2024-04-16T05:34:56.107010+0000","last_peered":"2024-04-16T05:34:56.107010+0000","last_clean":"2024-04-16T05:34:56.107010+0000","last_became_active":"2024-04-16T05:32:34.752150+0000","last_became_peered":"2024-04-16T05:32:34.752150+0000","last_unstale":"2024-04-16T05:34:56.107010+0000","last_undegraded":"2024-04-16T05:34:56.107010+0000","last_fullsized":"2024-04-16T05:34:56.107010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:41:16.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.114222+0000","last_change":"2024-04-16T05:32:34.758486+0000","last_active":"2024-04-16T05:34:56.114222+0000","last_peered":"2024-04-16T05:34:56.114222+0000","last_clean":"2024-04-16T05:34:56.114222+0000","last_became_active":"2024-04-16T05:32:34.754840+0000","last_became_peered":"2024-04-16T05:32:34.754840+0000","last_unstale":"2024-04-16T05:34:56.114222+0000","last_undegraded":"2024-04-16T05:34:56.114222+0000","last_fullsized":"2024-04-16T05:34:56.114222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T13:26:35.644337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.105877+0000","last_change":"2024-04-16T05:32:34.758933+0000","last_active":"2024-04-16T05:34:56.105877+0000","last_peered":"2024-04-16T05:34:56.105877+0000","last_clean":"2024-04-16T05:34:56.105877+0000","last_became_active":"2024-04-16T05:32:34.758587+0000","last_became_peered":"2024-04-16T05:32:34.758587+0000","last_unstale":"2024-04-16T05:34:56.105877+0000","last_undegraded":"2024-04-16T05:34:56.105877+0000","last_fullsized":"2024-04-16T05:34:56.105877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T12:19:20.674248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.264656+0000","last_change":"2024-04-16T05:32:34.750846+0000","last_active":"2024-04-16T05:34:56.264656+0000","last_peered":"2024-04-16T05:34:56.264656+0000","last_clean":"2024-04-16T05:34:56.264656+0000","last_became_active":"2024-04-16T05:32:34.750529+0000","last_became_peered":"2024-04-16T05:32:34.750529+0000","last_unstale":"2024-04-16T05:34:56.264656+0000","last_undegraded":"2024-04-16T05:34:56.264656+0000","last_fullsized":"2024-04-16T05:34:56.264656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T11:02:32.288459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.266353+0000","last_change":"2024-04-16T05:32:34.752566+0000","last_active":"2024-04-16T05:34:56.266353+0000","last_peered":"2024-04-16T05:34:56.266353+0000","last_clean":"2024-04-16T05:34:56.266353+0000","last_became_active":"2024-04-16T05:32:34.752236+0000","last_became_peered":"2024-04-16T05:32:34.752236+0000","last_unstale":"2024-04-16T05:34:56.266353+0000","last_undegraded":"2024-04-16T05:34:56.266353+0000","last_fullsized":"2024-04-16T05:34:56.266353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T09:43:36.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T05:34:56.639755+0000","last_change":"2024-04-16T05:32:34.750972+0000","last_active":"2024-04-16T05:34:56.639755+0000","last_peered":"2024-04-16T05:34:56.639755+0000","last_clean":"2024-04-16T05:34:56.639755+0000","last_became_active":"2024-04-16T05:32:34.750671+0000","last_became_peered":"2024-04-16T05:32:34.750671+0000","last_unstale":"2024-04-16T05:34:56.639755+0000","last_undegraded":"2024-04-16T05:34:56.639755+0000","last_fullsized":"2024-04-16T05:34:56.639755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T05:32:33.716599+0000","last_clean_scrub_stamp":"2024-04-16T05:32:33.716599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T08:19:45.331816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108342,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597977,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":858,"num_read_kb":541,"num_write":289,"num_write_kb":1442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1793931,"data_compressed":80799,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":510665,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1298518,"data_compressed":67519,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":908179,"data_compressed":42764,"data_compressed_allocated":57344,"data_compressed_original":458752,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":904003,"data_compressed":42764,"data_compressed_allocated":57344,"data_compressed_original":458752,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":1107760,"data_compressed":59732,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":506043,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1073152,"data_stored":574049,"data_compressed":25249,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":507431,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":565698,"data_compressed":22271,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":637821,"data_compressed":30048,"data_compressed_allocated":40960,"data_compressed_original":196608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":511225,"data_compressed":23618,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1101824,"data_stored":1234025,"data_compressed":65888,"data_compressed_allocated":86016,"data_compressed_original":778240,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36114,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68096,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69944,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133608,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":202197,"data_compressed":7787,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T05:34:59.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:34:59.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.641+0000 7ff765715640 1 -- 172.21.15.32:0/3022602568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 msgr2=0x7ff760100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.641+0000 7ff765715640 1 --2- 172.21.15.32:0/3022602568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 0x7ff760100530 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff748009900 tx=0x7ff74802bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 -- 172.21.15.32:0/3022602568 shutdown_connections 2024-04-16T05:34:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 --2- 172.21.15.32:0/3022602568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff760100a70 0x7ff76010a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 --2- 172.21.15.32:0/3022602568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 0x7ff760100530 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 --2- 172.21.15.32:0/3022602568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff760103640 0x7ff760103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 -- 172.21.15.32:0/3022602568 >> 172.21.15.32:0/3022602568 conn(0x7ff7600fbc70 msgr2=0x7ff7600fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 -- 172.21.15.32:0/3022602568 shutdown_connections 2024-04-16T05:34:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 -- 172.21.15.32:0/3022602568 wait complete. 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 Processor -- start 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.642+0000 7ff765715640 1 -- start start 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff765715640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 0x7ff760198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff765715640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff760100a70 0x7ff7601988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff765715640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff760103640 0x7ff76019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff765715640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff76010dd20 con 0x7ff7601000b0 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff765715640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff76010dba0 con 0x7ff760103640 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff765715640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff76010dea0 con 0x7ff760100a70 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 0x7ff760198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 0x7ff760198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35782/0 (socket says 172.21.15.32:35782) 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 -- 172.21.15.32:0/4234703704 learned_addr learned my addr 172.21.15.32:0/4234703704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75f7fe640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff760103640 0x7ff76019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff757fff640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff760100a70 0x7ff7601988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 -- 172.21.15.32:0/4234703704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff760100a70 msgr2=0x7ff7601988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff760100a70 0x7ff7601988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 -- 172.21.15.32:0/4234703704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff760103640 msgr2=0x7ff76019f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff760103640 0x7ff76019f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 -- 172.21.15.32:0/4234703704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76019ffa0 con 0x7ff7601000b0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75f7fe640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff760103640 0x7ff76019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.643+0000 7ff75effd640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 0x7ff760198380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff75000e9c0 tx=0x7ff75000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.644+0000 7ff75cff9640 1 -- 172.21.15.32:0/4234703704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75000cd80 con 0x7ff7601000b0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.644+0000 7ff75cff9640 1 -- 172.21.15.32:0/4234703704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7500035f0 con 0x7ff7601000b0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.644+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7601a0290 con 0x7ff7601000b0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.644+0000 7ff75cff9640 1 -- 172.21.15.32:0/4234703704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750010690 con 0x7ff7601000b0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.644+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7601a0760 con 0x7ff7601000b0 2024-04-16T05:34:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.645+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff760109500 con 0x7ff7601000b0 2024-04-16T05:34:59.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.646+0000 7ff75cff9640 1 -- 172.21.15.32:0/4234703704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7ff7500037a0 con 0x7ff7601000b0 2024-04-16T05:34:59.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.647+0000 7ff75cff9640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff74c0a3d20 0x7ff74c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.647+0000 7ff757fff640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff74c0a3d20 0x7ff74c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.647+0000 7ff75cff9640 1 -- 172.21.15.32:0/4234703704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff7500d0ea0 con 0x7ff7601000b0 2024-04-16T05:34:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.647+0000 7ff757fff640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff74c0a3d20 0x7ff74c0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff74802c7b0 tx=0x7ff748000bc0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.651+0000 7ff75cff9640 1 -- 172.21.15.32:0/4234703704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff75009d200 con 0x7ff7601000b0 2024-04-16T05:34:59.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.849+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff76002d620 con 0x7ff7601000b0 2024-04-16T05:34:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.851+0000 7ff75cff9640 1 -- 172.21.15.32:0/4234703704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24995 (secure 0 0 0) 0x7ff75009cb00 con 0x7ff7601000b0 2024-04-16T05:34:59.849 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:34:59.850 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:34:56.002698+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1356969979":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6800/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/558122612":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2362160984":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1729232468":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff74c0a3d20 msgr2=0x7ff74c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff74c0a3d20 0x7ff74c0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff74802c7b0 tx=0x7ff748000bc0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 msgr2=0x7ff760198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 0x7ff760198380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff75000e9c0 tx=0x7ff75000ee80 comp rx=0 tx=0).stop 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 shutdown_connections 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff760103640 0x7ff76019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff760100a70 0x7ff7601988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff74c0a3d20 0x7ff74c0a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 --2- 172.21.15.32:0/4234703704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7601000b0 0x7ff760198380 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 >> 172.21.15.32:0/4234703704 conn(0x7ff7600fbc70 msgr2=0x7ff7601066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 shutdown_connections 2024-04-16T05:34:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.853+0000 7ff765715640 1 -- 172.21.15.32:0/4234703704 wait complete. 2024-04-16T05:34:59.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:34:59.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T05:34:59.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:59.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.864 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.864 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.864 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.864 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:59.864 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:34:59 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:59.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:34:59.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:34:59.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:34:59.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T05:34:59.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:59.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:59.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:34:59 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:34:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T05:34:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:34:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:34:59.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:34:59.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:34:59 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:34:59.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.993+0000 7f44e26f8640 1 -- 172.21.15.32:0/785317948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc1020d0 msgr2=0x7f44dc102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.993+0000 7f44e26f8640 1 --2- 172.21.15.32:0/785317948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc1020d0 0x7f44dc102550 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009900 tx=0x7f44c402bc30 comp rx=0 tx=0).stop 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.993+0000 7f44e26f8640 1 -- 172.21.15.32:0/785317948 shutdown_connections 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.993+0000 7f44e26f8640 1 --2- 172.21.15.32:0/785317948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44dc102a90 0x7f44dc1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.993+0000 7f44e26f8640 1 --2- 172.21.15.32:0/785317948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc1020d0 0x7f44dc102550 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 --2- 172.21.15.32:0/785317948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44dc100f00 0x7f44dc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 -- 172.21.15.32:0/785317948 >> 172.21.15.32:0/785317948 conn(0x7f44dc0fc4c0 msgr2=0x7f44dc0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 -- 172.21.15.32:0/785317948 shutdown_connections 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 -- 172.21.15.32:0/785317948 wait complete. 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 Processor -- start 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 -- start start 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44dc100f00 0x7f44dc06b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44dc1020d0 0x7f44dc19d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc102a90 0x7f44dc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10cbd0 con 0x7f44dc102a90 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10ca50 con 0x7f44dc100f00 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.994+0000 7f44e26f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10cd50 con 0x7f44dc1020d0 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44e0c95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc102a90 0x7f44dc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44dc100f00 0x7f44dc06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44e0c95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc102a90 0x7f44dc19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35810/0 (socket says 172.21.15.32:35810) 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44e0c95640 1 -- 172.21.15.32:0/1171683918 learned_addr learned my addr 172.21.15.32:0/1171683918 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:34:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44d3fff640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44dc1020d0 0x7f44dc19d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44dbfff640 1 -- 172.21.15.32:0/1171683918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44dc1020d0 msgr2=0x7f44dc19d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44dbfff640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44dc1020d0 0x7f44dc19d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44dbfff640 1 -- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc102a90 msgr2=0x7f44dc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44dbfff640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc102a90 0x7f44dc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44dbfff640 1 -- 172.21.15.32:0/1171683918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc19ffc0 con 0x7f44dc100f00 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44dbfff640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44dc100f00 0x7f44dc06b110 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00e9c0 tx=0x7f44cc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44d9ffb640 1 -- 172.21.15.32:0/1171683918 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00cd80 con 0x7f44dc100f00 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44dc1a0250 con 0x7f44dc100f00 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44d9ffb640 1 -- 172.21.15.32:0/1171683918 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44cc010910 con 0x7f44dc100f00 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44d9ffb640 1 -- 172.21.15.32:0/1171683918 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc018980 con 0x7f44dc100f00 2024-04-16T05:34:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.995+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44dc1a07c0 con 0x7f44dc100f00 2024-04-16T05:34:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.997+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc1083b0 con 0x7f44dc100f00 2024-04-16T05:34:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.998+0000 7f44d9ffb640 1 -- 172.21.15.32:0/1171683918 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f44cc01f020 con 0x7f44dc100f00 2024-04-16T05:34:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.998+0000 7f44d9ffb640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44c80a3d20 0x7f44c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:34:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.998+0000 7f44d3fff640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44c80a3d20 0x7f44c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:34:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.999+0000 7f44d9ffb640 1 -- 172.21.15.32:0/1171683918 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f44cc0d0b90 con 0x7f44dc100f00 2024-04-16T05:34:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:34:59.999+0000 7f44d3fff640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44c80a3d20 0x7f44c80a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44c4006010 tx=0x7f44c4000bc0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.003+0000 7f44d9ffb640 1 -- 172.21.15.32:0/1171683918 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44cc09cef0 con 0x7f44dc100f00 2024-04-16T05:35:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.193+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44dc02d880 con 0x7f44dc100f00 2024-04-16T05:35:00.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.193+0000 7f44d9ffb640 1 -- 172.21.15.32:0/1171683918 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f44cc09c7f0 con 0x7f44dc100f00 2024-04-16T05:35:00.192 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:35:00.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44c80a3d20 msgr2=0x7f44c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44c80a3d20 0x7f44c80a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44c4006010 tx=0x7f44c4000bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44dc100f00 msgr2=0x7f44dc06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44dc100f00 0x7f44dc06b110 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00e9c0 tx=0x7f44cc00ee80 comp rx=0 tx=0).stop 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 shutdown_connections 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44c80a3d20 0x7f44c80a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44dc102a90 0x7f44dc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44dc1020d0 0x7f44dc19d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 --2- 172.21.15.32:0/1171683918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44dc100f00 0x7f44dc06b110 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.195+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 >> 172.21.15.32:0/1171683918 conn(0x7f44dc0fc4c0 msgr2=0x7f44dc105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.196+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 shutdown_connections 2024-04-16T05:35:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.196+0000 7f44e26f8640 1 -- 172.21.15.32:0/1171683918 wait complete. 2024-04-16T05:35:00.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:35:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.316+0000 7fd0af172640 1 -- 172.21.15.32:0/2045147461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8101140 msgr2=0x7fd0a8101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.316+0000 7fd0af172640 1 --2- 172.21.15.32:0/2045147461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8101140 0x7fd0a8101520 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd098009ca0 tx=0x7fd09802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.316+0000 7fd0af172640 1 -- 172.21.15.32:0/2045147461 shutdown_connections 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.316+0000 7fd0af172640 1 --2- 172.21.15.32:0/2045147461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0a8106210 0x7fd0a810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.316+0000 7fd0af172640 1 --2- 172.21.15.32:0/2045147461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a8101af0 0x7fd0a8105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.316+0000 7fd0af172640 1 --2- 172.21.15.32:0/2045147461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8101140 0x7fd0a8101520 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.316+0000 7fd0af172640 1 -- 172.21.15.32:0/2045147461 >> 172.21.15.32:0/2045147461 conn(0x7fd0a8078000 msgr2=0x7fd0a8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.316+0000 7fd0af172640 1 -- 172.21.15.32:0/2045147461 shutdown_connections 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 -- 172.21.15.32:0/2045147461 wait complete. 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 Processor -- start 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 -- start start 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a8101140 0x7fd0a8102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8106210 0x7fd0a8103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0a8103650 0x7fd0a81a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81112c0 con 0x7fd0a8106210 2024-04-16T05:35:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8111140 con 0x7fd0a8103650 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0af172640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8111440 con 0x7fd0a8101140 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8106210 0x7fd0a8103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8106210 0x7fd0a8103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35832/0 (socket says 172.21.15.32:35832) 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0a7fff640 1 -- 172.21.15.32:0/2227031770 learned_addr learned my addr 172.21.15.32:0/2227031770 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.317+0000 7fd0acf0e640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a8101140 0x7fd0a8102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0ad70f640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0a8103650 0x7fd0a81a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a7fff640 1 -- 172.21.15.32:0/2227031770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a8101140 msgr2=0x7fd0a8102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a7fff640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a8101140 0x7fd0a8102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a7fff640 1 -- 172.21.15.32:0/2227031770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0a8103650 msgr2=0x7fd0a81a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a7fff640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0a8103650 0x7fd0a81a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a7fff640 1 -- 172.21.15.32:0/2227031770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a81a71c0 con 0x7fd0a8106210 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0ad70f640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0a8103650 0x7fd0a81a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a7fff640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8106210 0x7fd0a8103110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd09000d740 tx=0x7fd09000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a5ffb640 1 -- 172.21.15.32:0/2227031770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090017070 con 0x7fd0a8106210 2024-04-16T05:35:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a5ffb640 1 -- 172.21.15.32:0/2227031770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09000de50 con 0x7fd0a8106210 2024-04-16T05:35:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81a74b0 con 0x7fd0a8106210 2024-04-16T05:35:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0a5ffb640 1 -- 172.21.15.32:0/2227031770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090004e00 con 0x7fd0a8106210 2024-04-16T05:35:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.318+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81a79b0 con 0x7fd0a8106210 2024-04-16T05:35:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.319+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a8068e20 con 0x7fd0a8106210 2024-04-16T05:35:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.320+0000 7fd0a5ffb640 1 -- 172.21.15.32:0/2227031770 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fd090004fa0 con 0x7fd0a8106210 2024-04-16T05:35:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.321+0000 7fd0a5ffb640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0940a3d20 0x7fd0940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.321+0000 7fd0acf0e640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0940a3d20 0x7fd0940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.321+0000 7fd0a5ffb640 1 -- 172.21.15.32:0/2227031770 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fd0900d0580 con 0x7fd0a8106210 2024-04-16T05:35:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.322+0000 7fd0acf0e640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0940a3d20 0x7fd0940a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd098004af0 tx=0x7fd098005c50 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.326+0000 7fd0a5ffb640 1 -- 172.21.15.32:0/2227031770 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd09009c880 con 0x7fd0a8106210 2024-04-16T05:35:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.553+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd0a810a450 con 0x7fd0a8106210 2024-04-16T05:35:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.553+0000 7fd0a5ffb640 1 -- 172.21.15.32:0/2227031770 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fd09009c180 con 0x7fd0a8106210 2024-04-16T05:35:00.552 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:35:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0940a3d20 msgr2=0x7fd0940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0940a3d20 0x7fd0940a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd098004af0 tx=0x7fd098005c50 comp rx=0 tx=0).stop 2024-04-16T05:35:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8106210 msgr2=0x7fd0a8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8106210 0x7fd0a8103110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd09000d740 tx=0x7fd09000dc00 comp rx=0 tx=0).stop 2024-04-16T05:35:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 shutdown_connections 2024-04-16T05:35:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0a8103650 0x7fd0a81a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0940a3d20 0x7fd0940a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a8106210 0x7fd0a8103110 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 --2- 172.21.15.32:0/2227031770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a8101140 0x7fd0a8102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 >> 172.21.15.32:0/2227031770 conn(0x7fd0a8078000 msgr2=0x7fd0a8069570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.555+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 shutdown_connections 2024-04-16T05:35:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.556+0000 7fd0af172640 1 -- 172.21.15.32:0/2227031770 wait complete. 2024-04-16T05:35:00.566 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:35:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 -- 172.21.15.32:0/3014411565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0102240 msgr2=0x7f47f0102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 --2- 172.21.15.32:0/3014411565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0102240 0x7f47f0102620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009900 tx=0x7f47e002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 -- 172.21.15.32:0/3014411565 shutdown_connections 2024-04-16T05:35:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 --2- 172.21.15.32:0/3014411565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f0107310 0x7f47f010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 --2- 172.21.15.32:0/3014411565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f0102bf0 0x7f47f0106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 --2- 172.21.15.32:0/3014411565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0102240 0x7f47f0102620 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 -- 172.21.15.32:0/3014411565 >> 172.21.15.32:0/3014411565 conn(0x7f47f00fda40 msgr2=0x7f47f00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 -- 172.21.15.32:0/3014411565 shutdown_connections 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.686+0000 7f47f5309640 1 -- 172.21.15.32:0/3014411565 wait complete. 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47f5309640 1 Processor -- start 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47f5309640 1 -- start start 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47f5309640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f0102240 0x7f47f019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47f5309640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f0107310 0x7f47f019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47f5309640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0198760 0x7f47f0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47f5309640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01123c0 con 0x7f47f0198760 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47f5309640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0112240 con 0x7f47f0107310 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47f5309640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0112540 con 0x7f47f0102240 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47ef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0198760 0x7f47f0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47ef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0198760 0x7f47f0198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35856/0 (socket says 172.21.15.32:35856) 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47ef7fe640 1 -- 172.21.15.32:0/3196745886 learned_addr learned my addr 172.21.15.32:0/3196745886 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47eeffd640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f0102240 0x7f47f019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.687+0000 7f47e67fc640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f0107310 0x7f47f019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ef7fe640 1 -- 172.21.15.32:0/3196745886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f0102240 msgr2=0x7f47f019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ef7fe640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f0102240 0x7f47f019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ef7fe640 1 -- 172.21.15.32:0/3196745886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f0107310 msgr2=0x7f47f019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ef7fe640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f0107310 0x7f47f019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ef7fe640 1 -- 172.21.15.32:0/3196745886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f0199360 con 0x7f47f0198760 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ef7fe640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0198760 0x7f47f0198be0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f47e800eaa0 tx=0x7f47e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ecff9640 1 -- 172.21.15.32:0/3196745886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e800cdb0 con 0x7f47f0198760 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f01a5440 con 0x7f47f0198760 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ecff9640 1 -- 172.21.15.32:0/3196745886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8005020 con 0x7f47f0198760 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47ecff9640 1 -- 172.21.15.32:0/3196745886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e80116b0 con 0x7f47f0198760 2024-04-16T05:35:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.688+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01a5840 con 0x7f47f0198760 2024-04-16T05:35:00.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.689+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f0103710 con 0x7f47f0198760 2024-04-16T05:35:00.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.690+0000 7f47ecff9640 1 -- 172.21.15.32:0/3196745886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f47e8005560 con 0x7f47f0198760 2024-04-16T05:35:00.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.691+0000 7f47ecff9640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f47dc0a3db0 0x7f47dc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:00.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.691+0000 7f47eeffd640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f47dc0a3db0 0x7f47dc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:00.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.691+0000 7f47ecff9640 1 -- 172.21.15.32:0/3196745886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f47e80d0e30 con 0x7f47f0198760 2024-04-16T05:35:00.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.691+0000 7f47eeffd640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f47dc0a3db0 0x7f47dc0a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47e0006010 tx=0x7f47e0005dd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:00.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.696+0000 7f47ecff9640 1 -- 172.21.15.32:0/3196745886 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47e8016030 con 0x7f47f0198760 2024-04-16T05:35:00.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.885+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f47f010a570 con 0x7f47f0198760 2024-04-16T05:35:00.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.886+0000 7f47ecff9640 1 -- 172.21.15.32:0/3196745886 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f47e809c9d0 con 0x7f47f0198760 2024-04-16T05:35:00.885 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:35:00.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:00.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi134 to 174.1M 2024-04-16T05:35:00.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Unable to set osd_memory_target on smithi134 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:35:00.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi082 to 174.1M 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Unable to set osd_memory_target on smithi082 to 182610534: error parsing value: Value '182610534' is below minimum 939524096 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Unable to set osd_memory_target on smithi032 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Updating smithi032:/etc/ceph/ceph.conf 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4234703704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:00.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1171683918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:00.887 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2227031770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f47dc0a3db0 msgr2=0x7f47dc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f47dc0a3db0 0x7f47dc0a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47e0006010 tx=0x7f47e0005dd0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0198760 msgr2=0x7f47f0198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0198760 0x7f47f0198be0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f47e800eaa0 tx=0x7f47e800ef60 comp rx=0 tx=0).stop 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 shutdown_connections 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f47dc0a3db0 0x7f47dc0a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47f0198760 0x7f47f0198be0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f0107310 0x7f47f019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 --2- 172.21.15.32:0/3196745886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47f0102240 0x7f47f019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 >> 172.21.15.32:0/3196745886 conn(0x7f47f00fda40 msgr2=0x7f47f00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:00.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 shutdown_connections 2024-04-16T05:35:00.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:00.888+0000 7f47f5309640 1 -- 172.21.15.32:0/3196745886 wait complete. 2024-04-16T05:35:00.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:00.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi134 to 174.1M 2024-04-16T05:35:00.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Unable to set osd_memory_target on smithi134 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:35:00.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi082 to 174.1M 2024-04-16T05:35:00.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Unable to set osd_memory_target on smithi082 to 182610534: error parsing value: Value '182610534' is below minimum 939524096 2024-04-16T05:35:00.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:00.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-04-16T05:35:00.897 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Unable to set osd_memory_target on smithi032 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Updating smithi032:/etc/ceph/ceph.conf 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4234703704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1171683918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:00.898 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2227031770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:00.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:35:00.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi134 to 174.1M 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Unable to set osd_memory_target on smithi134 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi082 to 174.1M 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Unable to set osd_memory_target on smithi082 to 182610534: error parsing value: Value '182610534' is below minimum 939524096 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Unable to set osd_memory_target on smithi032 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Updating smithi032:/etc/ceph/ceph.conf 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-16T05:35:00.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.conf 2024-04-16T05:35:00.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4234703704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:00.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1171683918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:35:00.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2227031770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 -- 172.21.15.32:0/3757042848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 msgr2=0x7fa90810eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 --2- 172.21.15.32:0/3757042848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 0x7fa90810eee0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa900009ca0 tx=0x7fa90002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 -- 172.21.15.32:0/3757042848 shutdown_connections 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 --2- 172.21.15.32:0/3757042848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 0x7fa90810eee0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 --2- 172.21.15.32:0/3757042848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa908102bf0 0x7fa908106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 --2- 172.21.15.32:0/3757042848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa908102240 0x7fa908102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 -- 172.21.15.32:0/3757042848 >> 172.21.15.32:0/3757042848 conn(0x7fa9080fda40 msgr2=0x7fa9080ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 -- 172.21.15.32:0/3757042848 shutdown_connections 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.012+0000 7fa90cafb640 1 -- 172.21.15.32:0/3757042848 wait complete. 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90cafb640 1 Processor -- start 2024-04-16T05:35:01.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90cafb640 1 -- start start 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90cafb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa908102240 0x7fa90819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90cafb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 0x7fa90819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90cafb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa908198630 0x7fa908198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90cafb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9081123d0 con 0x7fa908107310 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90cafb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa908112250 con 0x7fa908102240 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90cafb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa908112550 con 0x7fa908198630 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 0x7fa90819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 0x7fa90819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35866/0 (socket says 172.21.15.32:35866) 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 -- 172.21.15.32:0/707607503 learned_addr learned my addr 172.21.15.32:0/707607503 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa906d9d640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa908198630 0x7fa908198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90659c640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa908102240 0x7fa90819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 -- 172.21.15.32:0/707607503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa908198630 msgr2=0x7fa908198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa908198630 0x7fa908198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 -- 172.21.15.32:0/707607503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa908102240 msgr2=0x7fa90819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa908102240 0x7fa90819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 -- 172.21.15.32:0/707607503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa908199150 con 0x7fa908107310 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa90659c640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa908102240 0x7fa90819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.013+0000 7fa8fffff640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 0x7fa90819e970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000c970 tx=0x7fa8f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.014+0000 7fa8ff7fe640 1 -- 172.21.15.32:0/707607503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0014070 con 0x7fa908107310 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.014+0000 7fa8ff7fe640 1 -- 172.21.15.32:0/707607503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f0004440 con 0x7fa908107310 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.014+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa908199440 con 0x7fa908107310 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.014+0000 7fa8ff7fe640 1 -- 172.21.15.32:0/707607503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000f450 con 0x7fa908107310 2024-04-16T05:35:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.014+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9081a5640 con 0x7fa908107310 2024-04-16T05:35:01.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.015+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa908069270 con 0x7fa908107310 2024-04-16T05:35:01.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.016+0000 7fa8ff7fe640 1 -- 172.21.15.32:0/707607503 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa8f0017070 con 0x7fa908107310 2024-04-16T05:35:01.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.017+0000 7fa8ff7fe640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8f40a3db0 0x7fa8f40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.017+0000 7fa90659c640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8f40a3db0 0x7fa8f40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.017+0000 7fa8ff7fe640 1 -- 172.21.15.32:0/707607503 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fa8f00d0a90 con 0x7fa908107310 2024-04-16T05:35:01.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.017+0000 7fa90659c640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8f40a3db0 0x7fa8f40a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009c20 tx=0x7fa8f8005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:01.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.021+0000 7fa8ff7fe640 1 -- 172.21.15.32:0/707607503 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8f001e020 con 0x7fa908107310 2024-04-16T05:35:01.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.207+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa908102d40 con 0x7fa908107310 2024-04-16T05:35:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.207+0000 7fa8ff7fe640 1 -- 172.21.15.32:0/707607503 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fa8f009cc80 con 0x7fa908107310 2024-04-16T05:35:01.206 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:35:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8f40a3db0 msgr2=0x7fa8f40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8f40a3db0 0x7fa8f40a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009c20 tx=0x7fa8f8005f90 comp rx=0 tx=0).stop 2024-04-16T05:35:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 msgr2=0x7fa90819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 0x7fa90819e970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000c970 tx=0x7fa8f000ce30 comp rx=0 tx=0).stop 2024-04-16T05:35:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 shutdown_connections 2024-04-16T05:35:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa908198630 0x7fa908198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8f40a3db0 0x7fa8f40a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa908107310 0x7fa90819e970 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 --2- 172.21.15.32:0/707607503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa908102240 0x7fa90819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 >> 172.21.15.32:0/707607503 conn(0x7fa9080fda40 msgr2=0x7fa9080fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 shutdown_connections 2024-04-16T05:35:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.209+0000 7fa90cafb640 1 -- 172.21.15.32:0/707607503 wait complete. 2024-04-16T05:35:01.219 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:35:01.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 -- 172.21.15.32:0/228277370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420106210 msgr2=0x7f242010dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 --2- 172.21.15.32:0/228277370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420106210 0x7f242010dde0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2418009ca0 tx=0x7f241802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 -- 172.21.15.32:0/228277370 shutdown_connections 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 --2- 172.21.15.32:0/228277370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420106210 0x7f242010dde0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 --2- 172.21.15.32:0/228277370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2420101af0 0x7f2420105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 --2- 172.21.15.32:0/228277370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2420101140 0x7f2420101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 -- 172.21.15.32:0/228277370 >> 172.21.15.32:0/228277370 conn(0x7f2420077280 msgr2=0x7f2420077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 -- 172.21.15.32:0/228277370 shutdown_connections 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.326+0000 7f2428614640 1 -- 172.21.15.32:0/228277370 wait complete. 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f2428614640 1 Processor -- start 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f2428614640 1 -- start start 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f2428614640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420101140 0x7f242019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f2428614640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2420106210 0x7f242019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f2428614640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24201986e0 0x7f2420198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f2428614640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24201123e0 con 0x7f2420101140 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f2428614640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2420112260 con 0x7f24201986e0 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f2428614640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2420112560 con 0x7f2420106210 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f24263b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420101140 0x7f242019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f24263b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420101140 0x7f242019e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35890/0 (socket says 172.21.15.32:35890) 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.327+0000 7f24263b0640 1 -- 172.21.15.32:0/3978517360 learned_addr learned my addr 172.21.15.32:0/3978517360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f2426bb1640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24201986e0 0x7f2420198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f2425baf640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2420106210 0x7f242019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24263b0640 1 -- 172.21.15.32:0/3978517360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2420106210 msgr2=0x7f242019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24263b0640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2420106210 0x7f242019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24263b0640 1 -- 172.21.15.32:0/3978517360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24201986e0 msgr2=0x7f2420198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24263b0640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24201986e0 0x7f2420198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24263b0640 1 -- 172.21.15.32:0/3978517360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24201992e0 con 0x7f2420101140 2024-04-16T05:35:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f2426bb1640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24201986e0 0x7f2420198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f2425baf640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2420106210 0x7f242019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24263b0640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420101140 0x7f242019e5d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f241c00d950 tx=0x7f241c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24177fe640 1 -- 172.21.15.32:0/3978517360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c017070 con 0x7f2420101140 2024-04-16T05:35:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24201a5370 con 0x7f2420101140 2024-04-16T05:35:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24177fe640 1 -- 172.21.15.32:0/3978517360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241c009c30 con 0x7f2420101140 2024-04-16T05:35:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f24177fe640 1 -- 172.21.15.32:0/3978517360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c00f6b0 con 0x7f2420101140 2024-04-16T05:35:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.328+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24201a5770 con 0x7f2420101140 2024-04-16T05:35:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.329+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2420069eb0 con 0x7f2420101140 2024-04-16T05:35:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.330+0000 7f24177fe640 1 -- 172.21.15.32:0/3978517360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f241c0109d0 con 0x7f2420101140 2024-04-16T05:35:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.331+0000 7f24177fe640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f240c0a3db0 0x7f240c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.331+0000 7f2425baf640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f240c0a3db0 0x7f240c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.331+0000 7f24177fe640 1 -- 172.21.15.32:0/3978517360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f241c0d09d0 con 0x7f2420101140 2024-04-16T05:35:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.332+0000 7f2425baf640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f240c0a3db0 0x7f240c0a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2420199ee0 tx=0x7f2410005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:01.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.335+0000 7f24177fe640 1 -- 172.21.15.32:0/3978517360 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f241c009de0 con 0x7f2420101140 2024-04-16T05:35:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.560+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2420069270 con 0x7f2420101140 2024-04-16T05:35:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.561+0000 7f24177fe640 1 -- 172.21.15.32:0/3978517360 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f2420069270 con 0x7f2420101140 2024-04-16T05:35:01.561 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:35:01.561 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f240c0a3db0 msgr2=0x7f240c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f240c0a3db0 0x7f240c0a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2420199ee0 tx=0x7f2410005f90 comp rx=0 tx=0).stop 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420101140 msgr2=0x7f242019e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420101140 0x7f242019e5d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f241c00d950 tx=0x7f241c00de10 comp rx=0 tx=0).stop 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 shutdown_connections 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24201986e0 0x7f2420198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2420106210 0x7f242019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f240c0a3db0 0x7f240c0a6270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 --2- 172.21.15.32:0/3978517360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2420101140 0x7f242019e5d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 >> 172.21.15.32:0/3978517360 conn(0x7f2420077280 msgr2=0x7f24200fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 shutdown_connections 2024-04-16T05:35:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.563+0000 7f2428614640 1 -- 172.21.15.32:0/3978517360 wait complete. 2024-04-16T05:35:01.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 -- 172.21.15.32:0/577015895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8107330 msgr2=0x7f99e810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 --2- 172.21.15.32:0/577015895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8107330 0x7f99e810ef00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009ca0 tx=0x7f99e002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 -- 172.21.15.32:0/577015895 shutdown_connections 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 --2- 172.21.15.32:0/577015895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8107330 0x7f99e810ef00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 --2- 172.21.15.32:0/577015895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99e8102ca0 0x7f99e8106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 --2- 172.21.15.32:0/577015895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99e81022f0 0x7f99e81026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 -- 172.21.15.32:0/577015895 >> 172.21.15.32:0/577015895 conn(0x7f99e8079190 msgr2=0x7f99e80795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 -- 172.21.15.32:0/577015895 shutdown_connections 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.679+0000 7f99eed67640 1 -- 172.21.15.32:0/577015895 wait complete. 2024-04-16T05:35:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99eed67640 1 Processor -- start 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99eed67640 1 -- start start 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99eed67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99e81022f0 0x7f99e819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99eed67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99e8107330 0x7f99e819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99eed67640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8198680 0x7f99e8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99eed67640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99e81123e0 con 0x7f99e8198680 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99eed67640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8112260 con 0x7f99e8107330 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99eed67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8112560 con 0x7f99e81022f0 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ed304640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8198680 0x7f99e8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ed304640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8198680 0x7f99e8198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35906/0 (socket says 172.21.15.32:35906) 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ed304640 1 -- 172.21.15.32:0/2069944902 learned_addr learned my addr 172.21.15.32:0/2069944902 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ecb03640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99e81022f0 0x7f99e819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99e7fff640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99e8107330 0x7f99e819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ed304640 1 -- 172.21.15.32:0/2069944902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99e81022f0 msgr2=0x7f99e819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ed304640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99e81022f0 0x7f99e819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ed304640 1 -- 172.21.15.32:0/2069944902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99e8107330 msgr2=0x7f99e819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ed304640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99e8107330 0x7f99e819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ed304640 1 -- 172.21.15.32:0/2069944902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e8199280 con 0x7f99e8198680 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99e7fff640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99e8107330 0x7f99e819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.680+0000 7f99ecb03640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99e81022f0 0x7f99e819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.681+0000 7f99ed304640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8198680 0x7f99e8198b00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f99e002c700 tx=0x7f99e0035b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.681+0000 7f99e5ffb640 1 -- 172.21.15.32:0/2069944902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0039040 con 0x7f99e8198680 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.681+0000 7f99e5ffb640 1 -- 172.21.15.32:0/2069944902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e0044070 con 0x7f99e8198680 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.681+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e8199510 con 0x7f99e8198680 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.681+0000 7f99e5ffb640 1 -- 172.21.15.32:0/2069944902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e00043a0 con 0x7f99e8198680 2024-04-16T05:35:01.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.681+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e81a5600 con 0x7f99e8198680 2024-04-16T05:35:01.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.682+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e8103730 con 0x7f99e8198680 2024-04-16T05:35:01.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.683+0000 7f99e5ffb640 1 -- 172.21.15.32:0/2069944902 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f99e0004540 con 0x7f99e8198680 2024-04-16T05:35:01.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.684+0000 7f99e5ffb640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f99d80a3db0 0x7f99d80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:01.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.684+0000 7f99ecb03640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f99d80a3db0 0x7f99d80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:01.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.684+0000 7f99e5ffb640 1 -- 172.21.15.32:0/2069944902 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f99e00f38b0 con 0x7f99e8198680 2024-04-16T05:35:01.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.684+0000 7f99ecb03640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f99d80a3db0 0x7f99d80a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0058c0 tx=0x7f99dc005d30 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.687+0000 7f99e5ffb640 1 -- 172.21.15.32:0/2069944902 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99e00bfb50 con 0x7f99e8198680 2024-04-16T05:35:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.873+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99e8102ca0 con 0x7f99e8198680 2024-04-16T05:35:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.875+0000 7f99e5ffb640 1 -- 172.21.15.32:0/2069944902 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24995 (secure 0 0 0) 0x7f99e00bf450 con 0x7f99e8198680 2024-04-16T05:35:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:35:01.874 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:34:56.002698+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1356969979":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6800/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/558122612":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2362160984":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1729232468":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:35:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f99d80a3db0 msgr2=0x7f99d80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f99d80a3db0 0x7f99d80a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0058c0 tx=0x7f99dc005d30 comp rx=0 tx=0).stop 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8198680 msgr2=0x7f99e8198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8198680 0x7f99e8198b00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f99e002c700 tx=0x7f99e0035b60 comp rx=0 tx=0).stop 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 shutdown_connections 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f99d80a3db0 0x7f99d80a6270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e8198680 0x7f99e8198b00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99e8107330 0x7f99e819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 --2- 172.21.15.32:0/2069944902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99e81022f0 0x7f99e819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 >> 172.21.15.32:0/2069944902 conn(0x7f99e8079190 msgr2=0x7f99e81000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 shutdown_connections 2024-04-16T05:35:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:01.877+0000 7f99eed67640 1 -- 172.21.15.32:0/2069944902 wait complete. 2024-04-16T05:35:01.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:01.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:01.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3196745886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:01.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/707607503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:01.886 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3978517360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:01.886 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:01.886 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.886 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.886 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.886 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3196745886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:01.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/707607503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:01.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3978517360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:01.889 DEBUG:tasks.fwd_scrub:Yielding 2024-04-16T05:35:01.889 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T05:35:01.898 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T05:35:01.898 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:35:01.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-16T05:35:01.899 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-16T05:35:01.899 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:35:01.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:01.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:35:01.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:01.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: Updating smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: Updating smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: Updating smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:35:01.901 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3196745886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:35:01.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/707607503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:35:01.902 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3978517360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 -- 172.21.15.32:0/1509745951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b94103bf0 msgr2=0x7f7b9410c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/1509745951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b94103bf0 0x7f7b9410c0d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009900 tx=0x7f7b8002bcd0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 -- 172.21.15.32:0/1509745951 shutdown_connections 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/1509745951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b9410c610 0x7f7b9410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/1509745951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b94103bf0 0x7f7b9410c0d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/1509745951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b941032d0 0x7f7b941036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 -- 172.21.15.32:0/1509745951 >> 172.21.15.32:0/1509745951 conn(0x7f7b94078080 msgr2=0x7f7b94078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 -- 172.21.15.32:0/1509745951 shutdown_connections 2024-04-16T05:35:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 -- 172.21.15.32:0/1509745951 wait complete. 2024-04-16T05:35:02.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 -- 172.21.15.32:0/406323848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102ac0 msgr2=0x7fca30106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 --2- 172.21.15.32:0/406323848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102ac0 0x7fca30106b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fca24009860 tx=0x7fca2402bb70 comp rx=0 tx=0).stop 2024-04-16T05:35:02.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 -- 172.21.15.32:0/406323848 shutdown_connections 2024-04-16T05:35:02.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 --2- 172.21.15.32:0/406323848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca301071e0 0x7fca3010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 --2- 172.21.15.32:0/406323848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102ac0 0x7fca30106b00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 --2- 172.21.15.32:0/406323848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30102110 0x7fca301024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 -- 172.21.15.32:0/406323848 >> 172.21.15.32:0/406323848 conn(0x7fca300fd9d0 msgr2=0x7fca300ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 -- 172.21.15.32:0/406323848 shutdown_connections 2024-04-16T05:35:02.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7fca2f59e640 1 -- 172.21.15.32:0/406323848 wait complete. 2024-04-16T05:35:02.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.010+0000 7f7b9ab31640 1 Processor -- start 2024-04-16T05:35:02.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b9ab31640 1 -- start start 2024-04-16T05:35:02.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b9ab31640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b941032d0 0x7f7b9419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b9ab31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b94103bf0 0x7f7b9419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b9ab31640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9410c610 0x7f7b941987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b9ab31640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941123b0 con 0x7f7b9410c610 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b9ab31640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94112230 con 0x7f7b941032d0 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b9ab31640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94112530 con 0x7f7b94103bf0 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b990ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9410c610 0x7f7b941987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2f59e640 1 Processor -- start 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b990ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9410c610 0x7f7b941987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35920/0 (socket says 172.21.15.32:35920) 2024-04-16T05:35:02.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b990ce640 1 -- 172.21.15.32:0/3820604353 learned_addr learned my addr 172.21.15.32:0/3820604353 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2f59e640 1 -- start start 2024-04-16T05:35:02.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b988cd640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b941032d0 0x7f7b9419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b93fff640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b94103bf0 0x7f7b9419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b990ce640 1 -- 172.21.15.32:0/3820604353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b94103bf0 msgr2=0x7f7b9419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b990ce640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b94103bf0 0x7f7b9419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b990ce640 1 -- 172.21.15.32:0/3820604353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b941032d0 msgr2=0x7f7b9419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b990ce640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b941032d0 0x7f7b9419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2f59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102110 0x7fca30074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b990ce640 1 -- 172.21.15.32:0/3820604353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b94199070 con 0x7f7b9410c610 2024-04-16T05:35:02.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7f7b93fff640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b94103bf0 0x7f7b9419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:02.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca301071e0 0x7fca30074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca30074d80 0x7fca301a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2f59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca30112250 con 0x7fca30102110 2024-04-16T05:35:02.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2f59e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca301120d0 con 0x7fca301071e0 2024-04-16T05:35:02.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2f59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca301123d0 con 0x7fca30074d80 2024-04-16T05:35:02.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102110 0x7fca30074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102110 0x7fca30074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35930/0 (socket says 172.21.15.32:35930) 2024-04-16T05:35:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2e59c640 1 -- 172.21.15.32:0/559841521 learned_addr learned my addr 172.21.15.32:0/559841521 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca30074d80 0x7fca301a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.011+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca301071e0 0x7fca30074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7f7b990ce640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9410c610 0x7f7b941987b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009fd0 tx=0x7f7b8c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7fca2e59c640 1 -- 172.21.15.32:0/559841521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca30074d80 msgr2=0x7fca301a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7fca2e59c640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca30074d80 0x7fca301a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7fca2e59c640 1 -- 172.21.15.32:0/559841521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca301071e0 msgr2=0x7fca30074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7f7b91ffb640 1 -- 172.21.15.32:0/3820604353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c017070 con 0x7f7b9410c610 2024-04-16T05:35:02.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7fca2e59c640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca301071e0 0x7fca30074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7fca2e59c640 1 -- 172.21.15.32:0/559841521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca301a7060 con 0x7fca30102110 2024-04-16T05:35:02.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7f7b91ffb640 1 -- 172.21.15.32:0/3820604353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c003e50 con 0x7f7b9410c610 2024-04-16T05:35:02.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b94199360 con 0x7f7b9410c610 2024-04-16T05:35:02.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7f7b91ffb640 1 -- 172.21.15.32:0/3820604353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0106c0 con 0x7f7b9410c610 2024-04-16T05:35:02.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7fca2e59c640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102110 0x7fca30074300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fca2000e9c0 tx=0x7fca2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.012+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b941a5620 con 0x7f7b9410c610 2024-04-16T05:35:02.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.013+0000 7fca1f7fe640 1 -- 172.21.15.32:0/559841521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2000ccc0 con 0x7fca30102110 2024-04-16T05:35:02.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.013+0000 7fca1f7fe640 1 -- 172.21.15.32:0/559841521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20005500 con 0x7fca30102110 2024-04-16T05:35:02.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.013+0000 7fca2f59e640 1 -- 172.21.15.32:0/559841521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca301a7350 con 0x7fca30102110 2024-04-16T05:35:02.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.013+0000 7fca1f7fe640 1 -- 172.21.15.32:0/559841521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20010430 con 0x7fca30102110 2024-04-16T05:35:02.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.013+0000 7fca2f59e640 1 -- 172.21.15.32:0/559841521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca301a7870 con 0x7fca30102110 2024-04-16T05:35:02.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.013+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b94069e70 con 0x7f7b9410c610 2024-04-16T05:35:02.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.015+0000 7f7b91ffb640 1 -- 172.21.15.32:0/3820604353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7b8c003970 con 0x7f7b9410c610 2024-04-16T05:35:02.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.015+0000 7fca1f7fe640 1 -- 172.21.15.32:0/559841521 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fca200105d0 con 0x7fca30102110 2024-04-16T05:35:02.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca377df740 1 Processor -- start 2024-04-16T05:35:02.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7f7b91ffb640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7b840a3ee0 0x7f7b840a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca377df740 1 -- start start 2024-04-16T05:35:02.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7f7b988cd640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7b840a3ee0 0x7f7b840a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7f7b91ffb640 1 -- 172.21.15.32:0/3820604353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7b8c0d0980 con 0x7f7b9410c610 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca377df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca377df740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca377df740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca377df740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55feca436e90 con 0x55feca4d25e0 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca377df740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55feca3d44c0 con 0x55feca4db1c0 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca377df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55feca432e50 con 0x55feca4d1cc0 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca2e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca2e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4dac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35940/0 (socket says 172.21.15.32:35940) 2024-04-16T05:35:02.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca2e59c640 1 -- 172.21.15.32:0/398133821 learned_addr learned my addr 172.21.15.32:0/398133821 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7f7b988cd640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7b840a3ee0 0x7f7b840a63a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b8800b390 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.016+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca2e59c640 1 -- 172.21.15.32:0/398133821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 msgr2=0x55feca4d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca2e59c640 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca2e59c640 1 -- 172.21.15.32:0/398133821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 msgr2=0x55feca4dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca2e59c640 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca1f7fe640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca180a3db0 0x7fca180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca2e59c640 1 -- 172.21.15.32:0/398133821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feca2117e0 con 0x55feca4d25e0 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca180a3db0 0x7fca180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca1f7fe640 1 -- 172.21.15.32:0/559841521 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca200d0ff0 con 0x7fca30102110 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca2e59c640 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4dac80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fca200ce2e0 tx=0x7fca20098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11ee927444202ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.017+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca180a3db0 0x7fca180a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca28005fd0 tx=0x7fca28009500 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.021+0000 7fca1cff9640 1 -- 172.21.15.32:0/398133821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2009c8b0 con 0x55feca4d25e0 2024-04-16T05:35:02.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.021+0000 7f7b91ffb640 1 -- 172.21.15.32:0/3820604353 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b8c09cb70 con 0x7f7b9410c610 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.021+0000 7fca1cff9640 1 -- 172.21.15.32:0/398133821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2009d1e0 con 0x55feca4d25e0 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.021+0000 7fca1cff9640 1 -- 172.21.15.32:0/398133821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2009c440 con 0x55feca4d25e0 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 -- 172.21.15.32:0/398133821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 msgr2=0x55feca4dac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4dac80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fca200ce2e0 tx=0x7fca20098d80 comp rx=0 tx=0).stop 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 -- 172.21.15.32:0/398133821 shutdown_connections 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4dac80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 --2- 172.21.15.32:0/398133821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 -- 172.21.15.32:0/398133821 >> 172.21.15.32:0/398133821 conn(0x55feca4cc3f0 msgr2=0x55feca4cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 -- 172.21.15.32:0/398133821 shutdown_connections 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 -- 172.21.15.32:0/398133821 wait complete. 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 -- start start 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 Processor -- start 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4d82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.022+0000 7fca377df740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55feca4ddef0 con 0x55feca4d25e0 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca377df740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55feca3bf410 con 0x55feca4db1c0 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca377df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55feca4de070 con 0x55feca4d1cc0 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca377df740 10 client.0 _ll_register_callbacks cb 0x55feca3bab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4d87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35950/0 (socket says 172.21.15.32:35950) 2024-04-16T05:35:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 -- 172.21.15.32:0/3253373680 learned_addr learned my addr 172.21.15.32:0/3253373680 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2e59c640 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4d82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 -- 172.21.15.32:0/3253373680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 msgr2=0x55feca4d82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4d82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 -- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 msgr2=0x55feca4d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 -- 172.21.15.32:0/3253373680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feca271f70 con 0x55feca4d25e0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4d87f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fca24005e00 tx=0x7fca24009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a45d46df3e46075 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca15ffb640 1 -- 172.21.15.32:0/3253373680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24043070 con 0x55feca4d25e0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca15ffb640 1 -- 172.21.15.32:0/3253373680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca24004450 con 0x55feca4d25e0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.023+0000 7fca377df740 1 -- 172.21.15.32:0/3253373680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2009d1e0 con 0x55feca4d25e0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca15ffb640 1 -- 172.21.15.32:0/3253373680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca2404f050 con 0x55feca4d25e0 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca377df740 10 client.15639 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca377df740 1 -- 172.21.15.32:0/3253373680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55feca109170 con 0x55feca4d25e0 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca15ffb640 1 -- 172.21.15.32:0/3253373680 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2009d1e0 con 0x55feca4d25e0 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca15ffb640 1 -- 172.21.15.32:0/3253373680 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca24032640 con 0x55feca4d25e0 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca377df740 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca377df740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca377df740 20 client.15639 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca377df740 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55feca4eb4e0 0x55feca4ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca377df740 4 client.15639 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.024+0000 7fca377df740 1 -- 172.21.15.32:0/3253373680 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feca4448b0 con 0x55feca4eb4e0 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.025+0000 7fca2e59c640 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55feca4eb4e0 0x55feca4ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.025+0000 7fca15ffb640 10 client.15639 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.025+0000 7fca2e59c640 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55feca4eb4e0 0x55feca4ed940 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b0502a14000462 server_cookie=ac5d57f096fe3b03 in_seq=0 out_seq=0 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca15ffb640 1 -- 172.21.15.32:0/3253373680 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55feca4448b0 con 0x55feca4eb4e0 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca15ffb640 10 client.15639 handle_command_reply: tid=0 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 client.15639 shutdown 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 -- 172.21.15.32:0/3253373680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 msgr2=0x55feca4d87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4d87f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fca24005e00 tx=0x7fca24009540 comp rx=0 tx=0).stop 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 -- 172.21.15.32:0/3253373680 shutdown_connections 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4d87f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55feca4eb4e0 0x55feca4ed940 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 --2- 172.21.15.32:0/3253373680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4d82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca15ffb640 0 client.15639 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 -- 172.21.15.32:0/3253373680 >> 172.21.15.32:0/3253373680 conn(0x55feca4cc3f0 msgr2=0x55feca4ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 -- 172.21.15.32:0/3253373680 shutdown_connections 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 1 -- 172.21.15.32:0/3253373680 wait complete. 2024-04-16T05:35:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.026+0000 7fca377df740 20 client.15639 trim_cache size 0 max 16384 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca377df740 1 Processor -- start 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca377df740 1 -- start start 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca377df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca377df740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca377df740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca377df740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b3be0 con 0x55feca4d25e0 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca377df740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b14b0 con 0x55feca4db1c0 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca377df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca301af300 con 0x55feca4d1cc0 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca2dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca2dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4e6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35954/0 (socket says 172.21.15.32:35954) 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.044+0000 7fca2dd9b640 1 -- 172.21.15.32:0/924330568 learned_addr learned my addr 172.21.15.32:0/924330568 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2e59c640 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2dd9b640 1 -- 172.21.15.32:0/924330568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 msgr2=0x55feca4da9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4da9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2dd9b640 1 -- 172.21.15.32:0/924330568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 msgr2=0x55feca4d8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2dd9b640 1 -- 172.21.15.32:0/924330568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feca4dd920 con 0x55feca4d25e0 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4e6db0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fca2402c0b0 tx=0x7fca240095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55006b23c91bc36 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca17fff640 1 -- 172.21.15.32:0/924330568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24058070 con 0x55feca4d25e0 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca17fff640 1 -- 172.21.15.32:0/924330568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca24032640 con 0x55feca4d25e0 2024-04-16T05:35:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.045+0000 7fca17fff640 1 -- 172.21.15.32:0/924330568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24043070 con 0x55feca4d25e0 2024-04-16T05:35:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 -- 172.21.15.32:0/924330568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 msgr2=0x55feca4e6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4e6db0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fca2402c0b0 tx=0x7fca240095b0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 -- 172.21.15.32:0/924330568 shutdown_connections 2024-04-16T05:35:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4db1c0 0x55feca4d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4d25e0 0x55feca4e6db0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 --2- 172.21.15.32:0/924330568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4da9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 -- 172.21.15.32:0/924330568 >> 172.21.15.32:0/924330568 conn(0x55feca4cc3f0 msgr2=0x55feca4d8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 -- 172.21.15.32:0/924330568 shutdown_connections 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 -- 172.21.15.32:0/924330568 wait complete. 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 -- start start 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.046+0000 7fca377df740 1 Processor -- start 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca377df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca377df740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4d25e0 0x55feca4e3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca377df740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4db1c0 0x55feca4e0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca377df740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca301af970 con 0x55feca4db1c0 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca377df740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b0de0 con 0x55feca4d25e0 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca377df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b0040 con 0x55feca4d1cc0 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca377df740 10 client.0 _ll_register_callbacks cb 0x7fca280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4db1c0 0x55feca4e0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4db1c0 0x55feca4e0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35960/0 (socket says 172.21.15.32:35960) 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 -- 172.21.15.32:0/2442005502 learned_addr learned my addr 172.21.15.32:0/2442005502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2e59c640 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2dd9b640 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4d25e0 0x55feca4e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 -- 172.21.15.32:0/2442005502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 msgr2=0x55feca4e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4e32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 -- 172.21.15.32:0/2442005502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4d25e0 msgr2=0x55feca4e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4d25e0 0x55feca4e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 -- 172.21.15.32:0/2442005502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feca4e0ac0 con 0x55feca4db1c0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2e59c640 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca2ed9d640 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4db1c0 0x55feca4e0290 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fca280040d0 tx=0x7fca28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72e01751e0ea79e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.047+0000 7fca15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca15ffb640 1 -- 172.21.15.32:0/2442005502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2802c060 con 0x55feca4db1c0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca15ffb640 1 -- 172.21.15.32:0/2442005502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca28004800 con 0x55feca4db1c0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 1 -- 172.21.15.32:0/2442005502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca24032640 con 0x55feca4db1c0 2024-04-16T05:35:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca15ffb640 1 -- 172.21.15.32:0/2442005502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca28033050 con 0x55feca4db1c0 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 10 client.15645 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 1 -- 172.21.15.32:0/2442005502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55feca552540 con 0x55feca4db1c0 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca15ffb640 1 -- 172.21.15.32:0/2442005502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca24032640 con 0x55feca4db1c0 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca15ffb640 1 -- 172.21.15.32:0/2442005502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca2801c950 con 0x55feca4db1c0 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 10 client.15645 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 20 client.15645 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55feca552c60 0x55feca555080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 4 client.15645 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca377df740 1 -- 172.21.15.32:0/2442005502 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55feca43e720 con 0x55feca552c60 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.048+0000 7fca2e59c640 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55feca552c60 0x55feca555080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.049+0000 7fca15ffb640 10 client.15645 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.049+0000 7fca2e59c640 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55feca552c60 0x55feca555080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f58b05a11e14d18a server_cookie=643cac6b05c646fe in_seq=0 out_seq=0 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca15ffb640 1 -- 172.21.15.32:0/2442005502 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55feca43e720 con 0x55feca552c60 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca15ffb640 10 client.15645 handle_command_reply: tid=0 2024-04-16T05:35:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 client.15645 shutdown 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 -- 172.21.15.32:0/2442005502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4db1c0 msgr2=0x55feca4e0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4db1c0 0x55feca4e0290 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fca280040d0 tx=0x7fca28004ab0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 -- 172.21.15.32:0/2442005502 shutdown_connections 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55feca4db1c0 0x55feca4e0290 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feca4d25e0 0x55feca4e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55feca552c60 0x55feca555080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 --2- 172.21.15.32:0/2442005502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55feca4d1cc0 0x55feca4e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca15ffb640 0 client.15645 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 -- 172.21.15.32:0/2442005502 >> 172.21.15.32:0/2442005502 conn(0x55feca4cc3f0 msgr2=0x55feca4cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 -- 172.21.15.32:0/2442005502 shutdown_connections 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 1 -- 172.21.15.32:0/2442005502 wait complete. 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.050+0000 7fca377df740 20 client.15645 trim_cache size 0 max 16384 2024-04-16T05:35:02.053 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:35:02.054 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:35:02.054 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "a29d0917-d53a-4b46-98b9-1ef5a66a219d", 2024-04-16T05:35:02.054 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:35:02.054 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:35:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 -- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca180a3db0 msgr2=0x7fca180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca180a3db0 0x7fca180a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca28005fd0 tx=0x7fca28009500 comp rx=0 tx=0).stop 2024-04-16T05:35:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 -- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102110 msgr2=0x7fca30074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102110 0x7fca30074300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fca2000e9c0 tx=0x7fca2000ee80 comp rx=0 tx=0).stop 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 -- 172.21.15.32:0/559841521 shutdown_connections 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca30074d80 0x7fca301a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca301071e0 0x7fca30074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca180a3db0 0x7fca180a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 --2- 172.21.15.32:0/559841521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca30102110 0x7fca30074300 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 -- 172.21.15.32:0/559841521 >> 172.21.15.32:0/559841521 conn(0x7fca300fd9d0 msgr2=0x7fca300fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 -- 172.21.15.32:0/559841521 shutdown_connections 2024-04-16T05:35:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.051+0000 7fca2f59e640 1 -- 172.21.15.32:0/559841521 wait complete. 2024-04-16T05:35:02.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:35:02.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.169+0000 7f4e1b28f640 1 -- 172.21.15.32:0/711242748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e14100e00 msgr2=0x7f4e141011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.169+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/711242748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e14100e00 0x7f4e141011e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009900 tx=0x7f4e1002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.169+0000 7f4e1b28f640 1 -- 172.21.15.32:0/711242748 shutdown_connections 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.169+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/711242748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e1410e9b0 0x7f4e14110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.169+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/711242748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e14101720 0x7f4e1410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.169+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/711242748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e14100e00 0x7f4e141011e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.169+0000 7f4e1b28f640 1 -- 172.21.15.32:0/711242748 >> 172.21.15.32:0/711242748 conn(0x7f4e14078000 msgr2=0x7f4e14078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 -- 172.21.15.32:0/711242748 shutdown_connections 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 -- 172.21.15.32:0/711242748 wait complete. 2024-04-16T05:35:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 Processor -- start 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 -- start start 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e14100e00 0x7f4e141a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14101720 0x7f4e141a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1410e9b0 0x7f4e1419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141145d0 con 0x7f4e1410e9b0 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14114450 con 0x7f4e14100e00 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1b28f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14114750 con 0x7f4e14101720 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1982c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1410e9b0 0x7f4e1419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1982c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1410e9b0 0x7f4e1419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35976/0 (socket says 172.21.15.32:35976) 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.170+0000 7f4e1982c640 1 -- 172.21.15.32:0/2111822994 learned_addr learned my addr 172.21.15.32:0/2111822994 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1902b640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e14100e00 0x7f4e141a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14101720 0x7f4e141a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1982c640 1 -- 172.21.15.32:0/2111822994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14101720 msgr2=0x7f4e141a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1982c640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14101720 0x7f4e141a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1982c640 1 -- 172.21.15.32:0/2111822994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e14100e00 msgr2=0x7f4e141a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1982c640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e14100e00 0x7f4e141a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1982c640 1 -- 172.21.15.32:0/2111822994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1419b130 con 0x7f4e1410e9b0 2024-04-16T05:35:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1902b640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e14100e00 0x7f4e141a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1982c640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1410e9b0 0x7f4e1419a9b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00e9d0 tx=0x7f4e0c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/2111822994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00cd80 con 0x7f4e1410e9b0 2024-04-16T05:35:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/2111822994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e0c005020 con 0x7f4e1410e9b0 2024-04-16T05:35:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1b28f640 1 -- 172.21.15.32:0/2111822994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1419b420 con 0x7f4e1410e9b0 2024-04-16T05:35:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/2111822994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0116d0 con 0x7f4e1410e9b0 2024-04-16T05:35:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.171+0000 7f4e1b28f640 1 -- 172.21.15.32:0/2111822994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e141a76c0 con 0x7f4e1410e9b0 2024-04-16T05:35:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.173+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/2111822994 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4e0c0051d0 con 0x7f4e1410e9b0 2024-04-16T05:35:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1de7f740 1 Processor -- start 2024-04-16T05:35:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1de7f740 1 -- start start 2024-04-16T05:35:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9e54230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e5d350 0x55fed9e5f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fed9db9180 con 0x55fed9e54770 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e0a7fc640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e000a3db0 0x7f4e000a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fed9d56360 con 0x55fed9e5d350 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fed9db5140 con 0x55fed9e53e50 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1902b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1902b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35980/0 (socket says 172.21.15.32:35980) 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1982c640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e000a3db0 0x7f4e000a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1902b640 1 -- 172.21.15.32:0/2623779931 learned_addr learned my addr 172.21.15.32:0/2623779931 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9e54230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/2111822994 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4e0c0d12b0 con 0x7f4e1410e9b0 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.174+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e5d350 0x55fed9e5f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4e1882a640 1 -- 172.21.15.32:0/2623779931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e5d350 msgr2=0x55fed9e5f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e5d350 0x55fed9e5f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4e1882a640 1 -- 172.21.15.32:0/2623779931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 msgr2=0x55fed9e5ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4e1882a640 1 -- 172.21.15.32:0/2623779931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed9b957d0 con 0x55fed9e53e50 2024-04-16T05:35:02.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4e1902b640 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:02.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4e1982c640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e000a3db0 0x7f4e000a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0ce600 tx=0x7f4e0c005b40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9e54230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e1419c1a0 tx=0x7f4e0400c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=139835700d581820 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4dff7fe640 1 -- 172.21.15.32:0/2623779931 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004030 con 0x55fed9e53e50 2024-04-16T05:35:02.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4dff7fe640 1 -- 172.21.15.32:0/2623779931 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e0400fc40 con 0x55fed9e53e50 2024-04-16T05:35:02.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.175+0000 7f4dff7fe640 1 -- 172.21.15.32:0/2623779931 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04003850 con 0x55fed9e53e50 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2623779931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 msgr2=0x55fed9e54230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9e54230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e1419c1a0 tx=0x7f4e0400c430 comp rx=0 tx=0).stop 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2623779931 shutdown_connections 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e5d350 0x55fed9e5f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2623779931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9e54230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2623779931 >> 172.21.15.32:0/2623779931 conn(0x55fed9e4e580 msgr2=0x55fed9e4e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2623779931 shutdown_connections 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2623779931 wait complete. 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 -- start start 2024-04-16T05:35:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.176+0000 7f4e1de7f740 1 Processor -- start 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e53e50 0x55fed9e5a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9e5a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 0x55fed9e5adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fed9e60140 con 0x55fed9e53e50 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fed9d722c0 con 0x55fed9e54770 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fed9e602c0 con 0x55fed9e5d350 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1de7f740 10 client.0 _ll_register_callbacks cb 0x55fed9d59090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e53e50 0x55fed9e5a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e53e50 0x55fed9e5a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35988/0 (socket says 172.21.15.32:35988) 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 -- 172.21.15.32:0/1188938301 learned_addr learned my addr 172.21.15.32:0/1188938301 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1902b640 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9e5a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 0x55fed9e5adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 -- 172.21.15.32:0/1188938301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 msgr2=0x55fed9e5adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 0x55fed9e5adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 -- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 msgr2=0x55fed9e5a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9e5a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 -- 172.21.15.32:0/1188938301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed9acb3f0 con 0x55fed9e53e50 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1902b640 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9e5a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4e1882a640 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e53e50 0x55fed9e5a370 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400cc30 tx=0x7f4e04005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bea203ac065c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.177+0000 7f4dfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4dfcff9640 1 -- 172.21.15.32:0/1188938301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04012650 con 0x55fed9e53e50 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4dfcff9640 1 -- 172.21.15.32:0/1188938301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4e040127f0 con 0x55fed9e53e50 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1188938301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e0400fc40 con 0x55fed9e53e50 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4dfcff9640 1 -- 172.21.15.32:0/1188938301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e040129f0 con 0x55fed9e53e50 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 10 client.15657 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1188938301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fed9a8b0f0 con 0x55fed9e53e50 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4dfcff9640 1 -- 172.21.15.32:0/1188938301 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e0400fc40 con 0x55fed9e53e50 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4dfcff9640 1 -- 172.21.15.32:0/1188938301 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e0401fa40 con 0x55fed9e53e50 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 10 client.15657 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 20 client.15657 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fed9e6d6b0 0x55fed9e6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 4 client.15657 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.178+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1188938301 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fed9dc6ba0 con 0x55fed9e6d6b0 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.179+0000 7f4e1902b640 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fed9e6d6b0 0x55fed9e6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.179+0000 7f4dfcff9640 10 client.15657 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.179+0000 7f4e1902b640 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fed9e6d6b0 0x55fed9e6faa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d1278ecdb71475 server_cookie=994b92397bae19e8 in_seq=0 out_seq=0 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4dfcff9640 1 -- 172.21.15.32:0/1188938301 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fed9dc6ba0 con 0x55fed9e6d6b0 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4dfcff9640 10 client.15657 handle_command_reply: tid=0 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 client.15657 shutdown 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1188938301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e53e50 msgr2=0x55fed9e5a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e53e50 0x55fed9e5a370 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400cc30 tx=0x7f4e04005c50 comp rx=0 tx=0).stop 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1188938301 shutdown_connections 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fed9e6d6b0 0x55fed9e6faa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 0x55fed9e5adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9e5a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4dfcff9640 0 client.15657 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1188938301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e53e50 0x55fed9e5a370 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1188938301 >> 172.21.15.32:0/1188938301 conn(0x55fed9e4e580 msgr2=0x55fed9e502d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1188938301 shutdown_connections 2024-04-16T05:35:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1188938301 wait complete. 2024-04-16T05:35:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.180+0000 7f4e1de7f740 20 client.15657 trim_cache size 0 max 16384 2024-04-16T05:35:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1de7f740 1 Processor -- start 2024-04-16T05:35:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1de7f740 1 -- start start 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e53e50 0x55fed9e69060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 0x55fed9e4d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141b3f10 con 0x55fed9e54770 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141b17e0 con 0x55fed9e53e50 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141af630 con 0x55fed9e5d350 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1902b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1902b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35990/0 (socket says 172.21.15.32:35990) 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1902b640 1 -- 172.21.15.32:0/1568316821 learned_addr learned my addr 172.21.15.32:0/1568316821 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.198+0000 7f4e1882a640 1 --2- 172.21.15.32:0/1568316821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e53e50 0x55fed9e69060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4e1902b640 1 -- 172.21.15.32:0/1568316821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 msgr2=0x55fed9e4d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4e1902b640 1 --2- 172.21.15.32:0/1568316821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 0x55fed9e4d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4e1902b640 1 -- 172.21.15.32:0/1568316821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e53e50 msgr2=0x55fed9e69060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4e1902b640 1 --2- 172.21.15.32:0/1568316821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e53e50 0x55fed9e69060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4e1902b640 1 -- 172.21.15.32:0/1568316821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed9e5f740 con 0x55fed9e54770 2024-04-16T05:35:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4e1902b640 1 --2- 172.21.15.32:0/1568316821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ac10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4e10036040 tx=0x7f4e1002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1604cc0ed1cf0193 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4dfeffd640 1 -- 172.21.15.32:0/1568316821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10045b50 con 0x55fed9e54770 2024-04-16T05:35:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4dfeffd640 1 -- 172.21.15.32:0/1568316821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1004a070 con 0x55fed9e54770 2024-04-16T05:35:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.199+0000 7f4dfeffd640 1 -- 172.21.15.32:0/1568316821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10005570 con 0x55fed9e54770 2024-04-16T05:35:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1568316821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 msgr2=0x55fed9e5ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1568316821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ac10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4e10036040 tx=0x7f4e1002e040 comp rx=0 tx=0).stop 2024-04-16T05:35:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1568316821 shutdown_connections 2024-04-16T05:35:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1568316821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e5d350 0x55fed9e4d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1568316821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e54770 0x55fed9e5ac10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/1568316821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e53e50 0x55fed9e69060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1568316821 >> 172.21.15.32:0/1568316821 conn(0x55fed9e4e580 msgr2=0x55fed9e4ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1568316821 shutdown_connections 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- 172.21.15.32:0/1568316821 wait complete. 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- start start 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 Processor -- start 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9ecbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9ecc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e5d350 0x55fed9e62870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141afca0 con 0x55fed9e5d350 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141b1110 con 0x55fed9e54770 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141b0370 con 0x55fed9e53e50 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.200+0000 7f4e1de7f740 10 client.0 _ll_register_callbacks cb 0x7f4e10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e5d350 0x55fed9e62870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e5d350 0x55fed9e62870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35994/0 (socket says 172.21.15.32:35994) 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 -- 172.21.15.32:0/2844094300 learned_addr learned my addr 172.21.15.32:0/2844094300 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1902b640 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9ecc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9ecbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 -- 172.21.15.32:0/2844094300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 msgr2=0x55fed9ecbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9ecbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 -- 172.21.15.32:0/2844094300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 msgr2=0x55fed9ecc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9ecc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 -- 172.21.15.32:0/2844094300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed9e62f60 con 0x55fed9e5d350 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1882a640 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e5d350 0x55fed9e62870 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4e040059f0 tx=0x7f4e0400c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45363461be7a5646 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4dfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4dfcff9640 1 -- 172.21.15.32:0/2844094300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400fa70 con 0x55fed9e5d350 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2844094300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1004a070 con 0x55fed9e5d350 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4dfcff9640 1 -- 172.21.15.32:0/2844094300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4e0400fc10 con 0x55fed9e5d350 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4dfcff9640 1 -- 172.21.15.32:0/2844094300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e04012b90 con 0x55fed9e5d350 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1de7f740 10 client.15663 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.201+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2844094300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fed9ed3640 con 0x55fed9e5d350 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4dfcff9640 1 -- 172.21.15.32:0/2844094300 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1004a070 con 0x55fed9e5d350 2024-04-16T05:35:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4dfcff9640 1 -- 172.21.15.32:0/2844094300 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e0403a040 con 0x55fed9e5d350 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4e1de7f740 10 client.15663 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4e1de7f740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4e1de7f740 20 client.15663 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fed9ed3b20 0x55fed9ed5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4e1de7f740 4 client.15663 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2844094300 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fed9dc0a10 con 0x55fed9ed3b20 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.202+0000 7f4e1902b640 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fed9ed3b20 0x55fed9ed5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4dfcff9640 10 client.15663 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1902b640 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fed9ed3b20 0x55fed9ed5f10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b3d9990c578266 server_cookie=54bf986d8a6375ae in_seq=0 out_seq=0 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4dfcff9640 1 -- 172.21.15.32:0/2844094300 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fed9dc0a10 con 0x55fed9ed3b20 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4dfcff9640 10 client.15663 handle_command_reply: tid=0 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 client.15663 shutdown 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2844094300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e5d350 msgr2=0x55fed9e62870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e5d350 0x55fed9e62870 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4e040059f0 tx=0x7f4e0400c430 comp rx=0 tx=0).stop 2024-04-16T05:35:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2844094300 shutdown_connections 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fed9e5d350 0x55fed9e62870 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fed9e54770 0x55fed9ecc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fed9ed3b20 0x55fed9ed5f10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4dfcff9640 0 client.15663 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 --2- 172.21.15.32:0/2844094300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fed9e53e50 0x55fed9ecbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2844094300 >> 172.21.15.32:0/2844094300 conn(0x55fed9e4e580 msgr2=0x55fed9e4ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2844094300 shutdown_connections 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 1 -- 172.21.15.32:0/2844094300 wait complete. 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.203+0000 7f4e1de7f740 20 client.15663 trim_cache size 0 max 16384 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:35:02.207 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:35:02.208 INFO:teuthology.orchestra.run.smithi032.stdout: "a29d0917-d53a-4b46-98b9-1ef5a66a219d": { 2024-04-16T05:35:02.208 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:35:02.208 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "a29d0917-d53a-4b46-98b9-1ef5a66a219d", 2024-04-16T05:35:02.208 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:35:02.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:35:02.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:35:02.208 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 -- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e000a3db0 msgr2=0x7f4e000a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e000a3db0 0x7f4e000a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0ce600 tx=0x7f4e0c005b40 comp rx=0 tx=0).stop 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 -- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1410e9b0 msgr2=0x7f4e1419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1410e9b0 0x7f4e1419a9b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00e9d0 tx=0x7f4e0c00ee90 comp rx=0 tx=0).stop 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 -- 172.21.15.32:0/2111822994 shutdown_connections 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e000a3db0 0x7f4e000a6270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1410e9b0 0x7f4e1419a9b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14101720 0x7f4e141a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 --2- 172.21.15.32:0/2111822994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e14100e00 0x7f4e141a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 -- 172.21.15.32:0/2111822994 >> 172.21.15.32:0/2111822994 conn(0x7f4e14078000 msgr2=0x7f4e14110050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 -- 172.21.15.32:0/2111822994 shutdown_connections 2024-04-16T05:35:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.204+0000 7f4e1b28f640 1 -- 172.21.15.32:0/2111822994 wait complete. 2024-04-16T05:35:02.217 INFO:tasks.cephfs.filesystem:scrub status for tag:a29d0917-d53a-4b46-98b9-1ef5a66a219d - {'path': '/', 'tag': 'a29d0917-d53a-4b46-98b9-1ef5a66a219d', 'options': 'recursive,force'} 2024-04-16T05:35:02.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.245+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7b94062440 con 0x7f7b840a3ee0 2024-04-16T05:35:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.309+0000 7f7b91ffb640 1 -- 172.21.15.32:0/3820604353 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7b94062440 con 0x7f7b840a3ee0 2024-04-16T05:35:02.308 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path / 2024-04-16T05:35:02.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7b840a3ee0 msgr2=0x7f7b840a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7b840a3ee0 0x7f7b840a63a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b8800b390 comp rx=0 tx=0).stop 2024-04-16T05:35:02.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9410c610 msgr2=0x7f7b941987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9410c610 0x7f7b941987b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009fd0 tx=0x7f7b8c00b260 comp rx=0 tx=0).stop 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 shutdown_connections 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7b840a3ee0 0x7f7b840a63a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b9410c610 0x7f7b941987b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b94103bf0 0x7f7b9419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 --2- 172.21.15.32:0/3820604353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b941032d0 0x7f7b9419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 >> 172.21.15.32:0/3820604353 conn(0x7f7b94078080 msgr2=0x7f7b940ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 shutdown_connections 2024-04-16T05:35:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.312+0000 7f7b9ab31640 1 -- 172.21.15.32:0/3820604353 wait complete. 2024-04-16T05:35:02.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-16T05:35:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 -- 172.21.15.32:0/3017193283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7141011a0 msgr2=0x7fb714101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 --2- 172.21.15.32:0/3017193283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7141011a0 0x7fb714101580 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009ca0 tx=0x7fb70c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 -- 172.21.15.32:0/3017193283 shutdown_connections 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 --2- 172.21.15.32:0/3017193283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb714106220 0x7fb71410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 --2- 172.21.15.32:0/3017193283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb714101b50 0x7fb714105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 --2- 172.21.15.32:0/3017193283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7141011a0 0x7fb714101580 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 -- 172.21.15.32:0/3017193283 >> 172.21.15.32:0/3017193283 conn(0x7fb714078080 msgr2=0x7fb714078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 -- 172.21.15.32:0/3017193283 shutdown_connections 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.471+0000 7fb71904a640 1 -- 172.21.15.32:0/3017193283 wait complete. 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.472+0000 7fb71904a640 1 Processor -- start 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.472+0000 7fb71904a640 1 -- start start 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71904a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7141011a0 0x7fb71419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71904a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb714106220 0x7fb71419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71904a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb714198730 0x7fb714198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71904a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7141112d0 con 0x7fb714198730 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71904a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb714111150 con 0x7fb7141011a0 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71904a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb714111450 con 0x7fb714106220 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb714198730 0x7fb714198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb714198730 0x7fb714198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36022/0 (socket says 172.21.15.32:36022) 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 -- 172.21.15.32:0/3133370002 learned_addr learned my addr 172.21.15.32:0/3133370002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71259c640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb714106220 0x7fb71419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb712d9d640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7141011a0 0x7fb71419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 -- 172.21.15.32:0/3133370002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb714106220 msgr2=0x7fb71419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb714106220 0x7fb71419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 -- 172.21.15.32:0/3133370002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7141011a0 msgr2=0x7fb71419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7141011a0 0x7fb71419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 -- 172.21.15.32:0/3133370002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb714199330 con 0x7fb714198730 2024-04-16T05:35:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71359e640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb714198730 0x7fb714198bb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb70400d750 tx=0x7fb70400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb70bfff640 1 -- 172.21.15.32:0/3133370002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704017070 con 0x7fb714198730 2024-04-16T05:35:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb70bfff640 1 -- 172.21.15.32:0/3133370002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb704005500 con 0x7fb714198730 2024-04-16T05:35:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7141a5410 con 0x7fb714198730 2024-04-16T05:35:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb70bfff640 1 -- 172.21.15.32:0/3133370002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704004d90 con 0x7fb714198730 2024-04-16T05:35:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.473+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7141a57c0 con 0x7fb714198730 2024-04-16T05:35:02.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.474+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb714102620 con 0x7fb714198730 2024-04-16T05:35:02.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.476+0000 7fb70bfff640 1 -- 172.21.15.32:0/3133370002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fb704004f30 con 0x7fb714198730 2024-04-16T05:35:02.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.476+0000 7fb70bfff640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7000a3e60 0x7fb7000a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.476+0000 7fb712d9d640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7000a3e60 0x7fb7000a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.476+0000 7fb70bfff640 1 -- 172.21.15.32:0/3133370002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fb7040d0670 con 0x7fb714198730 2024-04-16T05:35:02.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.477+0000 7fb712d9d640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7000a3e60 0x7fb7000a6320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb70c000f80 tx=0x7fb70c005c30 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.481+0000 7fb70bfff640 1 -- 172.21.15.32:0/3133370002 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb70409c860 con 0x7fb714198730 2024-04-16T05:35:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.694+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb714101b50 con 0x7fb7000a3e60 2024-04-16T05:35:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.753+0000 7fb70bfff640 1 -- 172.21.15.32:0/3133370002 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb714101b50 con 0x7fb7000a3e60 2024-04-16T05:35:02.752 INFO:teuthology.orchestra.run.smithi032.stdout:Retention added to path / 2024-04-16T05:35:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.755+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7000a3e60 msgr2=0x7fb7000a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.755+0000 7fb71904a640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7000a3e60 0x7fb7000a6320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb70c000f80 tx=0x7fb70c005c30 comp rx=0 tx=0).stop 2024-04-16T05:35:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.755+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb714198730 msgr2=0x7fb714198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.755+0000 7fb71904a640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb714198730 0x7fb714198bb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb70400d750 tx=0x7fb70400dc10 comp rx=0 tx=0).stop 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.755+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 shutdown_connections 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.755+0000 7fb71904a640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7000a3e60 0x7fb7000a6320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.755+0000 7fb71904a640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb714198730 0x7fb714198bb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.756+0000 7fb71904a640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb714106220 0x7fb71419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.756+0000 7fb71904a640 1 --2- 172.21.15.32:0/3133370002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7141011a0 0x7fb71419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.756+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 >> 172.21.15.32:0/3133370002 conn(0x7fb714078080 msgr2=0x7fb714069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.756+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 shutdown_connections 2024-04-16T05:35:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.756+0000 7fb71904a640 1 -- 172.21.15.32:0/3133370002 wait complete. 2024-04-16T05:35:02.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-16T05:35:02.840 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:02 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2069944902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:02.840 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:02 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:35:02.840 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:02 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:02.840 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:02 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:35:02.840 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:02 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 -- 172.21.15.32:0/253224172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474103290 msgr2=0x7f8474103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 --2- 172.21.15.32:0/253224172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474103290 0x7f8474103670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f846402bd60 comp rx=0 tx=0).stop 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 -- 172.21.15.32:0/253224172 shutdown_connections 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 --2- 172.21.15.32:0/253224172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f847410c5d0 0x7f847410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 --2- 172.21.15.32:0/253224172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8474103bb0 0x7f847410c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 --2- 172.21.15.32:0/253224172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8474103290 0x7f8474103670 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 -- 172.21.15.32:0/253224172 >> 172.21.15.32:0/253224172 conn(0x7f8474078080 msgr2=0x7f8474078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 -- 172.21.15.32:0/253224172 shutdown_connections 2024-04-16T05:35:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 -- 172.21.15.32:0/253224172 wait complete. 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 Processor -- start 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.873+0000 7f8478b00640 1 -- start start 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8478b00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8474103290 0x7f847419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8478b00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8474103bb0 0x7f847419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8478b00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847410c5d0 0x7f8474198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8478b00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8474112460 con 0x7f847410c5d0 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8478b00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84741122e0 con 0x7f8474103bb0 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8478b00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84741125e0 con 0x7f8474103290 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847410c5d0 0x7f8474198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847410c5d0 0x7f8474198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36036/0 (socket says 172.21.15.32:36036) 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 -- 172.21.15.32:0/3302826800 learned_addr learned my addr 172.21.15.32:0/3302826800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8471d9b640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8474103bb0 0x7f847419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f847259c640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8474103290 0x7f847419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 -- 172.21.15.32:0/3302826800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8474103290 msgr2=0x7f847419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8474103290 0x7f847419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 -- 172.21.15.32:0/3302826800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8474103bb0 msgr2=0x7f847419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8474103bb0 0x7f847419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 -- 172.21.15.32:0/3302826800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8474198de0 con 0x7f847410c5d0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f847259c640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8474103290 0x7f847419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8472d9d640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847410c5d0 0x7f8474198660 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f846c00b970 tx=0x7f846c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f846b7fe640 1 -- 172.21.15.32:0/3302826800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c019070 con 0x7f847410c5d0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f846b7fe640 1 -- 172.21.15.32:0/3302826800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c014440 con 0x7f847410c5d0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84741990d0 con 0x7f847410c5d0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.874+0000 7f846b7fe640 1 -- 172.21.15.32:0/3302826800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c0075d0 con 0x7f847410c5d0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.875+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84741a5320 con 0x7f847410c5d0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.875+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847404fd90 con 0x7f847410c5d0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.877+0000 7f846b7fe640 1 -- 172.21.15.32:0/3302826800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f846c01c070 con 0x7f847410c5d0 2024-04-16T05:35:02.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.877+0000 7f846b7fe640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84600a3e60 0x7f84600a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:02.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.877+0000 7f847259c640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84600a3e60 0x7f84600a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:02.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.877+0000 7f846b7fe640 1 -- 172.21.15.32:0/3302826800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f846c0d0180 con 0x7f847410c5d0 2024-04-16T05:35:02.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.878+0000 7f847259c640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84600a3e60 0x7f84600a6320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8464006010 tx=0x7f8464005a20 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:02.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:02.882+0000 7f846b7fe640 1 -- 172.21.15.32:0/3302826800 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f846c09c480 con 0x7f847410c5d0 2024-04-16T05:35:02.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:02 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2069944902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:02.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:02 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:35:02.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:02 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:02.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:02 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:35:02.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:02 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:02.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:02 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2069944902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:35:02.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:02 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:35:02.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:02 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:02.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:02 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:35:02.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:02 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.092+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f84740fe5a0 con 0x7f84600a3e60 2024-04-16T05:35:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.100+0000 7f846b7fe640 1 -- 172.21.15.32:0/3302826800 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f84740fe5a0 con 0x7f84600a3e60 2024-04-16T05:35:03.098 INFO:teuthology.orchestra.run.smithi032.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-16T00:00:00", "created": "2024-04-16T05:35:02", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84600a3e60 msgr2=0x7f84600a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84600a3e60 0x7f84600a6320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8464006010 tx=0x7f8464005a20 comp rx=0 tx=0).stop 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847410c5d0 msgr2=0x7f8474198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847410c5d0 0x7f8474198660 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f846c00b970 tx=0x7f846c00bd70 comp rx=0 tx=0).stop 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 shutdown_connections 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84600a3e60 0x7f84600a6320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847410c5d0 0x7f8474198660 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8474103bb0 0x7f847419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 --2- 172.21.15.32:0/3302826800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8474103290 0x7f847419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 >> 172.21.15.32:0/3302826800 conn(0x7f8474078080 msgr2=0x7f84740fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 shutdown_connections 2024-04-16T05:35:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.102+0000 7f8478b00640 1 -- 172.21.15.32:0/3302826800 wait complete. 2024-04-16T05:35:03.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-16T05:35:03.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 -- 172.21.15.32:0/746726964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498102280 msgr2=0x7f4498102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 --2- 172.21.15.32:0/746726964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498102280 0x7f4498102660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4488009ca0 tx=0x7f448802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 -- 172.21.15.32:0/746726964 shutdown_connections 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 --2- 172.21.15.32:0/746726964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4498107350 0x7f449810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 --2- 172.21.15.32:0/746726964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4498102c30 0x7f4498106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 --2- 172.21.15.32:0/746726964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498102280 0x7f4498102660 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 -- 172.21.15.32:0/746726964 >> 172.21.15.32:0/746726964 conn(0x7f44980fdac0 msgr2=0x7f44980ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 -- 172.21.15.32:0/746726964 shutdown_connections 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 -- 172.21.15.32:0/746726964 wait complete. 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.220+0000 7f449f260640 1 Processor -- start 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449f260640 1 -- start start 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449f260640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4498102280 0x7f4498074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449f260640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4498107350 0x7f4498074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449f260640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498074d80 0x7f44981a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449f260640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4498112410 con 0x7f4498074d80 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449f260640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4498112290 con 0x7f4498107350 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449f260640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4498112590 con 0x7f4498102280 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498074d80 0x7f44981a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498074d80 0x7f44981a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36054/0 (socket says 172.21.15.32:36054) 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 -- 172.21.15.32:0/1484570578 learned_addr learned my addr 172.21.15.32:0/1484570578 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f4497fff640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4498107350 0x7f4498074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449cffc640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4498102280 0x7f4498074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 -- 172.21.15.32:0/1484570578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4498102280 msgr2=0x7f4498074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4498102280 0x7f4498074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 -- 172.21.15.32:0/1484570578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4498107350 msgr2=0x7f4498074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4498107350 0x7f4498074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 -- 172.21.15.32:0/1484570578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44981a70c0 con 0x7f4498074d80 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449d7fd640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498074d80 0x7f44981a6a20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f449000e970 tx=0x7f449000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f4495ffb640 1 -- 172.21.15.32:0/1484570578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449000ccc0 con 0x7f4498074d80 2024-04-16T05:35:03.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f4495ffb640 1 -- 172.21.15.32:0/1484570578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449000ce60 con 0x7f4498074d80 2024-04-16T05:35:03.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44981a73b0 con 0x7f4498074d80 2024-04-16T05:35:03.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.221+0000 7f4495ffb640 1 -- 172.21.15.32:0/1484570578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490010490 con 0x7f4498074d80 2024-04-16T05:35:03.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.222+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981a7900 con 0x7f4498074d80 2024-04-16T05:35:03.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.223+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449804fd90 con 0x7f4498074d80 2024-04-16T05:35:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.223+0000 7f4495ffb640 1 -- 172.21.15.32:0/1484570578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4490005020 con 0x7f4498074d80 2024-04-16T05:35:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.224+0000 7f4495ffb640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44840a3e60 0x7f44840a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.224+0000 7f449cffc640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44840a3e60 0x7f44840a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.224+0000 7f4495ffb640 1 -- 172.21.15.32:0/1484570578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f44900d11e0 con 0x7f4498074d80 2024-04-16T05:35:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.225+0000 7f449cffc640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44840a3e60 0x7f44840a6320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4488037040 tx=0x7f4488009820 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.229+0000 7f4495ffb640 1 -- 172.21.15.32:0/1484570578 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f449009d3d0 con 0x7f4498074d80 2024-04-16T05:35:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.438+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f449810a5f0 con 0x7f44840a3e60 2024-04-16T05:35:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.445+0000 7f4495ffb640 1 -- 172.21.15.32:0/1484570578 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f448803a040 con 0x7f44840a3e60 2024-04-16T05:35:03.443 INFO:teuthology.orchestra.run.smithi032.stdout:/ 1m 6m3h 2024-04-16T05:35:03.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44840a3e60 msgr2=0x7f44840a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44840a3e60 0x7f44840a6320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4488037040 tx=0x7f4488009820 comp rx=0 tx=0).stop 2024-04-16T05:35:03.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498074d80 msgr2=0x7f44981a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:03.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498074d80 0x7f44981a6a20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f449000e970 tx=0x7f449000ed70 comp rx=0 tx=0).stop 2024-04-16T05:35:03.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 shutdown_connections 2024-04-16T05:35:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44840a3e60 0x7f44840a6320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4498074d80 0x7f44981a6a20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4498107350 0x7f4498074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 --2- 172.21.15.32:0/1484570578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4498102280 0x7f4498074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 >> 172.21.15.32:0/1484570578 conn(0x7f44980fdac0 msgr2=0x7f44980fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 shutdown_connections 2024-04-16T05:35:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:03.447+0000 7f449f260640 1 -- 172.21.15.32:0/1484570578 wait complete. 2024-04-16T05:35:03.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-16T05:35:03.486 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-16T05:35:03.520 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-16T05:35:03.571 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:35:03.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-16T05:35:03.574 INFO:teuthology.run_tasks:Running task workunit... 2024-04-16T05:35:03.593 INFO:tasks.workunit:Pulling workunits from ref a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:35:03.593 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-16T05:35:03.593 INFO:tasks.workunit:timeout=3h 2024-04-16T05:35:03.593 INFO:tasks.workunit:cleanup=True 2024-04-16T05:35:03.593 DEBUG:teuthology.orchestra.run.smithi032:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:03.608 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-16T05:35:03.609 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-04-16T05:35:03.609 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T05:35:03.609 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-16T05:35:03.609 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-16 05:32:57.150732309 +0000 2024-04-16T05:35:03.609 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-16 05:32:57.150732309 +0000 2024-04-16T05:35:03.609 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-16 05:34:53.794383604 +0000 2024-04-16T05:35:03.609 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-16 05:32:57.150732309 +0000 2024-04-16T05:35:03.609 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-16T05:35:03.609 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-16T05:35:03.682 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a87fcff6340baeedd43af126e793f3241b6400ea 2024-04-16T05:35:03.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:03 smithi032 ceph-mon[32351]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:03.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:03 smithi032 ceph-mon[32351]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:03.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:03 smithi032 ceph-mon[32351]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:03.738 INFO:tasks.workunit.client.0.smithi032.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-16T05:35:03.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:03 smithi134 ceph-mon[36153]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:03.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:03 smithi134 ceph-mon[36153]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:03.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:03 smithi134 ceph-mon[36153]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:03.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:03 smithi082 ceph-mon[36195]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T05:35:03.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:03 smithi082 ceph-mon[36195]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:03.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:03 smithi082 ceph-mon[36195]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:04.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:04 smithi134 ceph-mon[36153]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:04.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:04 smithi134 ceph-mon[36153]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:04.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:04 smithi082 ceph-mon[36195]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:04.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:04 smithi082 ceph-mon[36195]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:04 smithi032 ceph-mon[32351]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:04 smithi032 ceph-mon[32351]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:35:05.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:05 smithi134 ceph-mon[36153]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-04-16T05:35:05.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:05 smithi082 ceph-mon[36195]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-04-16T05:35:05.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:05 smithi032 ceph-mon[32351]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-04-16T05:35:07.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:07 smithi134 ceph-mon[36153]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-16T05:35:07.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:07 smithi082 ceph-mon[36195]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-16T05:35:07.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:07 smithi032 ceph-mon[32351]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-16T05:35:09.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:09 smithi134 ceph-mon[36153]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-16T05:35:09.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:09 smithi082 ceph-mon[36195]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-16T05:35:09.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:09 smithi032 ceph-mon[32351]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-16T05:35:11.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:11 smithi134 ceph-mon[36153]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-16T05:35:11.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:11 smithi082 ceph-mon[36195]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-16T05:35:11.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:11 smithi032 ceph-mon[32351]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-16T05:35:13.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:13 smithi134 ceph-mon[36153]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-16T05:35:13.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:13 smithi082 ceph-mon[36195]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-16T05:35:13.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:13 smithi032 ceph-mon[32351]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-16T05:35:15.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:15 smithi134 ceph-mon[36153]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-16T05:35:15.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:15 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:35:15.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:15 smithi082 ceph-mon[36195]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-16T05:35:15.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:15 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:35:15.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:15 smithi032 ceph-mon[32351]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-16T05:35:15.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:15 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:35:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 -- 172.21.15.32:0/2352293685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae41038e0 msgr2=0x7f6ae410bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 --2- 172.21.15.32:0/2352293685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae41038e0 0x7f6ae410bdc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009900 tx=0x7f6acc02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 -- 172.21.15.32:0/2352293685 shutdown_connections 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 --2- 172.21.15.32:0/2352293685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae410c3e0 0x7f6ae410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 --2- 172.21.15.32:0/2352293685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae41038e0 0x7f6ae410bdc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 --2- 172.21.15.32:0/2352293685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae4102fc0 0x7f6ae41033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 -- 172.21.15.32:0/2352293685 >> 172.21.15.32:0/2352293685 conn(0x7f6ae40fc820 msgr2=0x7f6ae40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 -- 172.21.15.32:0/2352293685 shutdown_connections 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.720+0000 7f6ae359e640 1 -- 172.21.15.32:0/2352293685 wait complete. 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.721+0000 7f6ae359e640 1 Processor -- start 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.721+0000 7f6ae359e640 1 -- start start 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.721+0000 7f6ae359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4102fc0 0x7f6ae4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.721+0000 7f6ae359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41038e0 0x7f6ae4198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae410c3e0 0x7f6ae419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae359e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41121c0 con 0x7f6ae410c3e0 2024-04-16T05:35:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae359e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4112040 con 0x7f6ae4102fc0 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4112340 con 0x7f6ae41038e0 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae410c3e0 0x7f6ae419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae410c3e0 0x7f6ae419d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36568/0 (socket says 172.21.15.32:36568) 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/896976360 learned_addr learned my addr 172.21.15.32:0/896976360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6adbfff640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41038e0 0x7f6ae4198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae259c640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4102fc0 0x7f6ae4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/896976360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41038e0 msgr2=0x7f6ae4198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41038e0 0x7f6ae4198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/896976360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4102fc0 msgr2=0x7f6ae4198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4102fc0 0x7f6ae4198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/896976360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae419d750 con 0x7f6ae410c3e0 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6adbfff640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41038e0 0x7f6ae4198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae410c3e0 0x7f6ae419d020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009fd0 tx=0x7f6adc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6adb7fe640 1 -- 172.21.15.32:0/896976360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc017070 con 0x7f6ae410c3e0 2024-04-16T05:35:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6ae359e640 1 -- 172.21.15.32:0/896976360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae419da40 con 0x7f6ae410c3e0 2024-04-16T05:35:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.722+0000 7f6adb7fe640 1 -- 172.21.15.32:0/896976360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc003e50 con 0x7f6ae410c3e0 2024-04-16T05:35:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.723+0000 7f6adb7fe640 1 -- 172.21.15.32:0/896976360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0106c0 con 0x7f6ae410c3e0 2024-04-16T05:35:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.723+0000 7f6ae359e640 1 -- 172.21.15.32:0/896976360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae41a52d0 con 0x7f6ae410c3e0 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.725+0000 7f6adb7fe640 1 -- 172.21.15.32:0/896976360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6adc003970 con 0x7f6ae410c3e0 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.725+0000 7f6aeb5ec740 1 Processor -- start 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6aeb5ec740 1 -- start start 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6adb7fe640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ad00a3da0 0x7f6ad00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b3390 0x555e541c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555e541177d0 con 0x555e541c06d0 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555e540b4f70 con 0x555e541b2a70 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e54113790 con 0x555e541b3390 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6adbfff640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ad00a3da0 0x7f6ad00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6adb7fe640 1 -- 172.21.15.32:0/896976360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6adc0d0a00 con 0x7f6ae410c3e0 2024-04-16T05:35:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541c2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36580/0 (socket says 172.21.15.32:36580) 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/3128602033 learned_addr learned my addr 172.21.15.32:0/3128602033 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6adbfff640 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b3390 0x555e541c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/3128602033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b3390 msgr2=0x555e541c0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b3390 0x555e541c0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/3128602033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 msgr2=0x555e541b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/3128602033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e53e2a210 con 0x555e541c06d0 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6ae259c640 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.726+0000 7f6adbfff640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ad00a3da0 0x7f6ad00a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40fef20 tx=0x7f6acc02cc10 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.727+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541c2ac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6adc098950 tx=0x7f6adc004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f65380f5ef284d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.727+0000 7f6ad8ff9640 1 -- 172.21.15.32:0/3128602033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc020070 con 0x555e541c06d0 2024-04-16T05:35:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.727+0000 7f6ad8ff9640 1 -- 172.21.15.32:0/3128602033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc09cbf0 con 0x555e541c06d0 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.727+0000 7f6ad8ff9640 1 -- 172.21.15.32:0/3128602033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc09c4f0 con 0x555e541c06d0 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.727+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/3128602033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 msgr2=0x555e541c2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.727+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541c2ac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6adc098950 tx=0x7f6adc004940 comp rx=0 tx=0).stop 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/3128602033 shutdown_connections 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541c2ac0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b3390 0x555e541c0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/3128602033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/3128602033 >> 172.21.15.32:0/3128602033 conn(0x555e541ad4f0 msgr2=0x555e541ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/3128602033 shutdown_connections 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/3128602033 wait complete. 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 -- start start 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 Processor -- start 2024-04-16T05:35:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555e541c34b0 con 0x555e541c06d0 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555e5409fe90 con 0x555e541b3390 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e541c3630 con 0x555e541b2a70 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.728+0000 7f6aeb5ec740 10 client.0 _ll_register_callbacks cb 0x555e540d3cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541be190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36588/0 (socket says 172.21.15.32:36588) 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/4076321276 learned_addr learned my addr 172.21.15.32:0/4076321276 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6adbfff640 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae259c640 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/4076321276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 msgr2=0x555e541bd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 msgr2=0x555e541bdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/4076321276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e53ef6620 con 0x555e541c06d0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6adbfff640 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541be190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0ce410 tx=0x7f6adc0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50113f28f84f9594 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/4076321276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc020050 con 0x555e541c06d0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/4076321276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6adc09a7f0 con 0x555e541c06d0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/4076321276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6adc09cbf0 con 0x555e541c06d0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/4076321276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6adc09be50 con 0x555e541c06d0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6aeb5ec740 10 client.15693 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.729+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/4076321276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e541cf7e0 con 0x555e541c06d0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/4076321276 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc09cbf0 con 0x555e541c06d0 2024-04-16T05:35:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/4076321276 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6adc0ce000 con 0x555e541c06d0 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6aeb5ec740 10 client.15693 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6aeb5ec740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6aeb5ec740 20 client.15693 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555e541d0b30 0x555e541d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6aeb5ec740 4 client.15693 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/4076321276 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e540b7380 con 0x555e541d0b30 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.730+0000 7f6ae259c640 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555e541d0b30 0x555e541d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6ae259c640 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555e541d0b30 0x555e541d2f20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6a7892307c5153 server_cookie=9890ef2fc44d1bea in_seq=0 out_seq=0 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6ac9ffb640 10 client.15693 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/4076321276 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e540b7380 con 0x555e541d0b30 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6ac9ffb640 10 client.15693 handle_command_reply: tid=0 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 client.15693 shutdown 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/4076321276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 msgr2=0x555e541be190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541be190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0ce410 tx=0x7f6adc0042e0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/4076321276 shutdown_connections 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541be190 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555e541d0b30 0x555e541d2f20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/4076321276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/4076321276 >> 172.21.15.32:0/4076321276 conn(0x555e541ad4f0 msgr2=0x555e541af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.731+0000 7f6ac9ffb640 0 client.15693 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.732+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/4076321276 shutdown_connections 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.732+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/4076321276 wait complete. 2024-04-16T05:35:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.732+0000 7f6aeb5ec740 20 client.15693 trim_cache size 0 max 16384 2024-04-16T05:35:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6aeb5ec740 1 Processor -- start 2024-04-16T05:35:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6aeb5ec740 1 -- start start 2024-04-16T05:35:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41af900 con 0x555e541c06d0 2024-04-16T05:35:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ad1d0 con 0x555e541b3390 2024-04-16T05:35:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ab000 con 0x555e541b2a70 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36594/0 (socket says 172.21.15.32:36594) 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/1789308533 learned_addr learned my addr 172.21.15.32:0/1789308533 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae259c640 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6adbfff640 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/1789308533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 msgr2=0x555e541bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/1789308533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 msgr2=0x555e541bdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae2d9d640 1 -- 172.21.15.32:0/1789308533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e541b3850 con 0x555e541c06d0 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6adbfff640 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.747+0000 7f6ae259c640 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541d5f40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0bb3b0 tx=0x7f6adc09b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdce070b4b2cc24d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6acbfff640 1 -- 172.21.15.32:0/1789308533 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc014070 con 0x555e541c06d0 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6acbfff640 1 -- 172.21.15.32:0/1789308533 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc09cbf0 con 0x555e541c06d0 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6acbfff640 1 -- 172.21.15.32:0/1789308533 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc020070 con 0x555e541c06d0 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/1789308533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 msgr2=0x555e541d5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541d5f40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0bb3b0 tx=0x7f6adc09b940 comp rx=0 tx=0).stop 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/1789308533 shutdown_connections 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541c06d0 0x555e541d5f40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b3390 0x555e541bdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/1789308533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541b2a70 0x555e541bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/1789308533 >> 172.21.15.32:0/1789308533 conn(0x555e541ad4f0 msgr2=0x555e541ac550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/1789308533 shutdown_connections 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.748+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/1789308533 wait complete. 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 1 -- start start 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 1 Processor -- start 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541b3390 0x555e541d4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541c06d0 0x555e541d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ab690 con 0x555e541b3390 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41acb00 con 0x555e541b2a70 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41abd60 con 0x555e541c06d0 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6aeb5ec740 10 client.0 _ll_register_callbacks cb 0x7f6ad4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6adbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541b3390 0x555e541d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6adbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541b3390 0x555e541d4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36604/0 (socket says 172.21.15.32:36604) 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6adbfff640 1 -- 172.21.15.32:0/2456703287 learned_addr learned my addr 172.21.15.32:0/2456703287 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6ae259c640 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541c06d0 0x555e541d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6adbfff640 1 -- 172.21.15.32:0/2456703287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541c06d0 msgr2=0x555e541d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6adbfff640 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541c06d0 0x555e541d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6adbfff640 1 -- 172.21.15.32:0/2456703287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 msgr2=0x555e541d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6adbfff640 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6adbfff640 1 -- 172.21.15.32:0/2456703287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e54235f60 con 0x555e541b3390 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.749+0000 7f6ae2d9d640 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541c06d0 0x555e541d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6adbfff640 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541b3390 0x555e541d4120 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009900 tx=0x7f6acc02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abe51003e771924 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/2456703287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc051060 con 0x555e541b3390 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/2456703287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6acc004ad0 con 0x555e541b3390 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/2456703287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6adc09cbf0 con 0x555e541b3390 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/2456703287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6acc058050 con 0x555e541b3390 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 10 client.15699 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/2456703287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e542372a0 con 0x555e541b3390 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/2456703287 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc09cbf0 con 0x555e541b3390 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/2456703287 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6acc04b070 con 0x555e541b3390 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 10 client.15699 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 20 client.15699 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555e54237a80 0x555e54239ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 4 client.15699 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:35:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.750+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/2456703287 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e541251f0 con 0x555e54237a80 2024-04-16T05:35:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.751+0000 7f6ae259c640 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555e54237a80 0x555e54239ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.751+0000 7f6ae259c640 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555e54237a80 0x555e54239ea0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320a90aedf181637 server_cookie=9cbb09ec04270e87 in_seq=0 out_seq=0 2024-04-16T05:35:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.751+0000 7f6ac9ffb640 10 client.15699 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.764+0000 7f6ac9ffb640 1 -- 172.21.15.32:0/2456703287 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e541251f0 con 0x555e54237a80 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.764+0000 7f6ac9ffb640 10 client.15699 handle_command_reply: tid=0 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.764+0000 7f6aeb5ec740 1 client.15699 shutdown 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.764+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/2456703287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541b3390 msgr2=0x555e541d4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.764+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541b3390 0x555e541d4120 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009900 tx=0x7f6acc02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.764+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/2456703287 shutdown_connections 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.764+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555e54237a80 0x555e54239ea0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e541c06d0 0x555e541d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555e541b3390 0x555e541d4120 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ac9ffb640 0 client.15699 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6aeb5ec740 1 --2- 172.21.15.32:0/2456703287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e541b2a70 0x555e541d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/2456703287 >> 172.21.15.32:0/2456703287 conn(0x555e541ad4f0 msgr2=0x555e541cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/2456703287 shutdown_connections 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6aeb5ec740 1 -- 172.21.15.32:0/2456703287 wait complete. 2024-04-16T05:35:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6aeb5ec740 20 client.15699 trim_cache size 0 max 16384 2024-04-16T05:35:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2024-04-16T05:35:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2024-04-16T05:35:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2024-04-16T05:35:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2024-04-16T05:35:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 -- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ad00a3da0 msgr2=0x7f6ad00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ad00a3da0 0x7f6ad00a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40fef20 tx=0x7f6acc02cc10 comp rx=0 tx=0).stop 2024-04-16T05:35:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 -- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae410c3e0 msgr2=0x7f6ae419d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae410c3e0 0x7f6ae419d020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009fd0 tx=0x7f6adc00b260 comp rx=0 tx=0).stop 2024-04-16T05:35:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 -- 172.21.15.32:0/896976360 shutdown_connections 2024-04-16T05:35:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ad00a3da0 0x7f6ad00a6260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae410c3e0 0x7f6ae419d020 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41038e0 0x7f6ae4198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 --2- 172.21.15.32:0/896976360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4102fc0 0x7f6ae4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 -- 172.21.15.32:0/896976360 >> 172.21.15.32:0/896976360 conn(0x7f6ae40fc820 msgr2=0x7f6ae40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 -- 172.21.15.32:0/896976360 shutdown_connections 2024-04-16T05:35:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:16.765+0000 7f6ae359e640 1 -- 172.21.15.32:0/896976360 wait complete. 2024-04-16T05:35:17.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:17 smithi134 ceph-mon[36153]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-16T05:35:17.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:17 smithi082 ceph-mon[36195]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-16T05:35:17.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:17 smithi032 ceph-mon[32351]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-16T05:35:19.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:19 smithi134 ceph-mon[36153]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 44 op/s 2024-04-16T05:35:19.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:19 smithi082 ceph-mon[36195]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 44 op/s 2024-04-16T05:35:19.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:19 smithi032 ceph-mon[32351]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 44 op/s 2024-04-16T05:35:21.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:21 smithi134 ceph-mon[36153]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:21.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:21 smithi082 ceph-mon[36195]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:21.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:21 smithi032 ceph-mon[32351]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:23.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:23 smithi134 ceph-mon[36153]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:23.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:23 smithi082 ceph-mon[36195]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:23.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:23 smithi032 ceph-mon[32351]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:25.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:25 smithi134 ceph-mon[36153]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-16T05:35:25.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:25 smithi082 ceph-mon[36195]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-16T05:35:25.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:25 smithi032 ceph-mon[32351]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-16T05:35:27.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:27 smithi134 ceph-mon[36153]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:27.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:27 smithi082 ceph-mon[36195]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:27.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:27 smithi032 ceph-mon[32351]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T05:35:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 -- 172.21.15.32:0/442605890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c10c680 msgr2=0x7f817c10eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 --2- 172.21.15.32:0/442605890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c10c680 0x7f817c10eab0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8174009900 tx=0x7f817402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 -- 172.21.15.32:0/442605890 shutdown_connections 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 --2- 172.21.15.32:0/442605890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c10c680 0x7f817c10eab0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 --2- 172.21.15.32:0/442605890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c103bd0 0x7f817c10c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 --2- 172.21.15.32:0/442605890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c1032b0 0x7f817c103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 -- 172.21.15.32:0/442605890 >> 172.21.15.32:0/442605890 conn(0x7f817c077280 msgr2=0x7f817c077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 -- 172.21.15.32:0/442605890 shutdown_connections 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 -- 172.21.15.32:0/442605890 wait complete. 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 Processor -- start 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.865+0000 7f8181835640 1 -- start start 2024-04-16T05:35:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f8181835640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c1032b0 0x7f817c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f8181835640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c103bd0 0x7f817c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f8181835640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c10c680 0x7f817c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f8181835640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1123d0 con 0x7f817c1032b0 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f8181835640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f817c112250 con 0x7f817c10c680 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f8181835640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f817c112550 con 0x7f817c103bd0 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c1032b0 0x7f817c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c1032b0 0x7f817c198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41054/0 (socket says 172.21.15.32:41054) 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 -- 172.21.15.32:0/1599633128 learned_addr learned my addr 172.21.15.32:0/1599633128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817b7fe640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c10c680 0x7f817c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817a7fc640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c103bd0 0x7f817c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 -- 172.21.15.32:0/1599633128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c103bd0 msgr2=0x7f817c198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c103bd0 0x7f817c198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 -- 172.21.15.32:0/1599633128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c10c680 msgr2=0x7f817c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c10c680 0x7f817c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 -- 172.21.15.32:0/1599633128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c19d730 con 0x7f817c1032b0 2024-04-16T05:35:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.866+0000 7f817affd640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c1032b0 0x7f817c198790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f816c00cce0 tx=0x7f816c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.867+0000 7f8180833640 1 -- 172.21.15.32:0/1599633128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c00f650 con 0x7f817c1032b0 2024-04-16T05:35:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.867+0000 7f8180833640 1 -- 172.21.15.32:0/1599633128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816c00fcd0 con 0x7f817c1032b0 2024-04-16T05:35:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.867+0000 7f8180833640 1 -- 172.21.15.32:0/1599633128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c018c70 con 0x7f817c1032b0 2024-04-16T05:35:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.867+0000 7f8181835640 1 -- 172.21.15.32:0/1599633128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c19da20 con 0x7f817c1032b0 2024-04-16T05:35:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.867+0000 7f8181835640 1 -- 172.21.15.32:0/1599633128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c1a52b0 con 0x7f817c1032b0 2024-04-16T05:35:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f8180833640 1 -- 172.21.15.32:0/1599633128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f816c00f7f0 con 0x7f817c1032b0 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f818441d740 1 Processor -- start 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f818441d740 1 -- start start 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f818441d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14bc10 0x556cfd14bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f818441d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 0x556cfd154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f818441d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f818441d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556cfd0b07d0 con 0x556cfd155110 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f818441d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556cfd04df70 con 0x556cfd14bc10 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.869+0000 7f818441d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556cfd0ac790 con 0x556cfd14c530 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41056/0 (socket says 172.21.15.32:41056) 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 -- 172.21.15.32:0/24308698 learned_addr learned my addr 172.21.15.32:0/24308698 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f8180833640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f81680a3e60 0x7f81680a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817affd640 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 0x556cfd154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817b7fe640 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14bc10 0x556cfd14bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817b7fe640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f81680a3e60 0x7f81680a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 -- 172.21.15.32:0/24308698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 msgr2=0x556cfd154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 0x556cfd154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 -- 172.21.15.32:0/24308698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14bc10 msgr2=0x556cfd14bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14bc10 0x556cfd14bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 -- 172.21.15.32:0/24308698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfcdc3210 con 0x556cfd155110 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f8180833640 1 -- 172.21.15.32:0/1599633128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f816c0d11b0 con 0x7f817c1032b0 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817a7fc640 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd157500 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f816400bc10 tx=0x7f816400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89a78fd68937c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f8171ffb640 1 -- 172.21.15.32:0/24308698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164015070 con 0x556cfd155110 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f817b7fe640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f81680a3e60 0x7f81680a6320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f817402efd0 tx=0x7f8174005fd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f8171ffb640 1 -- 172.21.15.32:0/24308698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8164011920 con 0x556cfd155110 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.870+0000 7f8171ffb640 1 -- 172.21.15.32:0/24308698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164010910 con 0x556cfd155110 2024-04-16T05:35:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 -- 172.21.15.32:0/24308698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 msgr2=0x556cfd157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd157500 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f816400bc10 tx=0x7f816400f730 comp rx=0 tx=0).stop 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 -- 172.21.15.32:0/24308698 shutdown_connections 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd157500 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 0x556cfd154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 --2- 172.21.15.32:0/24308698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14bc10 0x556cfd14bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 -- 172.21.15.32:0/24308698 >> 172.21.15.32:0/24308698 conn(0x556cfd146340 msgr2=0x556cfd146720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 -- 172.21.15.32:0/24308698 shutdown_connections 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 -- 172.21.15.32:0/24308698 wait complete. 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.871+0000 7f818441d740 1 -- start start 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f818441d740 1 Processor -- start 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f818441d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 0x556cfd152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f818441d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 0x556cfd152680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f818441d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f818441d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556cfd157f00 con 0x556cfd155110 2024-04-16T05:35:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f818441d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556cfd038e90 con 0x556cfd14c530 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f818441d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556cfd158080 con 0x556cfd14bc10 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f818441d740 10 client.0 _ll_register_callbacks cb 0x556cfd06ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41072/0 (socket says 172.21.15.32:41072) 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f817b7fe640 1 -- 172.21.15.32:0/2327102596 learned_addr learned my addr 172.21.15.32:0/2327102596 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f817affd640 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 0x556cfd152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.872+0000 7f817a7fc640 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 0x556cfd152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f817b7fe640 1 -- 172.21.15.32:0/2327102596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 msgr2=0x556cfd152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f817b7fe640 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 0x556cfd152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f817b7fe640 1 -- 172.21.15.32:0/2327102596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 msgr2=0x556cfd152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f817b7fe640 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 0x556cfd152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f817b7fe640 1 -- 172.21.15.32:0/2327102596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfce8f620 con 0x556cfd155110 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f817b7fe640 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd152bc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8174004410 tx=0x7f8174009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896e01a504a64135 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f815affd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f815affd640 1 -- 172.21.15.32:0/2327102596 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817404f020 con 0x556cfd155110 2024-04-16T05:35:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f815affd640 1 -- 172.21.15.32:0/2327102596 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8174004ad0 con 0x556cfd155110 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f818441d740 1 -- 172.21.15.32:0/2327102596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8164011920 con 0x556cfd155110 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f815affd640 1 -- 172.21.15.32:0/2327102596 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f817405a050 con 0x556cfd155110 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f818441d740 10 client.15711 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f818441d740 1 -- 172.21.15.32:0/2327102596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cfd1641a0 con 0x556cfd155110 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.873+0000 7f815affd640 1 -- 172.21.15.32:0/2327102596 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8164011920 con 0x556cfd155110 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f815affd640 1 -- 172.21.15.32:0/2327102596 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f817404a070 con 0x556cfd155110 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f818441d740 10 client.15711 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f818441d740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f818441d740 20 client.15711 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f818441d740 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556cfd1656b0 0x556cfd167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f818441d740 4 client.15711 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f818441d740 1 -- 172.21.15.32:0/2327102596 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cfd050380 con 0x556cfd1656b0 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f817affd640 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556cfd1656b0 0x556cfd167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f817affd640 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556cfd1656b0 0x556cfd167aa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5645de93b11971e0 server_cookie=25da048401558448 in_seq=0 out_seq=0 2024-04-16T05:35:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.874+0000 7f815affd640 10 client.15711 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.875+0000 7f815affd640 1 -- 172.21.15.32:0/2327102596 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cfd050380 con 0x556cfd1656b0 2024-04-16T05:35:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.875+0000 7f815affd640 10 client.15711 handle_command_reply: tid=0 2024-04-16T05:35:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.875+0000 7f818441d740 1 client.15711 shutdown 2024-04-16T05:35:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.875+0000 7f818441d740 1 -- 172.21.15.32:0/2327102596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 msgr2=0x556cfd152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.875+0000 7f818441d740 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd152bc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8174004410 tx=0x7f8174009260 comp rx=0 tx=0).stop 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 1 -- 172.21.15.32:0/2327102596 shutdown_connections 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd152bc0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 0x556cfd152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556cfd1656b0 0x556cfd167aa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 1 --2- 172.21.15.32:0/2327102596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 0x556cfd152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f815affd640 0 client.15711 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 1 -- 172.21.15.32:0/2327102596 >> 172.21.15.32:0/2327102596 conn(0x556cfd146340 msgr2=0x556cfd148090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 1 -- 172.21.15.32:0/2327102596 shutdown_connections 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 1 -- 172.21.15.32:0/2327102596 wait complete. 2024-04-16T05:35:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.876+0000 7f818441d740 20 client.15711 trim_cache size 0 max 16384 2024-04-16T05:35:29.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:29 smithi134 ceph-mon[36153]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-16T05:35:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f818441d740 1 Processor -- start 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f818441d740 1 -- start start 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f818441d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd14bc10 0x556cfd160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f818441d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 0x556cfd148020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f818441d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd155110 0x556cfd152810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f818441d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1b1b50 con 0x556cfd14bc10 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f818441d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1af420 con 0x556cfd155110 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f818441d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1ad270 con 0x556cfd14c530 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd14bc10 0x556cfd160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd14bc10 0x556cfd160dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41088/0 (socket says 172.21.15.32:41088) 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817affd640 1 -- 172.21.15.32:0/591470726 learned_addr learned my addr 172.21.15.32:0/591470726 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817a7fc640 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 0x556cfd148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817b7fe640 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd155110 0x556cfd152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817affd640 1 -- 172.21.15.32:0/591470726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 msgr2=0x556cfd148020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817affd640 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 0x556cfd148020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817affd640 1 -- 172.21.15.32:0/591470726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd155110 msgr2=0x556cfd152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817affd640 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd155110 0x556cfd152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817affd640 1 -- 172.21.15.32:0/591470726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfd14af90 con 0x556cfd14bc10 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.891+0000 7f817b7fe640 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd155110 0x556cfd152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f817affd640 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd14bc10 0x556cfd160dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f816c0186d0 tx=0x7f816c0c7d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9889889878a9ed97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f81717fa640 1 -- 172.21.15.32:0/591470726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c0b5030 con 0x556cfd14bc10 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f81717fa640 1 -- 172.21.15.32:0/591470726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816c087040 con 0x556cfd14bc10 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f81717fa640 1 -- 172.21.15.32:0/591470726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c018e10 con 0x556cfd14bc10 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f818441d740 1 -- 172.21.15.32:0/591470726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd14bc10 msgr2=0x556cfd160dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f818441d740 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd14bc10 0x556cfd160dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f816c0186d0 tx=0x7f816c0c7d50 comp rx=0 tx=0).stop 2024-04-16T05:35:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f818441d740 1 -- 172.21.15.32:0/591470726 shutdown_connections 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f818441d740 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd155110 0x556cfd152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f818441d740 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14c530 0x556cfd148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f818441d740 1 --2- 172.21.15.32:0/591470726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd14bc10 0x556cfd160dc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.892+0000 7f818441d740 1 -- 172.21.15.32:0/591470726 >> 172.21.15.32:0/591470726 conn(0x556cfd146340 msgr2=0x556cfd152100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 -- 172.21.15.32:0/591470726 shutdown_connections 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 -- 172.21.15.32:0/591470726 wait complete. 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 -- start start 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 Processor -- start 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 0x556cfd15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 0x556cfd15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1ad8e0 con 0x556cfd155110 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1aed50 con 0x556cfd14c530 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1adfb0 con 0x556cfd14bc10 2024-04-16T05:35:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f818441d740 10 client.0 _ll_register_callbacks cb 0x7f8174005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd15a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41098/0 (socket says 172.21.15.32:41098) 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.893+0000 7f817b7fe640 1 -- 172.21.15.32:0/361685328 learned_addr learned my addr 172.21.15.32:0/361685328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f817a7fc640 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 0x556cfd15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f817affd640 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 0x556cfd15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f817b7fe640 1 -- 172.21.15.32:0/361685328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 msgr2=0x556cfd15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f817b7fe640 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 0x556cfd15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f817b7fe640 1 -- 172.21.15.32:0/361685328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 msgr2=0x556cfd15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f817b7fe640 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 0x556cfd15a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f817b7fe640 1 -- 172.21.15.32:0/361685328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfd157500 con 0x556cfd155110 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f817b7fe640 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd15a7e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8174003ae0 tx=0x7f81740038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2165350e71682599 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f815affd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f815affd640 1 -- 172.21.15.32:0/361685328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174063060 con 0x556cfd155110 2024-04-16T05:35:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f815affd640 1 -- 172.21.15.32:0/361685328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8174004ad0 con 0x556cfd155110 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f818441d740 1 -- 172.21.15.32:0/361685328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816c087040 con 0x556cfd155110 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f815affd640 1 -- 172.21.15.32:0/361685328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f817406a050 con 0x556cfd155110 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f818441d740 10 client.15717 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f818441d740 1 -- 172.21.15.32:0/361685328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cfd1cc860 con 0x556cfd155110 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.894+0000 7f815affd640 1 -- 172.21.15.32:0/361685328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816c087040 con 0x556cfd155110 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f815affd640 1 -- 172.21.15.32:0/361685328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f817404a070 con 0x556cfd155110 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f818441d740 10 client.15717 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f818441d740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f818441d740 20 client.15717 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f818441d740 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556cfd1ccfe0 0x556cfd1cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f818441d740 4 client.15717 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f818441d740 1 -- 172.21.15.32:0/361685328 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cfd0be1f0 con 0x556cfd1ccfe0 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f817affd640 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556cfd1ccfe0 0x556cfd1cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f815affd640 10 client.15717 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.895+0000 7f817affd640 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556cfd1ccfe0 0x556cfd1cf400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23a54e1de503ba server_cookie=888404ec04b087cb in_seq=0 out_seq=0 2024-04-16T05:35:29.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:29 smithi082 ceph-mon[36195]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-16T05:35:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f815affd640 1 -- 172.21.15.32:0/361685328 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cfd0be1f0 con 0x556cfd1ccfe0 2024-04-16T05:35:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f815affd640 10 client.15717 handle_command_reply: tid=0 2024-04-16T05:35:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 client.15717 shutdown 2024-04-16T05:35:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 -- 172.21.15.32:0/361685328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 msgr2=0x556cfd15a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd15a7e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8174003ae0 tx=0x7f81740038a0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 -- 172.21.15.32:0/361685328 shutdown_connections 2024-04-16T05:35:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556cfd155110 0x556cfd15a7e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556cfd14c530 0x556cfd15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556cfd1ccfe0 0x556cfd1cf400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 --2- 172.21.15.32:0/361685328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556cfd14bc10 0x556cfd15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f815affd640 0 client.15717 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 -- 172.21.15.32:0/361685328 >> 172.21.15.32:0/361685328 conn(0x556cfd146340 msgr2=0x556cfd152100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 -- 172.21.15.32:0/361685328 shutdown_connections 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 1 -- 172.21.15.32:0/361685328 wait complete. 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.902+0000 7f818441d740 20 client.15717 trim_cache size 0 max 16384 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2024-04-16T05:35:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2024-04-16T05:35:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 -- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f81680a3e60 msgr2=0x7f81680a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f81680a3e60 0x7f81680a6320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f817402efd0 tx=0x7f8174005fd0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 -- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c1032b0 msgr2=0x7f817c198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c1032b0 0x7f817c198790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f816c00cce0 tx=0x7f816c00a620 comp rx=0 tx=0).stop 2024-04-16T05:35:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 -- 172.21.15.32:0/1599633128 shutdown_connections 2024-04-16T05:35:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c10c680 0x7f817c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c103bd0 0x7f817c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f81680a3e60 0x7f81680a6320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 --2- 172.21.15.32:0/1599633128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f817c1032b0 0x7f817c198790 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 -- 172.21.15.32:0/1599633128 >> 172.21.15.32:0/1599633128 conn(0x7f817c077280 msgr2=0x7f817c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 -- 172.21.15.32:0/1599633128 shutdown_connections 2024-04-16T05:35:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:29.903+0000 7f8181835640 1 -- 172.21.15.32:0/1599633128 wait complete. 2024-04-16T05:35:29.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:29 smithi032 ceph-mon[32351]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-16T05:35:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:31 smithi134 ceph-mon[36153]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:35:31.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:31 smithi082 ceph-mon[36195]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:35:31.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:31 smithi032 ceph-mon[32351]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:35:32.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:35:32.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:35:32.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 -- 172.21.15.32:0/3131443998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90106220 msgr2=0x7fca9010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 --2- 172.21.15.32:0/3131443998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90106220 0x7fca9010ddf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fca88009ca0 tx=0x7fca8802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 -- 172.21.15.32:0/3131443998 shutdown_connections 2024-04-16T05:35:32.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 --2- 172.21.15.32:0/3131443998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90106220 0x7fca9010ddf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 --2- 172.21.15.32:0/3131443998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca90101b00 0x7fca90105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 --2- 172.21.15.32:0/3131443998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca90101150 0x7fca90101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 -- 172.21.15.32:0/3131443998 >> 172.21.15.32:0/3131443998 conn(0x7fca90077f60 msgr2=0x7fca90078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 -- 172.21.15.32:0/3131443998 shutdown_connections 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 -- 172.21.15.32:0/3131443998 wait complete. 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.344+0000 7fca965cd640 1 Processor -- start 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca965cd640 1 -- start start 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca965cd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90101150 0x7fca901996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca965cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca90106220 0x7fca90199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca965cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca9019a170 0x7fca901a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca965cd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca901112c0 con 0x7fca90101150 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca965cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca90111140 con 0x7fca90106220 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca965cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca90111440 con 0x7fca9019a170 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90101150 0x7fca901996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90101150 0x7fca901996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41104/0 (socket says 172.21.15.32:41104) 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 -- 172.21.15.32:0/1846500680 learned_addr learned my addr 172.21.15.32:0/1846500680 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca87fff640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca90106220 0x7fca90199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca94b6a640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca9019a170 0x7fca901a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 -- 172.21.15.32:0/1846500680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca9019a170 msgr2=0x7fca901a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca9019a170 0x7fca901a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 -- 172.21.15.32:0/1846500680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca90106220 msgr2=0x7fca90199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca90106220 0x7fca90199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 -- 172.21.15.32:0/1846500680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca901a4ef0 con 0x7fca90101150 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.345+0000 7fca8ffff640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90101150 0x7fca901996a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fca8000e9f0 tx=0x7fca8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.346+0000 7fca8dffb640 1 -- 172.21.15.32:0/1846500680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8000cd80 con 0x7fca90101150 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.346+0000 7fca965cd640 1 -- 172.21.15.32:0/1846500680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca901a51e0 con 0x7fca90101150 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.346+0000 7fca8dffb640 1 -- 172.21.15.32:0/1846500680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca80010910 con 0x7fca90101150 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.346+0000 7fca8dffb640 1 -- 172.21.15.32:0/1846500680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800188f0 con 0x7fca90101150 2024-04-16T05:35:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.346+0000 7fca965cd640 1 -- 172.21.15.32:0/1846500680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca901a5780 con 0x7fca90101150 2024-04-16T05:35:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.348+0000 7fca8dffb640 1 -- 172.21.15.32:0/1846500680 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fca80010ac0 con 0x7fca90101150 2024-04-16T05:35:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.348+0000 7fca991bc740 1 Processor -- start 2024-04-16T05:35:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.348+0000 7fca991bc740 1 -- start start 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca991bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca991bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca991bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca991bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562a1cd54180 con 0x562a1cdef770 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca991bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562a1ccf1360 con 0x562a1cdeee50 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8dffb640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca7c0a3db0 0x7fca7c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca991bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562a1cd50140 con 0x562a1cdf8350 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41116/0 (socket says 172.21.15.32:41116) 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca94b6a640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca7c0a3db0 0x7fca7c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 -- 172.21.15.32:0/3240097920 learned_addr learned my addr 172.21.15.32:0/3240097920 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8dffb640 1 -- 172.21.15.32:0/1846500680 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca800d1190 con 0x7fca90101150 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca94b6a640 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca87fff640 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 -- 172.21.15.32:0/3240097920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 msgr2=0x562a1cdfa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 -- 172.21.15.32:0/3240097920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 msgr2=0x562a1cdef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdef230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 -- 172.21.15.32:0/3240097920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1cb307d0 con 0x562a1cdef770 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca94b6a640 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca87fff640 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.349+0000 7fca8ffff640 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf7e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fca8009cbb0 tx=0x7fca8006e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd717a275452556e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca94b6a640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca7c0a3db0 0x7fca7c0a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca88009430 tx=0x7fca8802e040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca867fc640 1 -- 172.21.15.32:0/3240097920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80026040 con 0x562a1cdef770 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca867fc640 1 -- 172.21.15.32:0/3240097920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca800a0fd0 con 0x562a1cdef770 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca867fc640 1 -- 172.21.15.32:0/3240097920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800ce930 con 0x562a1cdef770 2024-04-16T05:35:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 -- 172.21.15.32:0/3240097920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 msgr2=0x562a1cdf7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf7e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fca8009cbb0 tx=0x7fca8006e720 comp rx=0 tx=0).stop 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 -- 172.21.15.32:0/3240097920 shutdown_connections 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf7e10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 --2- 172.21.15.32:0/3240097920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 -- 172.21.15.32:0/3240097920 >> 172.21.15.32:0/3240097920 conn(0x562a1cde9580 msgr2=0x562a1cde9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 -- 172.21.15.32:0/3240097920 shutdown_connections 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.350+0000 7fca991bc740 1 -- 172.21.15.32:0/3240097920 wait complete. 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 1 -- start start 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 1 Processor -- start 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdf5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562a1cdfb0e0 con 0x562a1cdef770 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562a1cd0d2c0 con 0x562a1cdeee50 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562a1cdfb260 con 0x562a1cdf8350 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca991bc740 10 client.0 _ll_register_callbacks cb 0x562a1ccf4090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41130/0 (socket says 172.21.15.32:41130) 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca87fff640 1 -- 172.21.15.32:0/491781870 learned_addr learned my addr 172.21.15.32:0/491781870 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.351+0000 7fca94b6a640 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdf5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca8ffff640 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca87fff640 1 -- 172.21.15.32:0/491781870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 msgr2=0x562a1cdf5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca87fff640 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdf5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca87fff640 1 -- 172.21.15.32:0/491781870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 msgr2=0x562a1cdf53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca87fff640 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca87fff640 1 -- 172.21.15.32:0/491781870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1ca663f0 con 0x562a1cdef770 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca94b6a640 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdf5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca87fff640 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf5930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fca78012780 tx=0x7fca78012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670262627f0a89e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca6f7fe640 1 -- 172.21.15.32:0/491781870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78010e80 con 0x562a1cdef770 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca6f7fe640 1 -- 172.21.15.32:0/491781870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca78004020 con 0x562a1cdef770 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca991bc740 1 -- 172.21.15.32:0/491781870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca800a0fd0 con 0x562a1cdef770 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca6f7fe640 1 -- 172.21.15.32:0/491781870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca78024050 con 0x562a1cdef770 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca991bc740 10 client.15729 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca991bc740 1 -- 172.21.15.32:0/491781870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a1ca260f0 con 0x562a1cdef770 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.352+0000 7fca6f7fe640 1 -- 172.21.15.32:0/491781870 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca800a0fd0 con 0x562a1cdef770 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca6f7fe640 1 -- 172.21.15.32:0/491781870 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca7801a430 con 0x562a1cdef770 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca991bc740 10 client.15729 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca991bc740 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca991bc740 20 client.15729 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca991bc740 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x562a1ce08820 0x562a1ce0ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca991bc740 4 client.15729 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca991bc740 1 -- 172.21.15.32:0/491781870 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a1cd61ba0 con 0x562a1ce08820 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca8ffff640 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x562a1ce08820 0x562a1ce0ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca8ffff640 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x562a1ce08820 0x562a1ce0ac40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5097b9e36ffc7f87 server_cookie=699481cc835adf28 in_seq=0 out_seq=0 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.353+0000 7fca6f7fe640 10 client.15729 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca6f7fe640 1 -- 172.21.15.32:0/491781870 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a1cd61ba0 con 0x562a1ce08820 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca6f7fe640 10 client.15729 handle_command_reply: tid=0 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 client.15729 shutdown 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 -- 172.21.15.32:0/491781870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 msgr2=0x562a1cdf5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf5930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fca78012780 tx=0x7fca78012c40 comp rx=0 tx=0).stop 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 -- 172.21.15.32:0/491781870 shutdown_connections 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x562a1ce08820 0x562a1ce0ac40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdf5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1cdf5930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca6f7fe640 0 client.15729 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 --2- 172.21.15.32:0/491781870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 -- 172.21.15.32:0/491781870 >> 172.21.15.32:0/491781870 conn(0x562a1cde9580 msgr2=0x562a1cdeb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 -- 172.21.15.32:0/491781870 shutdown_connections 2024-04-16T05:35:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 1 -- 172.21.15.32:0/491781870 wait complete. 2024-04-16T05:35:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.354+0000 7fca991bc740 20 client.15729 trim_cache size 0 max 16384 2024-04-16T05:35:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca991bc740 1 Processor -- start 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca991bc740 1 -- start start 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca991bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca991bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1ce03fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca991bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdf4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca991bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca901b1c40 con 0x562a1cdef770 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca991bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca901af510 con 0x562a1cdeee50 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca991bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca901ad360 con 0x562a1cdf8350 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1ce03fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1ce03fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41138/0 (socket says 172.21.15.32:41138) 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.372+0000 7fca87fff640 1 -- 172.21.15.32:0/2140925475 learned_addr learned my addr 172.21.15.32:0/2140925475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca8ffff640 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca87fff640 1 -- 172.21.15.32:0/2140925475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 msgr2=0x562a1cdf4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca87fff640 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdf4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca87fff640 1 -- 172.21.15.32:0/2140925475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 msgr2=0x562a1cdf7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca87fff640 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca87fff640 1 -- 172.21.15.32:0/2140925475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1ce07320 con 0x562a1cdef770 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca8ffff640 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca87fff640 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1ce03fa0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fca7800a520 tx=0x7fca78005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b292ea964c85975e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca85ffb640 1 -- 172.21.15.32:0/2140925475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7802c070 con 0x562a1cdef770 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca85ffb640 1 -- 172.21.15.32:0/2140925475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca78010c80 con 0x562a1cdef770 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.373+0000 7fca85ffb640 1 -- 172.21.15.32:0/2140925475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca780052d0 con 0x562a1cdef770 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 -- 172.21.15.32:0/2140925475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 msgr2=0x562a1ce03fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1ce03fa0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fca7800a520 tx=0x7fca78005a10 comp rx=0 tx=0).stop 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 -- 172.21.15.32:0/2140925475 shutdown_connections 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdf4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdef770 0x562a1ce03fa0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 --2- 172.21.15.32:0/2140925475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdeee50 0x562a1cdf7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 -- 172.21.15.32:0/2140925475 >> 172.21.15.32:0/2140925475 conn(0x562a1cde9580 msgr2=0x562a1cde7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 -- 172.21.15.32:0/2140925475 shutdown_connections 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 -- 172.21.15.32:0/2140925475 wait complete. 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 -- start start 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 Processor -- start 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdeee50 0x562a1ce66060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.374+0000 7fca991bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdef770 0x562a1ce665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca991bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca991bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca901ad9d0 con 0x562a1cdeee50 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca991bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca901aee40 con 0x562a1cdef770 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca991bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca901ae0a0 con 0x562a1cdf8350 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca991bc740 10 client.0 _ll_register_callbacks cb 0x7fca7800a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdeee50 0x562a1ce66060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdeee50 0x562a1ce66060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41152/0 (socket says 172.21.15.32:41152) 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 -- 172.21.15.32:0/3912111584 learned_addr learned my addr 172.21.15.32:0/3912111584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca94b6a640 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca87fff640 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdef770 0x562a1ce665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 -- 172.21.15.32:0/3912111584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 msgr2=0x562a1cdfd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 -- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdef770 msgr2=0x562a1ce665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdef770 0x562a1ce665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 -- 172.21.15.32:0/3912111584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1cdfe010 con 0x562a1cdeee50 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca87fff640 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdef770 0x562a1ce665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca8ffff640 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdeee50 0x562a1ce66060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fca8009c920 tx=0x7fca8001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dd2bd0175bd72a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca6f7fe640 1 -- 172.21.15.32:0/3912111584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800a0fd0 con 0x562a1cdeee50 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca6f7fe640 1 -- 172.21.15.32:0/3912111584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca800bd0b0 con 0x562a1cdeee50 2024-04-16T05:35:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca991bc740 1 -- 172.21.15.32:0/3912111584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca78010c80 con 0x562a1cdeee50 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca6f7fe640 1 -- 172.21.15.32:0/3912111584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca800ce730 con 0x562a1cdeee50 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca991bc740 10 client.15735 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.375+0000 7fca991bc740 1 -- 172.21.15.32:0/3912111584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a1ce6e7e0 con 0x562a1cdeee50 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca6f7fe640 1 -- 172.21.15.32:0/3912111584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca78010c80 con 0x562a1cdeee50 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca6f7fe640 1 -- 172.21.15.32:0/3912111584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca8009c100 con 0x562a1cdeee50 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca991bc740 10 client.15735 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca991bc740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca991bc740 20 client.15735 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca991bc740 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x562a1ce6eb20 0x562a1ce70f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca991bc740 4 client.15735 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca991bc740 1 -- 172.21.15.32:0/3912111584 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562a1cd5ba10 con 0x562a1ce6eb20 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca87fff640 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x562a1ce6eb20 0x562a1ce70f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca6f7fe640 10 client.15735 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.376+0000 7fca87fff640 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x562a1ce6eb20 0x562a1ce70f10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f889a1944585c57 server_cookie=d8feb6c91146c6 in_seq=0 out_seq=0 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca6f7fe640 1 -- 172.21.15.32:0/3912111584 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562a1cd5ba10 con 0x562a1ce6eb20 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca6f7fe640 10 client.15735 handle_command_reply: tid=0 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 client.15735 shutdown 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 -- 172.21.15.32:0/3912111584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdeee50 msgr2=0x562a1ce66060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdeee50 0x562a1ce66060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fca8009c920 tx=0x7fca8001f000 comp rx=0 tx=0).stop 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 -- 172.21.15.32:0/3912111584 shutdown_connections 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x562a1ce6eb20 0x562a1ce70f10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a1cdf8350 0x562a1cdfd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a1cdef770 0x562a1ce665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca6f7fe640 0 client.15735 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 --2- 172.21.15.32:0/3912111584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a1cdeee50 0x562a1ce66060 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 -- 172.21.15.32:0/3912111584 >> 172.21.15.32:0/3912111584 conn(0x562a1cde9580 msgr2=0x562a1cde7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 -- 172.21.15.32:0/3912111584 shutdown_connections 2024-04-16T05:35:32.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 1 -- 172.21.15.32:0/3912111584 wait complete. 2024-04-16T05:35:32.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca991bc740 20 client.15735 trim_cache size 0 max 16384 2024-04-16T05:35:32.383 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:35:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:35:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:35:32.383 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:35:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca965cd640 1 -- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca7c0a3db0 msgr2=0x7fca7c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca965cd640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca7c0a3db0 0x7fca7c0a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca88009430 tx=0x7fca8802e040 comp rx=0 tx=0).stop 2024-04-16T05:35:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca965cd640 1 -- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90101150 msgr2=0x7fca901996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.377+0000 7fca965cd640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90101150 0x7fca901996a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fca8000e9f0 tx=0x7fca8000eeb0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.378+0000 7fca965cd640 1 -- 172.21.15.32:0/1846500680 shutdown_connections 2024-04-16T05:35:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.378+0000 7fca965cd640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca9019a170 0x7fca901a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.378+0000 7fca965cd640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca90106220 0x7fca90199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.378+0000 7fca965cd640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca7c0a3db0 0x7fca7c0a6270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.378+0000 7fca965cd640 1 --2- 172.21.15.32:0/1846500680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca90101150 0x7fca901996a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.378+0000 7fca965cd640 1 -- 172.21.15.32:0/1846500680 >> 172.21.15.32:0/1846500680 conn(0x7fca90077f60 msgr2=0x7fca900fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.378+0000 7fca965cd640 1 -- 172.21.15.32:0/1846500680 shutdown_connections 2024-04-16T05:35:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.378+0000 7fca965cd640 1 -- 172.21.15.32:0/1846500680 wait complete. 2024-04-16T05:35:32.390 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:35:32.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 -- 172.21.15.32:0/2421189201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3238101b00 msgr2=0x7f3238105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 --2- 172.21.15.32:0/2421189201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3238101b00 0x7f3238105b40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3228009900 tx=0x7f322802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 -- 172.21.15.32:0/2421189201 shutdown_connections 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 --2- 172.21.15.32:0/2421189201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3238106220 0x7f323810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 --2- 172.21.15.32:0/2421189201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3238101b00 0x7f3238105b40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 --2- 172.21.15.32:0/2421189201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3238101150 0x7f3238101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 -- 172.21.15.32:0/2421189201 >> 172.21.15.32:0/2421189201 conn(0x7f3238078000 msgr2=0x7f3238078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 -- 172.21.15.32:0/2421189201 shutdown_connections 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.497+0000 7f323f8e4640 1 -- 172.21.15.32:0/2421189201 wait complete. 2024-04-16T05:35:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323f8e4640 1 Processor -- start 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323f8e4640 1 -- start start 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323f8e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3238101150 0x7f3238199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323f8e4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3238106220 0x7f3238199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323f8e4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323819a1e0 0x7f32381a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323f8e4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3238111290 con 0x7f323819a1e0 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323f8e4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3238111110 con 0x7f3238106220 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323f8e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3238111410 con 0x7f3238101150 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323de81640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323819a1e0 0x7f32381a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323de81640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323819a1e0 0x7f32381a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41172/0 (socket says 172.21.15.32:41172) 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323de81640 1 -- 172.21.15.32:0/1884285355 learned_addr learned my addr 172.21.15.32:0/1884285355 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323d680640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3238101150 0x7f3238199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323ce7f640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3238106220 0x7f3238199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323de81640 1 -- 172.21.15.32:0/1884285355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3238101150 msgr2=0x7f3238199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323de81640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3238101150 0x7f3238199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323de81640 1 -- 172.21.15.32:0/1884285355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3238106220 msgr2=0x7f3238199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323de81640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3238106220 0x7f3238199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.498+0000 7f323de81640 1 -- 172.21.15.32:0/1884285355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32381a4f90 con 0x7f323819a1e0 2024-04-16T05:35:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.499+0000 7f323de81640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323819a1e0 0x7f32381a4790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f323000eb10 tx=0x7f323000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.499+0000 7f322e7fc640 1 -- 172.21.15.32:0/1884285355 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230004b20 con 0x7f323819a1e0 2024-04-16T05:35:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.499+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32381a5280 con 0x7f323819a1e0 2024-04-16T05:35:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.499+0000 7f322e7fc640 1 -- 172.21.15.32:0/1884285355 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3230004cc0 con 0x7f323819a1e0 2024-04-16T05:35:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.499+0000 7f322e7fc640 1 -- 172.21.15.32:0/1884285355 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230010760 con 0x7f323819a1e0 2024-04-16T05:35:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.499+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32381a5750 con 0x7f323819a1e0 2024-04-16T05:35:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.500+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3238068e20 con 0x7f323819a1e0 2024-04-16T05:35:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.501+0000 7f322e7fc640 1 -- 172.21.15.32:0/1884285355 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f3230005000 con 0x7f323819a1e0 2024-04-16T05:35:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.502+0000 7f322e7fc640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f32240a3db0 0x7f32240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.502+0000 7f322e7fc640 1 -- 172.21.15.32:0/1884285355 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f32300d1400 con 0x7f323819a1e0 2024-04-16T05:35:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.502+0000 7f323d680640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f32240a3db0 0x7f32240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.502+0000 7f323d680640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f32240a3db0 0x7f32240a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f323400bbe0 tx=0x7f323400b390 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.506+0000 7f322e7fc640 1 -- 172.21.15.32:0/1884285355 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3230053ff0 con 0x7f323819a1e0 2024-04-16T05:35:32.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.733+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3238101b00 con 0x7f323819a1e0 2024-04-16T05:35:32.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.733+0000 7f322e7fc640 1 -- 172.21.15.32:0/1884285355 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f323009d5f0 con 0x7f323819a1e0 2024-04-16T05:35:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:35:32.732 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:35:32.732 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f32240a3db0 msgr2=0x7f32240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f32240a3db0 0x7f32240a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f323400bbe0 tx=0x7f323400b390 comp rx=0 tx=0).stop 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323819a1e0 msgr2=0x7f32381a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323819a1e0 0x7f32381a4790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f323000eb10 tx=0x7f323000c2d0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 shutdown_connections 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f32240a3db0 0x7f32240a6270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323819a1e0 0x7f32381a4790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3238106220 0x7f3238199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 --2- 172.21.15.32:0/1884285355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3238101150 0x7f3238199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 >> 172.21.15.32:0/1884285355 conn(0x7f3238078000 msgr2=0x7f32380ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 shutdown_connections 2024-04-16T05:35:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.736+0000 7f323f8e4640 1 -- 172.21.15.32:0/1884285355 wait complete. 2024-04-16T05:35:32.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 -- 172.21.15.32:0/2086537888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20105e80 msgr2=0x7f7e20106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 --2- 172.21.15.32:0/2086537888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20105e80 0x7f7e20106260 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009900 tx=0x7f7e1c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 -- 172.21.15.32:0/2086537888 shutdown_connections 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 --2- 172.21.15.32:0/2086537888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e20101af0 0x7f7e2010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 --2- 172.21.15.32:0/2086537888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e20101150 0x7f7e201015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 --2- 172.21.15.32:0/2086537888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20105e80 0x7f7e20106260 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 -- 172.21.15.32:0/2086537888 >> 172.21.15.32:0/2086537888 conn(0x7f7e20077cf0 msgr2=0x7f7e20078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 -- 172.21.15.32:0/2086537888 shutdown_connections 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.855+0000 7f7e2761b640 1 -- 172.21.15.32:0/2086537888 wait complete. 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e2761b640 1 Processor -- start 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e2761b640 1 -- start start 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e2761b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20101150 0x7f7e2019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e2761b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e20101af0 0x7f7e2019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e2761b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e20105e80 0x7f7e20198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e2761b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201124d0 con 0x7f7e20101150 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e2761b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20112350 con 0x7f7e20105e80 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e2761b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20112650 con 0x7f7e20101af0 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20101150 0x7f7e2019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20101150 0x7f7e2019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41190/0 (socket says 172.21.15.32:41190) 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 -- 172.21.15.32:0/1863088505 learned_addr learned my addr 172.21.15.32:0/1863088505 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e24bb6640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e20101af0 0x7f7e2019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 -- 172.21.15.32:0/1863088505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e20101af0 msgr2=0x7f7e2019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e20101af0 0x7f7e2019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 -- 172.21.15.32:0/1863088505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e20105e80 msgr2=0x7f7e20198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e20105e80 0x7f7e20198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e20105e80 0x7f7e20198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 -- 172.21.15.32:0/1863088505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e20198fd0 con 0x7f7e20101150 2024-04-16T05:35:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.856+0000 7f7e253b7640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20101150 0x7f7e2019e650 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c006010 tx=0x7f7e1c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.857+0000 7f7e167fc640 1 -- 172.21.15.32:0/1863088505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c041070 con 0x7f7e20101150 2024-04-16T05:35:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.857+0000 7f7e167fc640 1 -- 172.21.15.32:0/1863088505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e1c004450 con 0x7f7e20101150 2024-04-16T05:35:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.857+0000 7f7e2761b640 1 -- 172.21.15.32:0/1863088505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e20199260 con 0x7f7e20101150 2024-04-16T05:35:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.857+0000 7f7e167fc640 1 -- 172.21.15.32:0/1863088505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c003e00 con 0x7f7e20101150 2024-04-16T05:35:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.857+0000 7f7e2761b640 1 -- 172.21.15.32:0/1863088505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e201a53f0 con 0x7f7e20101150 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.859+0000 7f7e167fc640 1 -- 172.21.15.32:0/1863088505 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7e1c044070 con 0x7f7e20101150 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.859+0000 7f7e2a209740 1 Processor -- start 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e2a209740 1 -- start start 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fadb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 0x55a79fae6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e2a209740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a79fa400e0 con 0x55a79fadb780 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e2a209740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a79f9dd520 con 0x55a79fadae60 2024-04-16T05:35:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e2a209740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a79fa3c0a0 con 0x55a79fae4360 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e167fc640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7e0c0a3db0 0x7f7e0c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7e0c0a3db0 0x7f7e0c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41192/0 (socket says 172.21.15.32:41192) 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 -- 172.21.15.32:0/626151251 learned_addr learned my addr 172.21.15.32:0/626151251 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e24bb6640 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 0x55a79fae6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fadb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e167fc640 1 -- 172.21.15.32:0/1863088505 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7e1c0f2770 con 0x7f7e20101150 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 -- 172.21.15.32:0/626151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 msgr2=0x55a79fae6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 0x55a79fae6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 -- 172.21.15.32:0/626151251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 msgr2=0x55a79fadb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fadb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 -- 172.21.15.32:0/626151251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a79fa058d0 con 0x55a79fadb780 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7e0c0a3db0 0x7f7e0c0a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e18005fd0 tx=0x7f7e18009b30 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.860+0000 7f7e253b7640 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae3e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0ba7d0 tx=0x7f7e1c090e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb2624ba7deb5b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e0b7fe640 1 -- 172.21.15.32:0/626151251 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c04b070 con 0x55a79fadb780 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e0b7fe640 1 -- 172.21.15.32:0/626151251 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e1c02cd20 con 0x55a79fadb780 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e0b7fe640 1 -- 172.21.15.32:0/626151251 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0be370 con 0x55a79fadb780 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 -- 172.21.15.32:0/626151251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 msgr2=0x55a79fae3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae3e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0ba7d0 tx=0x7f7e1c090e60 comp rx=0 tx=0).stop 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 -- 172.21.15.32:0/626151251 shutdown_connections 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 0x55a79fae6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae3e20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 --2- 172.21.15.32:0/626151251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fadb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 -- 172.21.15.32:0/626151251 >> 172.21.15.32:0/626151251 conn(0x55a79fad5590 msgr2=0x55a79fad5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 -- 172.21.15.32:0/626151251 shutdown_connections 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.861+0000 7f7e2a209740 1 -- 172.21.15.32:0/626151251 wait complete. 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 1 -- start start 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 1 Processor -- start 2024-04-16T05:35:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fae1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 0x55a79fae1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a79fa400e0 con 0x55a79fadb780 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a79f9f94d0 con 0x55a79fadae60 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a79fae6f10 con 0x55a79fae4360 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e2a209740 10 client.0 _ll_register_callbacks cb 0x55a79f9de2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e24bb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e24bb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41208/0 (socket says 172.21.15.32:41208) 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.862+0000 7f7e24bb6640 1 -- 172.21.15.32:0/4292976152 learned_addr learned my addr 172.21.15.32:0/4292976152 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e24bb6640 1 -- 172.21.15.32:0/4292976152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 msgr2=0x55a79fae1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e24bb6640 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 0x55a79fae1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e253b7640 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fae1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e24bb6640 1 -- 172.21.15.32:0/4292976152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 msgr2=0x55a79fae1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e24bb6640 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fae1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e24bb6640 1 -- 172.21.15.32:0/4292976152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a79f7523f0 con 0x55a79fadb780 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e253b7640 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fae1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e24bb6640 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae19a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7e10012700 tx=0x7f7e10012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81924951cfd03682 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e08ff9640 1 -- 172.21.15.32:0/4292976152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10004260 con 0x55a79fadb780 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e08ff9640 1 -- 172.21.15.32:0/4292976152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7e10004400 con 0x55a79fadb780 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 1 -- 172.21.15.32:0/4292976152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1c02cd20 con 0x55a79fadb780 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e08ff9640 1 -- 172.21.15.32:0/4292976152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e100119a0 con 0x55a79fadb780 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 10 client.15753 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 1 -- 172.21.15.32:0/4292976152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a79f81c7d0 con 0x55a79fadb780 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e08ff9640 1 -- 172.21.15.32:0/4292976152 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e1c02cd20 con 0x55a79fadb780 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e08ff9640 1 -- 172.21.15.32:0/4292976152 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7e1000f440 con 0x55a79fadb780 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 10 client.15753 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 10 client.15753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:35:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 20 client.15753 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55a79faf40f0 0x55a79faf6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 4 client.15753 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.863+0000 7f7e2a209740 1 -- 172.21.15.32:0/4292976152 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a79fa4db00 con 0x55a79faf40f0 2024-04-16T05:35:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.864+0000 7f7e253b7640 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55a79faf40f0 0x55a79faf6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.864+0000 7f7e253b7640 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55a79faf40f0 0x55a79faf6560 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec9168bacaa37a34 server_cookie=602dc05536a6c278 in_seq=0 out_seq=0 2024-04-16T05:35:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.864+0000 7f7e08ff9640 10 client.15753 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e08ff9640 1 -- 172.21.15.32:0/4292976152 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a79fa4db00 con 0x55a79faf40f0 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e08ff9640 10 client.15753 handle_command_reply: tid=0 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 client.15753 shutdown 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 -- 172.21.15.32:0/4292976152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 msgr2=0x55a79fae19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae19a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7e10012700 tx=0x7f7e10012bc0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 -- 172.21.15.32:0/4292976152 shutdown_connections 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55a79faf40f0 0x55a79faf6560 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fae4360 0x55a79fae1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e08ff9640 0 client.15753 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:35:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fadb780 0x55a79fae19a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 --2- 172.21.15.32:0/4292976152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79fae1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 -- 172.21.15.32:0/4292976152 >> 172.21.15.32:0/4292976152 conn(0x55a79fad5590 msgr2=0x55a79fad72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 -- 172.21.15.32:0/4292976152 shutdown_connections 2024-04-16T05:35:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 1 -- 172.21.15.32:0/4292976152 wait complete. 2024-04-16T05:35:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.865+0000 7f7e2a209740 20 client.15753 trim_cache size 0 max 16384 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e2a209740 1 Processor -- start 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e2a209740 1 -- start start 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadae60 0x55a79fae3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadb780 0x55a79fafbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fafc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e2a209740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201b1ba0 con 0x55a79fae4360 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e2a209740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201af470 con 0x55a79fadb780 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e2a209740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201ad2c0 con 0x55a79fadae60 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fafc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fafc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41220/0 (socket says 172.21.15.32:41220) 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 -- 172.21.15.32:0/468831517 learned_addr learned my addr 172.21.15.32:0/468831517 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e24bb6640 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadb780 0x55a79fafbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e253b7640 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadae60 0x55a79fae3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 -- 172.21.15.32:0/468831517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadae60 msgr2=0x55a79fae3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadae60 0x55a79fae3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 -- 172.21.15.32:0/468831517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadb780 msgr2=0x55a79fafbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadb780 0x55a79fafbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 -- 172.21.15.32:0/468831517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a79fae6ac0 con 0x55a79fae4360 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e24bb6640 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadb780 0x55a79fafbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.879+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fafc3b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e18003bb0 tx=0x7f7e18019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303bf63b205612d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e0affd640 1 -- 172.21.15.32:0/468831517 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18025020 con 0x55a79fae4360 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e0affd640 1 -- 172.21.15.32:0/468831517 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e18004520 con 0x55a79fae4360 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e0affd640 1 -- 172.21.15.32:0/468831517 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18020df0 con 0x55a79fae4360 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 -- 172.21.15.32:0/468831517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 msgr2=0x55a79fafc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fafc3b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e18003bb0 tx=0x7f7e18019000 comp rx=0 tx=0).stop 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 -- 172.21.15.32:0/468831517 shutdown_connections 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fafc3b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadb780 0x55a79fafbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 --2- 172.21.15.32:0/468831517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadae60 0x55a79fae3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 -- 172.21.15.32:0/468831517 >> 172.21.15.32:0/468831517 conn(0x55a79fad5590 msgr2=0x55a79fad05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 -- 172.21.15.32:0/468831517 shutdown_connections 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 -- 172.21.15.32:0/468831517 wait complete. 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.880+0000 7f7e2a209740 1 -- start start 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e2a209740 1 Processor -- start 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79faec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadb780 0x55a79faddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e2a209740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fade250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e2a209740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201ad930 con 0x55a79fae4360 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e2a209740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201aeda0 con 0x55a79fadae60 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e2a209740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201ae000 con 0x55a79fadb780 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e2a209740 10 client.0 _ll_register_callbacks cb 0x7f7e100097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e25bb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fade250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e25bb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fade250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41232/0 (socket says 172.21.15.32:41232) 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e25bb8640 1 -- 172.21.15.32:0/1363871193 learned_addr learned my addr 172.21.15.32:0/1363871193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e253b7640 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79faec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e24bb6640 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadb780 0x55a79faddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e25bb8640 1 -- 172.21.15.32:0/1363871193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadb780 msgr2=0x55a79faddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadb780 0x55a79faddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e25bb8640 1 -- 172.21.15.32:0/1363871193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 msgr2=0x55a79faec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79faec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.881+0000 7f7e25bb8640 1 -- 172.21.15.32:0/1363871193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a79fade990 con 0x55a79fae4360 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e25bb8640 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fade250 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7e18016040 tx=0x7f7e18020710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7f29fd633f3cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e08ff9640 1 -- 172.21.15.32:0/1363871193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18025070 con 0x55a79fae4360 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e08ff9640 1 -- 172.21.15.32:0/1363871193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7e18020cf0 con 0x55a79fae4360 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 1 -- 172.21.15.32:0/1363871193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e18004520 con 0x55a79fae4360 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e08ff9640 1 -- 172.21.15.32:0/1363871193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e18003fb0 con 0x55a79fae4360 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 10 client.15759 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 1 -- 172.21.15.32:0/1363871193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a79fb5a5a0 con 0x55a79fae4360 2024-04-16T05:35:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e08ff9640 1 -- 172.21.15.32:0/1363871193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e18004520 con 0x55a79fae4360 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e08ff9640 1 -- 172.21.15.32:0/1363871193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7e180053d0 con 0x55a79fae4360 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 10 client.15759 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 10 client.15759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 20 client.15759 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55a79fb5ac60 0x55a79fb5d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 4 client.15759 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.882+0000 7f7e2a209740 1 -- 172.21.15.32:0/1363871193 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a79fa47970 con 0x55a79fb5ac60 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e253b7640 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55a79fb5ac60 0x55a79fb5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e253b7640 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55a79fb5ac60 0x55a79fb5d080 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d19d00ffef0b0e7 server_cookie=cdb7627dce6c55ed in_seq=0 out_seq=0 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e08ff9640 10 client.15759 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e08ff9640 1 -- 172.21.15.32:0/1363871193 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a79fa47970 con 0x55a79fb5ac60 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e08ff9640 10 client.15759 handle_command_reply: tid=0 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 client.15759 shutdown 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 -- 172.21.15.32:0/1363871193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 msgr2=0x55a79fade250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fade250 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7e18016040 tx=0x7f7e18020710 comp rx=0 tx=0).stop 2024-04-16T05:35:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 -- 172.21.15.32:0/1363871193 shutdown_connections 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55a79fb5ac60 0x55a79fb5d080 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a79fae4360 0x55a79fade250 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a79fadb780 0x55a79faddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e08ff9640 0 client.15759 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 --2- 172.21.15.32:0/1363871193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a79fadae60 0x55a79faec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.883+0000 7f7e2a209740 1 -- 172.21.15.32:0/1363871193 >> 172.21.15.32:0/1363871193 conn(0x55a79fad5590 msgr2=0x55a79fad3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2a209740 1 -- 172.21.15.32:0/1363871193 shutdown_connections 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2a209740 1 -- 172.21.15.32:0/1363871193 wait complete. 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2a209740 20 client.15759 trim_cache size 0 max 16384 2024-04-16T05:35:32.888 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:35:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 -- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7e0c0a3db0 msgr2=0x7f7e0c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7e0c0a3db0 0x7f7e0c0a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e18005fd0 tx=0x7f7e18009b30 comp rx=0 tx=0).stop 2024-04-16T05:35:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 -- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20101150 msgr2=0x7f7e2019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20101150 0x7f7e2019e650 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c006010 tx=0x7f7e1c02cb30 comp rx=0 tx=0).stop 2024-04-16T05:35:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 -- 172.21.15.32:0/1863088505 shutdown_connections 2024-04-16T05:35:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e20105e80 0x7f7e20198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e20101af0 0x7f7e2019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7e0c0a3db0 0x7f7e0c0a6270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 --2- 172.21.15.32:0/1863088505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e20101150 0x7f7e2019e650 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 -- 172.21.15.32:0/1863088505 >> 172.21.15.32:0/1863088505 conn(0x7f7e20077cf0 msgr2=0x7f7e200fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 -- 172.21.15.32:0/1863088505 shutdown_connections 2024-04-16T05:35:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:32.884+0000 7f7e2761b640 1 -- 172.21.15.32:0/1863088505 wait complete. 2024-04-16T05:35:32.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:35:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 -- 172.21.15.32:0/2109790786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874102c00 msgr2=0x7ff874106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 --2- 172.21.15.32:0/2109790786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874102c00 0x7ff874106c40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009900 tx=0x7ff85c02bc50 comp rx=0 tx=0).stop 2024-04-16T05:35:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 -- 172.21.15.32:0/2109790786 shutdown_connections 2024-04-16T05:35:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 --2- 172.21.15.32:0/2109790786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff874107320 0x7ff87410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 --2- 172.21.15.32:0/2109790786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874102c00 0x7ff874106c40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 --2- 172.21.15.32:0/2109790786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff874102250 0x7ff874102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 -- 172.21.15.32:0/2109790786 >> 172.21.15.32:0/2109790786 conn(0x7ff874079490 msgr2=0x7ff8740798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 -- 172.21.15.32:0/2109790786 shutdown_connections 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.002+0000 7ff8788d9640 1 -- 172.21.15.32:0/2109790786 wait complete. 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff8788d9640 1 Processor -- start 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff8788d9640 1 -- start start 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff8788d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff874102250 0x7ff87419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff8788d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874107320 0x7ff87419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff8788d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff874198650 0x7ff874198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff8788d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff874112390 con 0x7ff874107320 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff8788d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff874112210 con 0x7ff874102250 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff8788d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff874112510 con 0x7ff874198650 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff871d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874107320 0x7ff87419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff871d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874107320 0x7ff87419ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41242/0 (socket says 172.21.15.32:41242) 2024-04-16T05:35:33.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff871d9b640 1 -- 172.21.15.32:0/136556721 learned_addr learned my addr 172.21.15.32:0/136556721 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff872d9d640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff874198650 0x7ff874198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff87259c640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff874102250 0x7ff87419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff871d9b640 1 -- 172.21.15.32:0/136556721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff874198650 msgr2=0x7ff874198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff871d9b640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff874198650 0x7ff874198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff871d9b640 1 -- 172.21.15.32:0/136556721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff874102250 msgr2=0x7ff87419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff871d9b640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff874102250 0x7ff87419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.003+0000 7ff871d9b640 1 -- 172.21.15.32:0/136556721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff874199250 con 0x7ff874107320 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.004+0000 7ff871d9b640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874107320 0x7ff87419ead0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff85c000c00 tx=0x7ff85c02cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.004+0000 7ff86b7fe640 1 -- 172.21.15.32:0/136556721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c042020 con 0x7ff874107320 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.004+0000 7ff86b7fe640 1 -- 172.21.15.32:0/136556721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85c004450 con 0x7ff874107320 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.004+0000 7ff8788d9640 1 -- 172.21.15.32:0/136556721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8741994e0 con 0x7ff874107320 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.004+0000 7ff86b7fe640 1 -- 172.21.15.32:0/136556721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c03e460 con 0x7ff874107320 2024-04-16T05:35:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.004+0000 7ff8788d9640 1 -- 172.21.15.32:0/136556721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8741a56d0 con 0x7ff874107320 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.006+0000 7ff86b7fe640 1 -- 172.21.15.32:0/136556721 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff85c045020 con 0x7ff874107320 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.006+0000 7ff87b5a7740 1 Processor -- start 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.006+0000 7ff87b5a7740 1 -- start start 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6709240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc6711e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc6714750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561fc666e0e0 con 0x561fc6708e60 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561fc660b520 con 0x561fc6709780 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fc666a0a0 con 0x561fc6712360 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6709240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6709240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41252/0 (socket says 172.21.15.32:41252) 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 -- 172.21.15.32:0/1498771143 learned_addr learned my addr 172.21.15.32:0/1498771143 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff86b7fe640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8600a3e30 0x7ff8600a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff87259c640 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc6711e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff871d9b640 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc6714750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8600a3e30 0x7ff8600a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff86b7fe640 1 -- 172.21.15.32:0/136556721 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff85c0f2ce0 con 0x7ff874107320 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 -- 172.21.15.32:0/1498771143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 msgr2=0x561fc6714750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc6714750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 -- 172.21.15.32:0/1498771143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 msgr2=0x561fc6711e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc6711e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 -- 172.21.15.32:0/1498771143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc66338d0 con 0x561fc6708e60 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff871d9b640 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc6714750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff87259c640 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc6711e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.007+0000 7ff872d9d640 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6709240 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff8640139d0 tx=0x7ff864013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5513f374cd5a9cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff868ff9640 1 -- 172.21.15.32:0/1498771143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864013050 con 0x561fc6708e60 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff868ff9640 1 -- 172.21.15.32:0/1498771143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff864022070 con 0x561fc6708e60 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff868ff9640 1 -- 172.21.15.32:0/1498771143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864012650 con 0x561fc6708e60 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff872d9d640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8600a3e30 0x7ff8600a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff864021000 tx=0x7ff86401f000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 -- 172.21.15.32:0/1498771143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 msgr2=0x561fc6709240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6709240 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff8640139d0 tx=0x7ff864013e90 comp rx=0 tx=0).stop 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 -- 172.21.15.32:0/1498771143 shutdown_connections 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc6714750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc6711e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/1498771143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6709240 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 -- 172.21.15.32:0/1498771143 >> 172.21.15.32:0/1498771143 conn(0x561fc6703590 msgr2=0x561fc6703970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 -- 172.21.15.32:0/1498771143 shutdown_connections 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.008+0000 7ff87b5a7740 1 -- 172.21.15.32:0/1498771143 wait complete. 2024-04-16T05:35:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 1 -- start start 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 1 Processor -- start 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc670f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc670f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc670fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561fc6715000 con 0x561fc6708e60 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561fc66274d0 con 0x561fc6709780 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fc6715180 con 0x561fc6712360 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87b5a7740 10 client.0 _ll_register_callbacks cb 0x561fc660c2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc670f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc670f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41264/0 (socket says 172.21.15.32:41264) 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87259c640 1 -- 172.21.15.32:0/3943686202 learned_addr learned my addr 172.21.15.32:0/3943686202 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff872d9d640 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc670fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87259c640 1 -- 172.21.15.32:0/3943686202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 msgr2=0x561fc670fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff871d9b640 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc670f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.009+0000 7ff87259c640 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc670fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87259c640 1 -- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 msgr2=0x561fc670f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87259c640 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc670f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87259c640 1 -- 172.21.15.32:0/3943686202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc63803f0 con 0x561fc6708e60 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff871d9b640 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc670f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87259c640 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc670f380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00ce10 tx=0x7ff86c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4707db3f0d77ecab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff859ffb640 1 -- 172.21.15.32:0/3943686202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c011970 con 0x561fc6708e60 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff859ffb640 1 -- 172.21.15.32:0/3943686202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff86c011b10 con 0x561fc6708e60 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3943686202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff864022070 con 0x561fc6708e60 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff859ffb640 1 -- 172.21.15.32:0/3943686202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff86c011d10 con 0x561fc6708e60 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87b5a7740 10 client.15771 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3943686202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561fc644a7d0 con 0x561fc6708e60 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff859ffb640 1 -- 172.21.15.32:0/3943686202 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff864022070 con 0x561fc6708e60 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff859ffb640 1 -- 172.21.15.32:0/3943686202 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff86c018070 con 0x561fc6708e60 2024-04-16T05:35:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87b5a7740 10 client.15771 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87b5a7740 10 client.15771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87b5a7740 20 client.15771 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.010+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x561fc6722390 0x561fc6724800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.011+0000 7ff87b5a7740 4 client.15771 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.011+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3943686202 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fc667bb00 con 0x561fc6722390 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.011+0000 7ff871d9b640 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x561fc6722390 0x561fc6724800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.011+0000 7ff871d9b640 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x561fc6722390 0x561fc6724800 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43edc494ce541cb3 server_cookie=e9b4f6e5a8aa90b3 in_seq=0 out_seq=0 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.011+0000 7ff859ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff859ffb640 1 -- 172.21.15.32:0/3943686202 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fc667bb00 con 0x561fc6722390 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff859ffb640 10 client.15771 handle_command_reply: tid=0 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 client.15771 shutdown 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3943686202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 msgr2=0x561fc670f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc670f380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00ce10 tx=0x7ff86c0039c0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3943686202 shutdown_connections 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x561fc6722390 0x561fc6724800 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6712360 0x561fc670fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6709780 0x561fc670f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff859ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3943686202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc670f380 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3943686202 >> 172.21.15.32:0/3943686202 conn(0x561fc6703590 msgr2=0x561fc67052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3943686202 shutdown_connections 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3943686202 wait complete. 2024-04-16T05:35:33.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.012+0000 7ff87b5a7740 20 client.15771 trim_cache size 0 max 16384 2024-04-16T05:35:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.025+0000 7ff87b5a7740 1 Processor -- start 2024-04-16T05:35:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.025+0000 7ff87b5a7740 1 -- start start 2024-04-16T05:35:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6711ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 0x561fc671e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 0x561fc67025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741b1c00 con 0x561fc6708e60 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741af4d0 con 0x561fc6712360 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741ad320 con 0x561fc6709780 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6711ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6711ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41278/0 (socket says 172.21.15.32:41278) 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 -- 172.21.15.32:0/2070683324 learned_addr learned my addr 172.21.15.32:0/2070683324 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 -- 172.21.15.32:0/2070683324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 msgr2=0x561fc671e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 --2- 172.21.15.32:0/2070683324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 0x561fc671e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 -- 172.21.15.32:0/2070683324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 msgr2=0x561fc67025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 --2- 172.21.15.32:0/2070683324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 0x561fc67025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 -- 172.21.15.32:0/2070683324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc6704160 con 0x561fc6708e60 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff87259c640 1 --2- 172.21.15.32:0/2070683324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6711ab0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff86c004220 tx=0x7ff86c019a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dec2a8140065758 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff85bfff640 1 -- 172.21.15.32:0/2070683324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c019df0 con 0x561fc6708e60 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff85bfff640 1 -- 172.21.15.32:0/2070683324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86c005030 con 0x561fc6708e60 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.026+0000 7ff85bfff640 1 -- 172.21.15.32:0/2070683324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c004850 con 0x561fc6708e60 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- 172.21.15.32:0/2070683324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 msgr2=0x561fc6711ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/2070683324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6711ab0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff86c004220 tx=0x7ff86c019a30 comp rx=0 tx=0).stop 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- 172.21.15.32:0/2070683324 shutdown_connections 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/2070683324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 0x561fc67025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/2070683324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 0x561fc671e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/2070683324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc6711ab0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- 172.21.15.32:0/2070683324 >> 172.21.15.32:0/2070683324 conn(0x561fc6703590 msgr2=0x561fc6701ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- 172.21.15.32:0/2070683324 shutdown_connections 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- 172.21.15.32:0/2070683324 wait complete. 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- start start 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 Processor -- start 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc67160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 0x561fc67165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 0x561fc6716b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741ad990 con 0x561fc6708e60 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741aee00 con 0x561fc6712360 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.027+0000 7ff87b5a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741ae060 con 0x561fc6709780 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87b5a7740 10 client.0 _ll_register_callbacks cb 0x7ff86c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc67160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc67160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41284/0 (socket says 172.21.15.32:41284) 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 -- 172.21.15.32:0/3519401772 learned_addr learned my addr 172.21.15.32:0/3519401772 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff871d9b640 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 0x561fc67165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 -- 172.21.15.32:0/3519401772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 msgr2=0x561fc67165e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 0x561fc67165e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff872d9d640 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 0x561fc6716b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 -- 172.21.15.32:0/3519401772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 msgr2=0x561fc6716b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 0x561fc6716b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 -- 172.21.15.32:0/3519401772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc6787af0 con 0x561fc6708e60 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87259c640 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc67160a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c6f0 tx=0x7ff86c00c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f360d1da65634471 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff859ffb640 1 -- 172.21.15.32:0/3519401772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0051e0 con 0x561fc6708e60 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff859ffb640 1 -- 172.21.15.32:0/3519401772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff86c005380 con 0x561fc6708e60 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3519401772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff86c005030 con 0x561fc6708e60 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff859ffb640 1 -- 172.21.15.32:0/3519401772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff86c044050 con 0x561fc6708e60 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87b5a7740 10 client.15777 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.028+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3519401772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561fc6788db0 con 0x561fc6708e60 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff859ffb640 1 -- 172.21.15.32:0/3519401772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86c005030 con 0x561fc6708e60 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff859ffb640 1 -- 172.21.15.32:0/3519401772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff86c018070 con 0x561fc6708e60 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff87b5a7740 10 client.15777 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff87b5a7740 10 client.15777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:35:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff87b5a7740 20 client.15777 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x561fc67893c0 0x561fc678b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff87b5a7740 4 client.15777 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3519401772 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fc6675970 con 0x561fc67893c0 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff871d9b640 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x561fc67893c0 0x561fc678b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff871d9b640 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x561fc67893c0 0x561fc678b7e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f8a0ea9836b0636 server_cookie=75e549d4e91117bf in_seq=0 out_seq=0 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.029+0000 7ff859ffb640 10 client.15777 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff859ffb640 1 -- 172.21.15.32:0/3519401772 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fc6675970 con 0x561fc67893c0 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff859ffb640 10 client.15777 handle_command_reply: tid=0 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 client.15777 shutdown 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3519401772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 msgr2=0x561fc67160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc67160a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c6f0 tx=0x7ff86c00c4b0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3519401772 shutdown_connections 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x561fc67893c0 0x561fc678b7e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc6712360 0x561fc6716b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc6709780 0x561fc67165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff859ffb640 0 client.15777 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:35:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 --2- 172.21.15.32:0/3519401772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc6708e60 0x561fc67160a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3519401772 >> 172.21.15.32:0/3519401772 conn(0x561fc6703590 msgr2=0x561fc670fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3519401772 shutdown_connections 2024-04-16T05:35:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 1 -- 172.21.15.32:0/3519401772 wait complete. 2024-04-16T05:35:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff87b5a7740 20 client.15777 trim_cache size 0 max 16384 2024-04-16T05:35:33.035 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 -- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8600a3e30 msgr2=0x7ff8600a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8600a3e30 0x7ff8600a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff864021000 tx=0x7ff86401f000 comp rx=0 tx=0).stop 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 -- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874107320 msgr2=0x7ff87419ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874107320 0x7ff87419ead0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff85c000c00 tx=0x7ff85c02cc90 comp rx=0 tx=0).stop 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 -- 172.21.15.32:0/136556721 shutdown_connections 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff874198650 0x7ff874198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8600a3e30 0x7ff8600a62f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff874107320 0x7ff87419ead0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 --2- 172.21.15.32:0/136556721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff874102250 0x7ff87419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.030+0000 7ff8788d9640 1 -- 172.21.15.32:0/136556721 >> 172.21.15.32:0/136556721 conn(0x7ff874079490 msgr2=0x7ff8740ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.031+0000 7ff8788d9640 1 -- 172.21.15.32:0/136556721 shutdown_connections 2024-04-16T05:35:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.031+0000 7ff8788d9640 1 -- 172.21.15.32:0/136556721 wait complete. 2024-04-16T05:35:33.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:35:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/1959911745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e78100f50 msgr2=0x7f1e78101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/1959911745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e78100f50 0x7f1e78101330 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009900 tx=0x7f1e6802bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/1959911745 shutdown_connections 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/1959911745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e78106020 0x7f1e7810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/1959911745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e78101900 0x7f1e78105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/1959911745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e78100f50 0x7f1e78101330 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/1959911745 >> 172.21.15.32:0/1959911745 conn(0x7f1e780fc790 msgr2=0x7f1e780febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/1959911745 shutdown_connections 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/1959911745 wait complete. 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.149+0000 7f1e7d3ff640 1 Processor -- start 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e7d3ff640 1 -- start start 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e7d3ff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e78100f50 0x7f1e781a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e7d3ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e78106020 0x7f1e781a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e7d3ff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e7819a660 0x7f1e7819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e7d3ff640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781110c0 con 0x7f1e7819a660 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e7d3ff640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78110f40 con 0x7f1e78106020 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e7d3ff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78111240 con 0x7f1e78100f50 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e7819a660 0x7f1e7819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e7819a660 0x7f1e7819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41302/0 (socket says 172.21.15.32:41302) 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e777fe640 1 -- 172.21.15.32:0/44931529 learned_addr learned my addr 172.21.15.32:0/44931529 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e767fc640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e78106020 0x7f1e781a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e78100f50 0x7f1e781a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e777fe640 1 -- 172.21.15.32:0/44931529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e78100f50 msgr2=0x7f1e781a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e777fe640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e78100f50 0x7f1e781a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e777fe640 1 -- 172.21.15.32:0/44931529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e78106020 msgr2=0x7f1e781a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e777fe640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e78106020 0x7f1e781a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e777fe640 1 -- 172.21.15.32:0/44931529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7819b260 con 0x7f1e7819a660 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.150+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e78100f50 0x7f1e781a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.151+0000 7f1e777fe640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e7819a660 0x7f1e7819aae0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cce0 tx=0x7f1e7000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.151+0000 7f1e6ffff640 1 -- 172.21.15.32:0/44931529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000f450 con 0x7f1e7819a660 2024-04-16T05:35:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.151+0000 7f1e6ffff640 1 -- 172.21.15.32:0/44931529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7000fad0 con 0x7f1e7819a660 2024-04-16T05:35:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.151+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/44931529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e781a7340 con 0x7f1e7819a660 2024-04-16T05:35:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.151+0000 7f1e6ffff640 1 -- 172.21.15.32:0/44931529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70018c10 con 0x7f1e7819a660 2024-04-16T05:35:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.151+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/44931529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e781a76f0 con 0x7f1e7819a660 2024-04-16T05:35:33.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.153+0000 7f1e6ffff640 1 -- 172.21.15.32:0/44931529 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1e7000f5f0 con 0x7f1e7819a660 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.153+0000 7f1e7ffef740 1 Processor -- start 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.153+0000 7f1e7ffef740 1 -- start start 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745010 0x55d4947453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745930 0x55d49474dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d494750900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d4946aa0e0 con 0x55d494745010 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d494647520 con 0x55d494745930 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4946a60a0 con 0x55d49474e510 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745010 0x55d4947453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745010 0x55d4947453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41308/0 (socket says 172.21.15.32:41308) 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 -- 172.21.15.32:0/4173132540 learned_addr learned my addr 172.21.15.32:0/4173132540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e6ffff640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e640a3db0 0x7f1e640a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d494750900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745930 0x55d49474dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 -- 172.21.15.32:0/4173132540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 msgr2=0x55d494750900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d494750900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 -- 172.21.15.32:0/4173132540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745930 msgr2=0x55d49474dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745930 0x55d49474dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 -- 172.21.15.32:0/4173132540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d49466f8d0 con 0x55d494745010 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745930 0x55d49474dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e777fe640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e640a3db0 0x7f1e640a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e767fc640 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745010 0x55d4947453f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1e600134b0 tx=0x7f1e6000a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcfbf31e0e532eb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e6ffff640 1 -- 172.21.15.32:0/44931529 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1e700d13d0 con 0x7f1e7819a660 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e6d7fa640 1 -- 172.21.15.32:0/4173132540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60019030 con 0x55d494745010 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e6d7fa640 1 -- 172.21.15.32:0/4173132540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e60005500 con 0x55d494745010 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.154+0000 7f1e6d7fa640 1 -- 172.21.15.32:0/4173132540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60004e60 con 0x55d494745010 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 -- 172.21.15.32:0/4173132540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745010 msgr2=0x55d4947453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e777fe640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e640a3db0 0x7f1e640a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e700ce720 tx=0x7f1e7006e8a0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745010 0x55d4947453f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1e600134b0 tx=0x7f1e6000a720 comp rx=0 tx=0).stop 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 -- 172.21.15.32:0/4173132540 shutdown_connections 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d494750900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745930 0x55d49474dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/4173132540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745010 0x55d4947453f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 -- 172.21.15.32:0/4173132540 >> 172.21.15.32:0/4173132540 conn(0x55d49473f740 msgr2=0x55d49473fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 -- 172.21.15.32:0/4173132540 shutdown_connections 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.155+0000 7f1e7ffef740 1 -- 172.21.15.32:0/4173132540 wait complete. 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 1 -- start start 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 1 Processor -- start 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49474b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d49474be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d494751300 con 0x55d494745930 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d4946634d0 con 0x55d494745010 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d494751480 con 0x55d49474e510 2024-04-16T05:35:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e7ffef740 10 client.0 _ll_register_callbacks cb 0x55d4946482c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49474b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49474b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41310/0 (socket says 172.21.15.32:41310) 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 -- 172.21.15.32:0/3016000793 learned_addr learned my addr 172.21.15.32:0/3016000793 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e767fc640 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d49474be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e767fc640 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 -- 172.21.15.32:0/3016000793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 msgr2=0x55d49474be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d49474be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 -- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 msgr2=0x55d49474b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 -- 172.21.15.32:0/3016000793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4943bc3f0 con 0x55d494745930 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e767fc640 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49474b940 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1e68005ab0 tx=0x7f1e68005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b928f5697f4293 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.156+0000 7f1e5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/3016000793 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68005030 con 0x55d494745930 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/3016000793 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1e680051d0 con 0x55d494745930 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3016000793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e60005500 con 0x55d494745930 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/3016000793 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e6804c050 con 0x55d494745930 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 10 client.15789 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3016000793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4944867d0 con 0x55d494745930 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/3016000793 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e60005500 con 0x55d494745930 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/3016000793 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1e680036f0 con 0x55d494745930 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 10 client.15789 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 20 client.15789 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d49475e410 0x55d494760860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 4 client.15789 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.157+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3016000793 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4946b7b00 con 0x55d49475e410 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.158+0000 7f1e767fc640 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d49475e410 0x55d494760860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.158+0000 7f1e767fc640 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d49475e410 0x55d494760860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660a59424aac43fd server_cookie=7887d385d8827a9d in_seq=0 out_seq=0 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.158+0000 7f1e5e7fc640 10 client.15789 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/3016000793 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4946b7b00 con 0x55d49475e410 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e5e7fc640 10 client.15789 handle_command_reply: tid=0 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 client.15789 shutdown 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3016000793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 msgr2=0x55d49474b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49474b940 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1e68005ab0 tx=0x7f1e68005de0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3016000793 shutdown_connections 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d49475e410 0x55d494760860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d49474be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e5e7fc640 0 client.15789 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49474b940 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3016000793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3016000793 >> 172.21.15.32:0/3016000793 conn(0x55d49473f740 msgr2=0x55d494741490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3016000793 shutdown_connections 2024-04-16T05:35:33.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3016000793 wait complete. 2024-04-16T05:35:33.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.159+0000 7f1e7ffef740 20 client.15789 trim_cache size 0 max 16384 2024-04-16T05:35:33.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.172+0000 7f1e7ffef740 1 Processor -- start 2024-04-16T05:35:33.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.172+0000 7f1e7ffef740 1 -- start start 2024-04-16T05:35:33.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.172+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.172+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49475a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d49473e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b3b60 con 0x55d494745930 2024-04-16T05:35:33.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b1430 con 0x55d494745010 2024-04-16T05:35:33.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781af280 con 0x55d49474e510 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49475a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49475a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41320/0 (socket says 172.21.15.32:41320) 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 -- 172.21.15.32:0/3534765521 learned_addr learned my addr 172.21.15.32:0/3534765521 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e767fc640 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 -- 172.21.15.32:0/3534765521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 msgr2=0x55d49473e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e767fc640 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d49473e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d49473e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 -- 172.21.15.32:0/3534765521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 msgr2=0x55d49474db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 -- 172.21.15.32:0/3534765521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d49474a3b0 con 0x55d494745930 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49475a220 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1e6802bc70 tx=0x7f1e6802cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac954296c43b0138 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e6cff9640 1 -- 172.21.15.32:0/3534765521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e680035f0 con 0x55d494745930 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e6cff9640 1 -- 172.21.15.32:0/3534765521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e68003790 con 0x55d494745930 2024-04-16T05:35:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.173+0000 7f1e6cff9640 1 -- 172.21.15.32:0/3534765521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e680046c0 con 0x55d494745930 2024-04-16T05:35:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3534765521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 msgr2=0x55d49475a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49475a220 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1e6802bc70 tx=0x7f1e6802cc90 comp rx=0 tx=0).stop 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3534765521 shutdown_connections 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d49473e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d49475a220 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/3534765521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d49474db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3534765521 >> 172.21.15.32:0/3534765521 conn(0x55d49473f740 msgr2=0x55d49473fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3534765521 shutdown_connections 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 -- 172.21.15.32:0/3534765521 wait complete. 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 -- start start 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.174+0000 7f1e7ffef740 1 Processor -- start 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d494752330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d494752870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e7ffef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d494752db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781af8f0 con 0x55d494745930 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b0d60 con 0x55d494745010 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e7ffef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781affc0 con 0x55d49474e510 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e7ffef740 10 client.0 _ll_register_callbacks cb 0x7f1e68009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e76ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d494752870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e767fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d494752330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e76ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d494752870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41324/0 (socket says 172.21.15.32:41324) 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e76ffd640 1 -- 172.21.15.32:0/144828328 learned_addr learned my addr 172.21.15.32:0/144828328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e767fc640 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d494752db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e767fc640 1 -- 172.21.15.32:0/144828328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 msgr2=0x55d494752db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e767fc640 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d494752db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e767fc640 1 -- 172.21.15.32:0/144828328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 msgr2=0x55d494752870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e767fc640 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d494752870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.175+0000 7f1e767fc640 1 -- 172.21.15.32:0/144828328 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4947c3ba0 con 0x55d494745010 2024-04-16T05:35:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e767fc640 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d494752330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e600058c0 tx=0x7f1e60020040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79ce6aa3d676ada server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/144828328 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60019040 con 0x55d494745010 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 1 -- 172.21.15.32:0/144828328 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e68003790 con 0x55d494745010 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/144828328 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1e6000ace0 con 0x55d494745010 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/144828328 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e60039050 con 0x55d494745010 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 10 client.25346 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 1 -- 172.21.15.32:0/144828328 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4947c4e80 con 0x55d494745010 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/144828328 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e68003790 con 0x55d494745010 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/144828328 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1e6002b3a0 con 0x55d494745010 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 10 client.25346 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 10 client.25346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 20 client.25346 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d4947c5270 0x55d4947c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 4 client.25346 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.176+0000 7f1e7ffef740 1 -- 172.21.15.32:0/144828328 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4946b1970 con 0x55d4947c5270 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d4947c5270 0x55d4947c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e76ffd640 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d4947c5270 0x55d4947c7660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb95b685d5e5794 server_cookie=eee858477459e25d in_seq=0 out_seq=0 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e5e7fc640 10 client.25346 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e5e7fc640 1 -- 172.21.15.32:0/144828328 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4946b1970 con 0x55d4947c5270 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e5e7fc640 10 client.25346 handle_command_reply: tid=0 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 client.25346 shutdown 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 -- 172.21.15.32:0/144828328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 msgr2=0x55d494752330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d494752330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e600058c0 tx=0x7f1e60020040 comp rx=0 tx=0).stop 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 -- 172.21.15.32:0/144828328 shutdown_connections 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d4947c5270 0x55d4947c7660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d49474e510 0x55d494752db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d494745930 0x55d494752870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e5e7fc640 0 client.25346 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 --2- 172.21.15.32:0/144828328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d494745010 0x55d494752330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 -- 172.21.15.32:0/144828328 >> 172.21.15.32:0/144828328 conn(0x55d49473f740 msgr2=0x55d49474bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 -- 172.21.15.32:0/144828328 shutdown_connections 2024-04-16T05:35:33.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 1 -- 172.21.15.32:0/144828328 wait complete. 2024-04-16T05:35:33.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.177+0000 7f1e7ffef740 20 client.25346 trim_cache size 0 max 16384 2024-04-16T05:35:33.182 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e640a3db0 msgr2=0x7f1e640a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e640a3db0 0x7f1e640a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e700ce720 tx=0x7f1e7006e8a0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e7819a660 msgr2=0x7f1e7819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e7819a660 0x7f1e7819aae0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cce0 tx=0x7f1e7000a620 comp rx=0 tx=0).stop 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/44931529 shutdown_connections 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e640a3db0 0x7f1e640a6270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e7819a660 0x7f1e7819aae0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e78106020 0x7f1e781a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 --2- 172.21.15.32:0/44931529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e78100f50 0x7f1e781a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/44931529 >> 172.21.15.32:0/44931529 conn(0x7f1e780fc790 msgr2=0x7f1e780fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/44931529 shutdown_connections 2024-04-16T05:35:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:33.178+0000 7f1e7d3ff640 1 -- 172.21.15.32:0/44931529 wait complete. 2024-04-16T05:35:33.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:33 smithi134 ceph-mon[36153]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:35:33.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:33 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1884285355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:33.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:33 smithi082 ceph-mon[36195]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:35:33.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:33 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1884285355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:33.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:33 smithi032 ceph-mon[32351]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:35:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:33 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1884285355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:35:34.187 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:35:34.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:35:34.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:35:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 -- 172.21.15.32:0/1160754169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14107310 msgr2=0x7fbb1410eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 --2- 172.21.15.32:0/1160754169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14107310 0x7fbb1410eee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009ca0 tx=0x7fbb0c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 -- 172.21.15.32:0/1160754169 shutdown_connections 2024-04-16T05:35:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 --2- 172.21.15.32:0/1160754169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14107310 0x7fbb1410eee0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 --2- 172.21.15.32:0/1160754169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb14102bf0 0x7fbb14106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 --2- 172.21.15.32:0/1160754169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb14102240 0x7fbb14102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 -- 172.21.15.32:0/1160754169 >> 172.21.15.32:0/1160754169 conn(0x7fbb140fda40 msgr2=0x7fbb140ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 -- 172.21.15.32:0/1160754169 shutdown_connections 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.312+0000 7fbb1882a640 1 -- 172.21.15.32:0/1160754169 wait complete. 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1882a640 1 Processor -- start 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1882a640 1 -- start start 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1882a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14102240 0x7fbb14074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1882a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb14107310 0x7fbb14074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1882a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb14074d80 0x7fbb141a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1882a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141123b0 con 0x7fbb14102240 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1882a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14112230 con 0x7fbb14074d80 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1882a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14112530 con 0x7fbb14107310 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14102240 0x7fbb14074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14102240 0x7fbb14074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41346/0 (socket says 172.21.15.32:41346) 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.313+0000 7fbb1259c640 1 -- 172.21.15.32:0/2267796751 learned_addr learned my addr 172.21.15.32:0/2267796751 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb12d9d640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb14074d80 0x7fbb141a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb1259c640 1 -- 172.21.15.32:0/2267796751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb14107310 msgr2=0x7fbb14074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb1259c640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb14107310 0x7fbb14074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb1259c640 1 -- 172.21.15.32:0/2267796751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb14074d80 msgr2=0x7fbb141a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb1259c640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb14074d80 0x7fbb141a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb1259c640 1 -- 172.21.15.32:0/2267796751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb141a7180 con 0x7fbb14102240 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb12d9d640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb14074d80 0x7fbb141a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb1259c640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14102240 0x7fbb14074300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400d950 tx=0x7fbb0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb0b7fe640 1 -- 172.21.15.32:0/2267796751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04017070 con 0x7fbb14102240 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb0b7fe640 1 -- 172.21.15.32:0/2267796751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04009c30 con 0x7fbb14102240 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb1882a640 1 -- 172.21.15.32:0/2267796751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb141a7470 con 0x7fbb14102240 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb0b7fe640 1 -- 172.21.15.32:0/2267796751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400f6b0 con 0x7fbb14102240 2024-04-16T05:35:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.314+0000 7fbb1882a640 1 -- 172.21.15.32:0/2267796751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb141a79c0 con 0x7fbb14102240 2024-04-16T05:35:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.316+0000 7fbb0b7fe640 1 -- 172.21.15.32:0/2267796751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fbb040109d0 con 0x7fbb14102240 2024-04-16T05:35:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1b411740 1 Processor -- start 2024-04-16T05:35:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1b411740 1 -- start start 2024-04-16T05:35:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbbab0 0x55f705cbbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 0x55f705cc73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1b411740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f705c20e90 con 0x55f705cbc3d0 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1b411740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f705bbe4c0 con 0x55f705cbbab0 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1b411740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f705c1ce50 con 0x55f705cc4fb0 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb0b7fe640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbb000a3de0 0x7fbb000a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb12d9d640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbb000a3de0 0x7fbb000a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41360/0 (socket says 172.21.15.32:41360) 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb1259c640 1 -- 172.21.15.32:0/1513306664 learned_addr learned my addr 172.21.15.32:0/1513306664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.317+0000 7fbb12d9d640 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbbab0 0x55f705cbbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 0x55f705cc73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb0b7fe640 1 -- 172.21.15.32:0/2267796751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbb040d0990 con 0x7fbb14102240 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb1259c640 1 -- 172.21.15.32:0/1513306664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 msgr2=0x55f705cc73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb1259c640 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 0x55f705cc73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb1259c640 1 -- 172.21.15.32:0/1513306664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbbab0 msgr2=0x55f705cbbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb1259c640 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbbab0 0x55f705cbbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb1259c640 1 -- 172.21.15.32:0/1513306664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7059fb7e0 con 0x55f705cbc3d0 2024-04-16T05:35:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb12d9d640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbb000a3de0 0x7fbb000a62a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005e30 tx=0x7fbb0c032df0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb1259c640 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc4a70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbb0409c410 tx=0x7fbb0400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8bda6cdb33acf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb08ff9640 1 -- 172.21.15.32:0/1513306664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04020070 con 0x55f705cbc3d0 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb08ff9640 1 -- 172.21.15.32:0/1513306664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0409cb80 con 0x55f705cbc3d0 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.318+0000 7fbb08ff9640 1 -- 172.21.15.32:0/1513306664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04098af0 con 0x55f705cbc3d0 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 -- 172.21.15.32:0/1513306664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 msgr2=0x55f705cc4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc4a70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbb0409c410 tx=0x7fbb0400bf60 comp rx=0 tx=0).stop 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 -- 172.21.15.32:0/1513306664 shutdown_connections 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 0x55f705cc73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc4a70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 --2- 172.21.15.32:0/1513306664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbbab0 0x55f705cbbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 -- 172.21.15.32:0/1513306664 >> 172.21.15.32:0/1513306664 conn(0x55f705cb61e0 msgr2=0x55f705cb65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 -- 172.21.15.32:0/1513306664 shutdown_connections 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 -- 172.21.15.32:0/1513306664 wait complete. 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 -- start start 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.319+0000 7fbb1b411740 1 Processor -- start 2024-04-16T05:35:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 0x55f705cc20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 0x55f705cc2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1b411740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f705cc7d40 con 0x55f705cbc3d0 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1b411740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f705ba9410 con 0x55f705cc4fb0 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1b411740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f705cc7ec0 con 0x55f705cbbab0 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1b411740 10 client.0 _ll_register_callbacks cb 0x55f705ba4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41364/0 (socket says 172.21.15.32:41364) 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 -- 172.21.15.32:0/3599194561 learned_addr learned my addr 172.21.15.32:0/3599194561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb12d9d640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 0x55f705cc2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1259c640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 0x55f705cc20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 -- 172.21.15.32:0/3599194561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 msgr2=0x55f705cc20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 0x55f705cc20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 -- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 msgr2=0x55f705cc2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 0x55f705cc2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 -- 172.21.15.32:0/3599194561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f705a5bf70 con 0x55f705cbc3d0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb12d9d640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 0x55f705cc2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1259c640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 0x55f705cc20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc2610 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00b880 tx=0x7fbafc00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82faeee544ca8f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbaf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/3599194561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc011440 con 0x55f705cbc3d0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/3599194561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbafc0115e0 con 0x55f705cbc3d0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.320+0000 7fbb1b411740 1 -- 172.21.15.32:0/3599194561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0409cb80 con 0x55f705cbc3d0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/3599194561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbafc022050 con 0x55f705cbc3d0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1b411740 10 client.15807 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1b411740 1 -- 172.21.15.32:0/3599194561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7058f3170 con 0x55f705cbc3d0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/3599194561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0409cb80 con 0x55f705cbc3d0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/3599194561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbafc01a950 con 0x55f705cbc3d0 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1b411740 10 client.15807 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1b411740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1b411740 20 client.15807 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1b411740 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f705cd53c0 0x55f705cd7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1b411740 4 client.15807 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1b411740 1 -- 172.21.15.32:0/3599194561 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f705c2e8b0 con 0x55f705cd53c0 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.321+0000 7fbb1259c640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f705cd53c0 0x55f705cd7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbaf9ffb640 10 client.15807 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1259c640 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f705cd53c0 0x55f705cd7820 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5714419e899c4d2 server_cookie=11d6aa4b0131eea1 in_seq=0 out_seq=0 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/3599194561 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f705c2e8b0 con 0x55f705cd53c0 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbaf9ffb640 10 client.15807 handle_command_reply: tid=0 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 client.15807 shutdown 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 -- 172.21.15.32:0/3599194561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 msgr2=0x55f705cc2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc2610 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00b880 tx=0x7fbafc00bd40 comp rx=0 tx=0).stop 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 -- 172.21.15.32:0/3599194561 shutdown_connections 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 0x55f705cc2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cc2610 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f705cd53c0 0x55f705cd7820 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 --2- 172.21.15.32:0/3599194561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 0x55f705cc20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbaf9ffb640 0 client.15807 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:34.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.322+0000 7fbb1b411740 1 -- 172.21.15.32:0/3599194561 >> 172.21.15.32:0/3599194561 conn(0x55f705cb61e0 msgr2=0x55f705cb7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.323+0000 7fbb1b411740 1 -- 172.21.15.32:0/3599194561 shutdown_connections 2024-04-16T05:35:34.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.323+0000 7fbb1b411740 1 -- 172.21.15.32:0/3599194561 wait complete. 2024-04-16T05:35:34.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.323+0000 7fbb1b411740 20 client.15807 trim_cache size 0 max 16384 2024-04-16T05:35:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.340+0000 7fbb1b411740 1 Processor -- start 2024-04-16T05:35:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.340+0000 7fbb1b411740 1 -- start start 2024-04-16T05:35:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.340+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 0x55f705cc4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cd0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 0x55f705cdd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb1b411740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141b3e70 con 0x55f705cbc3d0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb1b411740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141b1740 con 0x55f705cc4fb0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb1b411740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141af590 con 0x55f705cbbab0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cd0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cd0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41368/0 (socket says 172.21.15.32:41368) 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 -- 172.21.15.32:0/1278520773 learned_addr learned my addr 172.21.15.32:0/1278520773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 -- 172.21.15.32:0/1278520773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 msgr2=0x55f705cc4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/1278520773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 0x55f705cc4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 -- 172.21.15.32:0/1278520773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 msgr2=0x55f705cdd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/1278520773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 0x55f705cdd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 -- 172.21.15.32:0/1278520773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f705cc7710 con 0x55f705cbc3d0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/1278520773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cd0c00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbafc010040 tx=0x7fbafc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7144aed9635593cd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbafbfff640 1 -- 172.21.15.32:0/1278520773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc01a420 con 0x55f705cbc3d0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbafbfff640 1 -- 172.21.15.32:0/1278520773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc015030 con 0x55f705cbc3d0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.341+0000 7fbafbfff640 1 -- 172.21.15.32:0/1278520773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc011b20 con 0x55f705cbc3d0 2024-04-16T05:35:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- 172.21.15.32:0/1278520773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 msgr2=0x55f705cd0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 --2- 172.21.15.32:0/1278520773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cd0c00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbafc010040 tx=0x7fbafc0058d0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- 172.21.15.32:0/1278520773 shutdown_connections 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 --2- 172.21.15.32:0/1278520773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cc4fb0 0x55f705cdd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 --2- 172.21.15.32:0/1278520773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbc3d0 0x55f705cd0c00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 --2- 172.21.15.32:0/1278520773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cbbab0 0x55f705cc4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- 172.21.15.32:0/1278520773 >> 172.21.15.32:0/1278520773 conn(0x55f705cb61e0 msgr2=0x55f705cc2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- 172.21.15.32:0/1278520773 shutdown_connections 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- 172.21.15.32:0/1278520773 wait complete. 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- start start 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 Processor -- start 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbbab0 0x55f705cbf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbc3d0 0x55f705cbf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 0x55f705d3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141afc00 con 0x55f705cbbab0 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141b1070 con 0x55f705cbc3d0 2024-04-16T05:35:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141b02d0 con 0x55f705cc4fb0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.342+0000 7fbb1b411740 10 client.0 _ll_register_callbacks cb 0x7fbafc0043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbbab0 0x55f705cbf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbbab0 0x55f705cbf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41380/0 (socket says 172.21.15.32:41380) 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 -- 172.21.15.32:0/2371889887 learned_addr learned my addr 172.21.15.32:0/2371889887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbc3d0 0x55f705cbf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb12d9d640 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 0x55f705d3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 -- 172.21.15.32:0/2371889887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 msgr2=0x55f705d3a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 0x55f705d3a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 -- 172.21.15.32:0/2371889887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbc3d0 msgr2=0x55f705cbf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbc3d0 0x55f705cbf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 -- 172.21.15.32:0/2371889887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f705d3aea0 con 0x55f705cbbab0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1259c640 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbbab0 0x55f705cbf3e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbb040cddc0 tx=0x7fbb0409bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3592850d7698d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbaf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/2371889887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04020040 con 0x55f705cbbab0 2024-04-16T05:35:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/2371889887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbb04098af0 con 0x55f705cbbab0 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1b411740 1 -- 172.21.15.32:0/2371889887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc015030 con 0x55f705cbbab0 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/2371889887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb0409bfc0 con 0x55f705cbbab0 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1b411740 10 client.15813 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbb1b411740 1 -- 172.21.15.32:0/2371889887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f705d3c150 con 0x55f705cbbab0 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.343+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/2371889887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc015030 con 0x55f705cbbab0 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/2371889887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb040bb2a0 con 0x55f705cbbab0 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbb1b411740 10 client.15813 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbb1b411740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbb1b411740 20 client.15813 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbb1b411740 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f705d3c440 0x55f705d3e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbb1b411740 4 client.15813 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbb1b411740 1 -- 172.21.15.32:0/2371889887 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f705c28720 con 0x55f705d3c440 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f705d3c440 0x55f705d3e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbb0bfff640 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f705d3c440 0x55f705d3e830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8e14c07fb4575d server_cookie=32d42cb4c43642c2 in_seq=0 out_seq=0 2024-04-16T05:35:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.344+0000 7fbaf9ffb640 10 client.15813 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.345+0000 7fbaf9ffb640 1 -- 172.21.15.32:0/2371889887 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f705c28720 con 0x55f705d3c440 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.345+0000 7fbaf9ffb640 10 client.15813 handle_command_reply: tid=0 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.345+0000 7fbb1b411740 1 client.15813 shutdown 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.345+0000 7fbb1b411740 1 -- 172.21.15.32:0/2371889887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbbab0 msgr2=0x55f705cbf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.345+0000 7fbb1b411740 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbbab0 0x55f705cbf3e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbb040cddc0 tx=0x7fbb0409bf00 comp rx=0 tx=0).stop 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.345+0000 7fbb1b411740 1 -- 172.21.15.32:0/2371889887 shutdown_connections 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.345+0000 7fbb1b411740 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f705d3c440 0x55f705d3e830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1b411740 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f705cc4fb0 0x55f705d3a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1b411740 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f705cbc3d0 0x55f705cbf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbaf9ffb640 0 client.15813 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1b411740 1 --2- 172.21.15.32:0/2371889887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f705cbbab0 0x55f705cbf3e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1b411740 1 -- 172.21.15.32:0/2371889887 >> 172.21.15.32:0/2371889887 conn(0x55f705cb61e0 msgr2=0x55f705cc1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1b411740 1 -- 172.21.15.32:0/2371889887 shutdown_connections 2024-04-16T05:35:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1b411740 1 -- 172.21.15.32:0/2371889887 wait complete. 2024-04-16T05:35:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1b411740 20 client.15813 trim_cache size 0 max 16384 2024-04-16T05:35:34.350 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:35:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:35:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "00ea095b-0798-4adc-99ab-e1c3ae642c9a", 2024-04-16T05:35:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:35:34.350 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 -- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbb000a3de0 msgr2=0x7fbb000a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbb000a3de0 0x7fbb000a62a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005e30 tx=0x7fbb0c032df0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 -- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14102240 msgr2=0x7fbb14074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14102240 0x7fbb14074300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400d950 tx=0x7fbb0400de10 comp rx=0 tx=0).stop 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 -- 172.21.15.32:0/2267796751 shutdown_connections 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb14074d80 0x7fbb141a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb14107310 0x7fbb14074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbb000a3de0 0x7fbb000a62a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 --2- 172.21.15.32:0/2267796751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb14102240 0x7fbb14074300 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 -- 172.21.15.32:0/2267796751 >> 172.21.15.32:0/2267796751 conn(0x7fbb140fda40 msgr2=0x7fbb140ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 -- 172.21.15.32:0/2267796751 shutdown_connections 2024-04-16T05:35:34.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.346+0000 7fbb1882a640 1 -- 172.21.15.32:0/2267796751 wait complete. 2024-04-16T05:35:34.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:35:34.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:35:34.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 -- 172.21.15.32:0/533584477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b70101550 msgr2=0x7f8b70106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 --2- 172.21.15.32:0/533584477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b70101550 0x7f8b70106c80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009900 tx=0x7f8b5c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 -- 172.21.15.32:0/533584477 shutdown_connections 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 --2- 172.21.15.32:0/533584477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b70107360 0x7f8b7010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 --2- 172.21.15.32:0/533584477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b70101550 0x7f8b70106c80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 --2- 172.21.15.32:0/533584477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b70100ba0 0x7f8b70100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 -- 172.21.15.32:0/533584477 >> 172.21.15.32:0/533584477 conn(0x7f8b70076190 msgr2=0x7f8b700765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 -- 172.21.15.32:0/533584477 shutdown_connections 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.465+0000 7f8b76f22640 1 -- 172.21.15.32:0/533584477 wait complete. 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b76f22640 1 Processor -- start 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b76f22640 1 -- start start 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b76f22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b70100ba0 0x7f8b701a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b76f22640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b70107360 0x7f8b701a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b76f22640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b7019a990 0x7f8b7019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b76f22640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70112410 con 0x7f8b7019a990 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b76f22640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70112290 con 0x7f8b70100ba0 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b76f22640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70112590 con 0x7f8b70107360 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b7019a990 0x7f8b7019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b7019a990 0x7f8b7019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41410/0 (socket says 172.21.15.32:41410) 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 -- 172.21.15.32:0/2422836943 learned_addr learned my addr 172.21.15.32:0/2422836943 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b70100ba0 0x7f8b701a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b70107360 0x7f8b701a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 -- 172.21.15.32:0/2422836943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b70107360 msgr2=0x7f8b701a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b70107360 0x7f8b701a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 -- 172.21.15.32:0/2422836943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b70100ba0 msgr2=0x7f8b701a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b70100ba0 0x7f8b701a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 -- 172.21.15.32:0/2422836943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7019b4b0 con 0x7f8b7019a990 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.466+0000 7f8b754bf640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b7019a990 0x7f8b7019ae40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800eaa0 tx=0x7f8b6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.467+0000 7f8b6dffb640 1 -- 172.21.15.32:0/2422836943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800cdb0 con 0x7f8b7019a990 2024-04-16T05:35:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.467+0000 7f8b6dffb640 1 -- 172.21.15.32:0/2422836943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b68005020 con 0x7f8b7019a990 2024-04-16T05:35:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.467+0000 7f8b76f22640 1 -- 172.21.15.32:0/2422836943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7019b7a0 con 0x7f8b7019a990 2024-04-16T05:35:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.467+0000 7f8b6dffb640 1 -- 172.21.15.32:0/2422836943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b680116b0 con 0x7f8b7019a990 2024-04-16T05:35:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.467+0000 7f8b76f22640 1 -- 172.21.15.32:0/2422836943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b701a79a0 con 0x7f8b7019a990 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.469+0000 7f8b6dffb640 1 -- 172.21.15.32:0/2422836943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8b68005560 con 0x7f8b7019a990 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.469+0000 7f8b79bdf740 1 Processor -- start 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.469+0000 7f8b79bdf740 1 -- start start 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6dffb640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b600a3db0 0x7f8b600a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420be50 0x56425420c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x564254214e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254217740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564254171180 con 0x56425420c770 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56425410e360 con 0x564254215350 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56425416d140 con 0x56425420be50 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b600a3db0 0x7f8b600a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x564254214e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6dffb640 1 -- 172.21.15.32:0/2422836943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b680d0f20 con 0x7f8b7019a990 2024-04-16T05:35:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x564254214e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41418/0 (socket says 172.21.15.32:41418) 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 -- 172.21.15.32:0/4146418149 learned_addr learned my addr 172.21.15.32:0/4146418149 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b754bf640 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254217740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420be50 0x56425420c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 -- 172.21.15.32:0/4146418149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420be50 msgr2=0x56425420c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420be50 0x56425420c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 -- 172.21.15.32:0/4146418149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 msgr2=0x564254217740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254217740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b6ffff640 1 -- 172.21.15.32:0/4146418149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564253f4d7d0 con 0x56425420c770 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420be50 0x56425420c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.470+0000 7f8b754bf640 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254217740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x564254214e10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c005c40 tx=0x7f8b5c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c483dcae480e2e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b600a3db0 0x7f8b600a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c02cfd0 tx=0x7f8b5c02e000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b5affd640 1 -- 172.21.15.32:0/4146418149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c0039c0 con 0x56425420c770 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b5affd640 1 -- 172.21.15.32:0/4146418149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c050070 con 0x56425420c770 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b5affd640 1 -- 172.21.15.32:0/4146418149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c043b90 con 0x56425420c770 2024-04-16T05:35:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b79bdf740 1 -- 172.21.15.32:0/4146418149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 msgr2=0x564254214e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x564254214e10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c005c40 tx=0x7f8b5c0040c0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b79bdf740 1 -- 172.21.15.32:0/4146418149 shutdown_connections 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254217740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x564254214e10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/4146418149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420be50 0x56425420c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.471+0000 7f8b79bdf740 1 -- 172.21.15.32:0/4146418149 >> 172.21.15.32:0/4146418149 conn(0x564254206580 msgr2=0x564254206960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 -- 172.21.15.32:0/4146418149 shutdown_connections 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 -- 172.21.15.32:0/4146418149 wait complete. 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 -- start start 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 Processor -- start 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56425420be50 0x5642542124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x5642542129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564254215350 0x564254212f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5642542180e0 con 0x56425420c770 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56425412a2c0 con 0x56425420be50 2024-04-16T05:35:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564254218260 con 0x564254215350 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b79bdf740 10 client.0 _ll_register_callbacks cb 0x564254111090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.472+0000 7f8b6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x5642542129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x5642542129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41422/0 (socket says 172.21.15.32:41422) 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b6ffff640 1 -- 172.21.15.32:0/2354497761 learned_addr learned my addr 172.21.15.32:0/2354497761 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56425420be50 0x5642542124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b754bf640 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564254215350 0x564254212f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b6ffff640 1 -- 172.21.15.32:0/2354497761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564254215350 msgr2=0x564254212f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564254215350 0x564254212f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b6ffff640 1 -- 172.21.15.32:0/2354497761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56425420be50 msgr2=0x5642542124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56425420be50 0x5642542124a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b6ffff640 1 -- 172.21.15.32:0/2354497761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564253e833f0 con 0x56425420c770 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b754bf640 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564254215350 0x564254212f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x5642542129e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c02bc70 tx=0x7f8b5c043680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2e38300c1d8ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b4ffff640 1 -- 172.21.15.32:0/2354497761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c058070 con 0x56425420c770 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b4ffff640 1 -- 172.21.15.32:0/2354497761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b5c043900 con 0x56425420c770 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b79bdf740 1 -- 172.21.15.32:0/2354497761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b5c050070 con 0x56425420c770 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b4ffff640 1 -- 172.21.15.32:0/2354497761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b5c043b00 con 0x56425420c770 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b79bdf740 10 client.15825 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b79bdf740 1 -- 172.21.15.32:0/2354497761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564253e430f0 con 0x56425420c770 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.473+0000 7f8b4ffff640 1 -- 172.21.15.32:0/2354497761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c050070 con 0x56425420c770 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b4ffff640 1 -- 172.21.15.32:0/2354497761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8b5c04a030 con 0x56425420c770 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b79bdf740 10 client.15825 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b79bdf740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b79bdf740 20 client.15825 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564254225630 0x564254227a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b79bdf740 4 client.15825 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b79bdf740 1 -- 172.21.15.32:0/2354497761 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425417eba0 con 0x564254225630 2024-04-16T05:35:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564254225630 0x564254227a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b4ffff640 10 client.15825 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.474+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564254225630 0x564254227a20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d421e4ce29e4f7a1 server_cookie=c0663fc3f032ceda in_seq=0 out_seq=0 2024-04-16T05:35:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b4ffff640 1 -- 172.21.15.32:0/2354497761 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56425417eba0 con 0x564254225630 2024-04-16T05:35:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b4ffff640 10 client.15825 handle_command_reply: tid=0 2024-04-16T05:35:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 client.15825 shutdown 2024-04-16T05:35:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 -- 172.21.15.32:0/2354497761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 msgr2=0x5642542129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x5642542129e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c02bc70 tx=0x7f8b5c043680 comp rx=0 tx=0).stop 2024-04-16T05:35:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 -- 172.21.15.32:0/2354497761 shutdown_connections 2024-04-16T05:35:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564254225630 0x564254227a20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564254215350 0x564254212f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420c770 0x5642542129e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/2354497761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56425420be50 0x5642542124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b4ffff640 0 client.15825 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:34.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 -- 172.21.15.32:0/2354497761 >> 172.21.15.32:0/2354497761 conn(0x564254206580 msgr2=0x5642542082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 -- 172.21.15.32:0/2354497761 shutdown_connections 2024-04-16T05:35:34.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.475+0000 7f8b79bdf740 1 -- 172.21.15.32:0/2354497761 wait complete. 2024-04-16T05:35:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.476+0000 7f8b79bdf740 20 client.15825 trim_cache size 0 max 16384 2024-04-16T05:35:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b79bdf740 1 Processor -- start 2024-04-16T05:35:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b79bdf740 1 -- start start 2024-04-16T05:35:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254214bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x564254212d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254220bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701b3d70 con 0x56425420be50 2024-04-16T05:35:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701b1640 con 0x564254215350 2024-04-16T05:35:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701af490 con 0x56425420c770 2024-04-16T05:35:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.493+0000 7f8b74cbe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254214bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b74cbe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254214bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41434/0 (socket says 172.21.15.32:41434) 2024-04-16T05:35:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b74cbe640 1 -- 172.21.15.32:0/1710854483 learned_addr learned my addr 172.21.15.32:0/1710854483 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/1710854483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x564254212d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b74cbe640 1 -- 172.21.15.32:0/1710854483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 msgr2=0x564254212d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/1710854483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x564254212d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b74cbe640 1 -- 172.21.15.32:0/1710854483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 msgr2=0x564254220bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/1710854483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254220bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b74cbe640 1 -- 172.21.15.32:0/1710854483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425420cbb0 con 0x56425420be50 2024-04-16T05:35:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/1710854483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254214bd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8b64005d00 tx=0x7f8b6400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883874be44115031 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b5a7fc640 1 -- 172.21.15.32:0/1710854483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64004220 con 0x56425420be50 2024-04-16T05:35:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b5a7fc640 1 -- 172.21.15.32:0/1710854483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b64023030 con 0x56425420be50 2024-04-16T05:35:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.494+0000 7f8b5a7fc640 1 -- 172.21.15.32:0/1710854483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64003b30 con 0x56425420be50 2024-04-16T05:35:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 -- 172.21.15.32:0/1710854483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 msgr2=0x564254214bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/1710854483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254214bd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8b64005d00 tx=0x7f8b6400b410 comp rx=0 tx=0).stop 2024-04-16T05:35:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 -- 172.21.15.32:0/1710854483 shutdown_connections 2024-04-16T05:35:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/1710854483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254220bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/1710854483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x564254212d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/1710854483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254214bd0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 -- 172.21.15.32:0/1710854483 >> 172.21.15.32:0/1710854483 conn(0x564254206580 msgr2=0x5642542204b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 -- 172.21.15.32:0/1710854483 shutdown_connections 2024-04-16T05:35:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 -- 172.21.15.32:0/1710854483 wait complete. 2024-04-16T05:35:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 -- start start 2024-04-16T05:35:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.495+0000 7f8b79bdf740 1 Processor -- start 2024-04-16T05:35:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254229de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x56425422a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b79bdf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254283850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701afb00 con 0x56425420be50 2024-04-16T05:35:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701b0f70 con 0x564254215350 2024-04-16T05:35:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b79bdf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701b01d0 con 0x56425420c770 2024-04-16T05:35:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b79bdf740 10 client.0 _ll_register_callbacks cb 0x7f8b680ce0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:34.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254229de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254229de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41440/0 (socket says 172.21.15.32:41440) 2024-04-16T05:35:34.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 -- 172.21.15.32:0/3268367391 learned_addr learned my addr 172.21.15.32:0/3268367391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:34.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x56425422a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 -- 172.21.15.32:0/3268367391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 msgr2=0x56425422a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x56425422a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b754bf640 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254283850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 -- 172.21.15.32:0/3268367391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 msgr2=0x564254283850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254283850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 -- 172.21.15.32:0/3268367391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564254283e90 con 0x56425420be50 2024-04-16T05:35:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x56425422a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b74cbe640 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254229de0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400bfd0 tx=0x7f8b640051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e4a32e9f99432e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:34.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:34.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b4ffff640 1 -- 172.21.15.32:0/3268367391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640055c0 con 0x56425420be50 2024-04-16T05:35:34.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b4ffff640 1 -- 172.21.15.32:0/3268367391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b64004730 con 0x56425420be50 2024-04-16T05:35:34.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.496+0000 7f8b79bdf740 1 -- 172.21.15.32:0/3268367391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b64023030 con 0x56425420be50 2024-04-16T05:35:34.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b4ffff640 1 -- 172.21.15.32:0/3268367391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b64038050 con 0x56425420be50 2024-04-16T05:35:34.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b79bdf740 10 client.15831 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:34.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b79bdf740 1 -- 172.21.15.32:0/3268367391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56425428b270 con 0x56425420be50 2024-04-16T05:35:34.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b4ffff640 1 -- 172.21.15.32:0/3268367391 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b64023030 con 0x56425420be50 2024-04-16T05:35:34.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b4ffff640 1 -- 172.21.15.32:0/3268367391 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8b640324d0 con 0x56425420be50 2024-04-16T05:35:34.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b79bdf740 10 client.15831 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:34.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b79bdf740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:34.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b79bdf740 20 client.15831 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56425428b910 0x56425428dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b79bdf740 4 client.15831 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:35:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b79bdf740 1 -- 172.21.15.32:0/3268367391 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564254178a10 con 0x56425428b910 2024-04-16T05:35:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.497+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56425428b910 0x56425428dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b4ffff640 10 client.15831 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b6ffff640 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56425428b910 0x56425428dd30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96e93004f32dd1b server_cookie=c97c3cf6511ee045 in_seq=0 out_seq=0 2024-04-16T05:35:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b4ffff640 1 -- 172.21.15.32:0/3268367391 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564254178a10 con 0x56425428b910 2024-04-16T05:35:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b4ffff640 10 client.15831 handle_command_reply: tid=0 2024-04-16T05:35:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 client.15831 shutdown 2024-04-16T05:35:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 -- 172.21.15.32:0/3268367391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 msgr2=0x564254229de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254229de0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400bfd0 tx=0x7f8b640051a0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 -- 172.21.15.32:0/3268367391 shutdown_connections 2024-04-16T05:35:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564254215350 0x564254283850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56425428b910 0x56425428dd30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56425420c770 0x56425422a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 --2- 172.21.15.32:0/3268367391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56425420be50 0x564254229de0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 -- 172.21.15.32:0/3268367391 >> 172.21.15.32:0/3268367391 conn(0x564254206580 msgr2=0x564254205760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b4ffff640 0 client.15831 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 -- 172.21.15.32:0/3268367391 shutdown_connections 2024-04-16T05:35:34.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 1 -- 172.21.15.32:0/3268367391 wait complete. 2024-04-16T05:35:34.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.498+0000 7f8b79bdf740 20 client.15831 trim_cache size 0 max 16384 2024-04-16T05:35:34.519 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:35:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:35:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:35:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "00ea095b-0798-4adc-99ab-e1c3ae642c9a": { 2024-04-16T05:35:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:35:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "00ea095b-0798-4adc-99ab-e1c3ae642c9a", 2024-04-16T05:35:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:35:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:35:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:35:34.521 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:35:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 -- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b600a3db0 msgr2=0x7f8b600a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b600a3db0 0x7f8b600a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c02cfd0 tx=0x7f8b5c02e000 comp rx=0 tx=0).stop 2024-04-16T05:35:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 -- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b7019a990 msgr2=0x7f8b7019ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b7019a990 0x7f8b7019ae40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800eaa0 tx=0x7f8b6800ef60 comp rx=0 tx=0).stop 2024-04-16T05:35:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 -- 172.21.15.32:0/2422836943 shutdown_connections 2024-04-16T05:35:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b600a3db0 0x7f8b600a6270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b7019a990 0x7f8b7019ae40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b70107360 0x7f8b701a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 --2- 172.21.15.32:0/2422836943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b70100ba0 0x7f8b701a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 -- 172.21.15.32:0/2422836943 >> 172.21.15.32:0/2422836943 conn(0x7f8b70076190 msgr2=0x7f8b700fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 -- 172.21.15.32:0/2422836943 shutdown_connections 2024-04-16T05:35:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:35:34.499+0000 7f8b76f22640 1 -- 172.21.15.32:0/2422836943 wait complete. 2024-04-16T05:35:34.525 INFO:tasks.cephfs.filesystem:scrub status for tag:00ea095b-0798-4adc-99ab-e1c3ae642c9a - {'path': '/', 'tag': '00ea095b-0798-4adc-99ab-e1c3ae642c9a', 'options': 'recursive,force'} 2024-04-16T05:35:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:34 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:35:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:34 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:34 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:35:34.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:34 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:34.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:34 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:35:34.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:34 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:34.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:34 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:35:34.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:34 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:35.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:34 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:35:35.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:34 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:35.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:34 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:35:35.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:34 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:35:35.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:35 smithi082 ceph-mon[36195]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-16T05:35:35.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:35 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:35:35.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:35 smithi032 ceph-mon[32351]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-16T05:35:35.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:35 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:35:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:35 smithi134 ceph-mon[36153]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-16T05:35:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:35 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:35:37.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:37 smithi082 ceph-mon[36195]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-16T05:35:37.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:37 smithi032 ceph-mon[32351]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-16T05:35:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:37 smithi134 ceph-mon[36153]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-16T05:35:39.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:39 smithi082 ceph-mon[36195]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-16T05:35:39.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:39 smithi032 ceph-mon[32351]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-16T05:35:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:39 smithi134 ceph-mon[36153]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-16T05:35:41.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:41 smithi082 ceph-mon[36195]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-16T05:35:41.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:41 smithi032 ceph-mon[32351]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-16T05:35:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:41 smithi134 ceph-mon[36153]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-16T05:35:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 -- 172.21.15.32:0/4053864619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 msgr2=0x7f1548101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 --2- 172.21.15.32:0/4053864619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 0x7f1548101580 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1538009ca0 tx=0x7f153802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 -- 172.21.15.32:0/4053864619 shutdown_connections 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 --2- 172.21.15.32:0/4053864619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1548106220 0x7f154810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 --2- 172.21.15.32:0/4053864619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1548101b50 0x7f1548105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 --2- 172.21.15.32:0/4053864619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 0x7f1548101580 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 -- 172.21.15.32:0/4053864619 >> 172.21.15.32:0/4053864619 conn(0x7f1548078080 msgr2=0x7f1548078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 -- 172.21.15.32:0/4053864619 shutdown_connections 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 -- 172.21.15.32:0/4053864619 wait complete. 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 Processor -- start 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.005+0000 7f154fb3d640 1 -- start start 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154fb3d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 0x7f154810e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154fb3d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1548106220 0x7f154810eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154fb3d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154810f000 0x7f15481a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154fb3d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15481123e0 con 0x7f15481011a0 2024-04-16T05:35:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154fb3d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1548112260 con 0x7f154810f000 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154fb3d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1548112560 con 0x7f1548106220 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 0x7f154810e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 0x7f154810e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54628/0 (socket says 172.21.15.32:54628) 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 -- 172.21.15.32:0/1089368528 learned_addr learned my addr 172.21.15.32:0/1089368528 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154e0da640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154810f000 0x7f15481a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d0d8640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1548106220 0x7f154810eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 -- 172.21.15.32:0/1089368528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1548106220 msgr2=0x7f154810eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1548106220 0x7f154810eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 -- 172.21.15.32:0/1089368528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154810f000 msgr2=0x7f15481a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154810f000 0x7f15481a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 -- 172.21.15.32:0/1089368528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15481a70f0 con 0x7f15481011a0 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f154d8d9640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 0x7f154810e580 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1538005e30 tx=0x7f1538032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.006+0000 7f1546ffd640 1 -- 172.21.15.32:0/1089368528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538041020 con 0x7f15481011a0 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.007+0000 7f1546ffd640 1 -- 172.21.15.32:0/1089368528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1538034d90 con 0x7f15481011a0 2024-04-16T05:35:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.007+0000 7f154fb3d640 1 -- 172.21.15.32:0/1089368528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15481a7380 con 0x7f15481011a0 2024-04-16T05:35:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.007+0000 7f1546ffd640 1 -- 172.21.15.32:0/1089368528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538039040 con 0x7f15481011a0 2024-04-16T05:35:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.007+0000 7f154fb3d640 1 -- 172.21.15.32:0/1089368528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15481a7870 con 0x7f15481011a0 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.009+0000 7f1546ffd640 1 -- 172.21.15.32:0/1089368528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1538044020 con 0x7f15481011a0 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.009+0000 7f1552804740 1 Processor -- start 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1552804740 1 -- start start 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1552804740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634baaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1552804740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1546ffd640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f15300a3e90 0x7f15300a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1552804740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 0x55f634bb6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1552804740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f634b0f7d0 con 0x55f634bab530 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1552804740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f634aacf70 con 0x55f634bb4110 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1552804740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f634b0b790 con 0x55f634baac10 2024-04-16T05:35:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d0d8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d8d9640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f15300a3e90 0x7f15300a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d0d8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54638/0 (socket says 172.21.15.32:54638) 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d0d8640 1 -- 172.21.15.32:0/673082641 learned_addr learned my addr 172.21.15.32:0/673082641 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154e0da640 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634baaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f1546ffd640 1 -- 172.21.15.32:0/1089368528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f15380f2a40 con 0x7f15481011a0 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154e0da640 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 0x55f634bb6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d0d8640 1 -- 172.21.15.32:0/673082641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 msgr2=0x55f634baaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d0d8640 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634baaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d0d8640 1 -- 172.21.15.32:0/673082641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 msgr2=0x55f634bb6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d0d8640 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 0x55f634bb6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.010+0000 7f154d0d8640 1 -- 172.21.15.32:0/673082641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f634822210 con 0x55f634bab530 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.011+0000 7f154d8d9640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f15300a3e90 0x7f15300a6350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15380eff00 tx=0x7f15380b3ad0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.011+0000 7f154d0d8640 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb3bd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f153c009bf0 tx=0x7f153c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7539e40682a1d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.011+0000 7f152ffff640 1 -- 172.21.15.32:0/673082641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c004030 con 0x55f634bab530 2024-04-16T05:35:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.011+0000 7f152ffff640 1 -- 172.21.15.32:0/673082641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f153c0041d0 con 0x55f634bab530 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.011+0000 7f152ffff640 1 -- 172.21.15.32:0/673082641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c012690 con 0x55f634bab530 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 -- 172.21.15.32:0/673082641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 msgr2=0x55f634bb3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb3bd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f153c009bf0 tx=0x7f153c0117b0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 -- 172.21.15.32:0/673082641 shutdown_connections 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 0x55f634bb6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb3bd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 --2- 172.21.15.32:0/673082641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634baaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 -- 172.21.15.32:0/673082641 >> 172.21.15.32:0/673082641 conn(0x55f634ba5340 msgr2=0x55f634ba5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 -- 172.21.15.32:0/673082641 shutdown_connections 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 -- 172.21.15.32:0/673082641 wait complete. 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 -- start start 2024-04-16T05:35:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.012+0000 7f1552804740 1 Processor -- start 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f1552804740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f1552804740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 0x55f634bb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f1552804740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bb4110 0x55f634bb1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f1552804740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f634bb6f00 con 0x55f634bb4110 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f1552804740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f634a97e90 con 0x55f634bab530 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f1552804740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f634bb7080 con 0x55f634baac10 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f1552804740 10 client.0 _ll_register_callbacks cb 0x55f634acbcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bb4110 0x55f634bb1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bb4110 0x55f634bb1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54654/0 (socket says 172.21.15.32:54654) 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 -- 172.21.15.32:0/656009524 learned_addr learned my addr 172.21.15.32:0/656009524 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154d0d8640 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 0x55f634bb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 -- 172.21.15.32:0/656009524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 msgr2=0x55f634bb1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 -- 172.21.15.32:0/656009524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 msgr2=0x55f634bb1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 0x55f634bb1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 -- 172.21.15.32:0/656009524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6348ee620 con 0x55f634bb4110 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f154e0da640 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bb4110 0x55f634bb1bc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1540013830 tx=0x7f154000ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513225ed21e89764 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.013+0000 7f152d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f152d7fa640 1 -- 172.21.15.32:0/656009524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540022060 con 0x55f634bb4110 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f152d7fa640 1 -- 172.21.15.32:0/656009524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1540011cf0 con 0x55f634bb4110 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 1 -- 172.21.15.32:0/656009524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f153c0041d0 con 0x55f634bb4110 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f152d7fa640 1 -- 172.21.15.32:0/656009524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1540029050 con 0x55f634bb4110 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 10 client.15843 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 1 -- 172.21.15.32:0/656009524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f634bc3230 con 0x55f634bb4110 2024-04-16T05:35:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f152d7fa640 1 -- 172.21.15.32:0/656009524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f153c0041d0 con 0x55f634bb4110 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f152d7fa640 1 -- 172.21.15.32:0/656009524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f154001c430 con 0x55f634bb4110 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 10 client.15843 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 20 client.15843 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f634bc46f0 0x55f634bc6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 4 client.15843 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.014+0000 7f1552804740 1 -- 172.21.15.32:0/656009524 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f634aaf380 con 0x55f634bc46f0 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.015+0000 7f154d0d8640 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f634bc46f0 0x55f634bc6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.015+0000 7f154d0d8640 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f634bc46f0 0x55f634bc6ae0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8318520758b74e server_cookie=52f3a231a66d927e in_seq=0 out_seq=0 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.015+0000 7f152d7fa640 10 client.15843 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f152d7fa640 1 -- 172.21.15.32:0/656009524 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f634aaf380 con 0x55f634bc46f0 2024-04-16T05:35:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f152d7fa640 10 client.15843 handle_command_reply: tid=0 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 client.15843 shutdown 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 -- 172.21.15.32:0/656009524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bb4110 msgr2=0x55f634bb1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bb4110 0x55f634bb1bc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1540013830 tx=0x7f154000ba60 comp rx=0 tx=0).stop 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 -- 172.21.15.32:0/656009524 shutdown_connections 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bb4110 0x55f634bb1bc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 0x55f634bb1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f634bc46f0 0x55f634bc6ae0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 --2- 172.21.15.32:0/656009524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f152d7fa640 0 client.15843 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 -- 172.21.15.32:0/656009524 >> 172.21.15.32:0/656009524 conn(0x55f634ba5340 msgr2=0x55f634ba7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 -- 172.21.15.32:0/656009524 shutdown_connections 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 1 -- 172.21.15.32:0/656009524 wait complete. 2024-04-16T05:35:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.016+0000 7f1552804740 20 client.15843 trim_cache size 0 max 16384 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.031+0000 7f1552804740 1 Processor -- start 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.031+0000 7f1552804740 1 -- start start 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f1552804740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634baac10 0x55f634bb3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f1552804740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 0x55f634bbfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f1552804740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634bb4110 0x55f634ba42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f1552804740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15481b3d10 con 0x55f634baac10 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f1552804740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f15481b15e0 con 0x55f634bab530 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f1552804740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15481af430 con 0x55f634bb4110 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634baac10 0x55f634bb3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634baac10 0x55f634bb3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54658/0 (socket says 172.21.15.32:54658) 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 -- 172.21.15.32:0/116833099 learned_addr learned my addr 172.21.15.32:0/116833099 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 -- 172.21.15.32:0/116833099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634bb4110 msgr2=0x55f634ba42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 --2- 172.21.15.32:0/116833099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634bb4110 0x55f634ba42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 -- 172.21.15.32:0/116833099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 msgr2=0x55f634bbfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 --2- 172.21.15.32:0/116833099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 0x55f634bbfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 -- 172.21.15.32:0/116833099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f634bb6500 con 0x55f634baac10 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f154e0da640 1 --2- 172.21.15.32:0/116833099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634baac10 0x55f634bb3870 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1540005bc0 tx=0x7f154000bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30105a764336fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f152f7fe640 1 -- 172.21.15.32:0/116833099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540032070 con 0x55f634baac10 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f152f7fe640 1 -- 172.21.15.32:0/116833099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154001cc20 con 0x55f634baac10 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.032+0000 7f152f7fe640 1 -- 172.21.15.32:0/116833099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540013400 con 0x55f634baac10 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 -- 172.21.15.32:0/116833099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634baac10 msgr2=0x55f634bb3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 --2- 172.21.15.32:0/116833099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634baac10 0x55f634bb3870 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1540005bc0 tx=0x7f154000bd20 comp rx=0 tx=0).stop 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 -- 172.21.15.32:0/116833099 shutdown_connections 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 --2- 172.21.15.32:0/116833099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634bb4110 0x55f634ba42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 --2- 172.21.15.32:0/116833099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bab530 0x55f634bbfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 --2- 172.21.15.32:0/116833099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634baac10 0x55f634bb3870 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 -- 172.21.15.32:0/116833099 >> 172.21.15.32:0/116833099 conn(0x55f634ba5340 msgr2=0x55f634ba3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 -- 172.21.15.32:0/116833099 shutdown_connections 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 -- 172.21.15.32:0/116833099 wait complete. 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 -- start start 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.033+0000 7f1552804740 1 Processor -- start 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f1552804740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f1552804740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f1552804740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 0x55f634bb8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f1552804740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15481afaa0 con 0x55f634bab530 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f1552804740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f15481b0f10 con 0x55f634bb4110 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f1552804740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15481b0170 con 0x55f634baac10 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f1552804740 10 client.0 _ll_register_callbacks cb 0x7f154000bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154d0d8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154d0d8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54664/0 (socket says 172.21.15.32:54664) 2024-04-16T05:35:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154d0d8640 1 -- 172.21.15.32:0/2316832889 learned_addr learned my addr 172.21.15.32:0/2316832889 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154e0da640 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154e0da640 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 0x55f634bb8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154d0d8640 1 -- 172.21.15.32:0/2316832889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 msgr2=0x55f634bb7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154d0d8640 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154d0d8640 1 -- 172.21.15.32:0/2316832889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 msgr2=0x55f634bb8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154d0d8640 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 0x55f634bb8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154d0d8640 1 -- 172.21.15.32:0/2316832889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f634c29c60 con 0x55f634bab530 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.034+0000 7f154e0da640 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f154d0d8640 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb8300 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55f634baa6e0 tx=0x7f153c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1508f58daa77255 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f152d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f152d7fa640 1 -- 172.21.15.32:0/2316832889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c02ae60 con 0x55f634bab530 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f152d7fa640 1 -- 172.21.15.32:0/2316832889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f153c0035f0 con 0x55f634bab530 2024-04-16T05:35:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 1 -- 172.21.15.32:0/2316832889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154001cc20 con 0x55f634bab530 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f152d7fa640 1 -- 172.21.15.32:0/2316832889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f153c034050 con 0x55f634bab530 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 10 client.15849 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 1 -- 172.21.15.32:0/2316832889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f634c2af70 con 0x55f634bab530 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f152d7fa640 1 -- 172.21.15.32:0/2316832889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154001cc20 con 0x55f634bab530 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f152d7fa640 1 -- 172.21.15.32:0/2316832889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f153c02a220 con 0x55f634bab530 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 10 client.15849 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 20 client.15849 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f634c2b440 0x55f634c2d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 4 client.15849 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.035+0000 7f1552804740 1 -- 172.21.15.32:0/2316832889 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f634b1d1f0 con 0x55f634c2b440 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.036+0000 7f154e0da640 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f634c2b440 0x55f634c2d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.036+0000 7f154e0da640 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f634c2b440 0x55f634c2d860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf9f5d1490eb637 server_cookie=82fd59a0021b1a28 in_seq=0 out_seq=0 2024-04-16T05:35:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.036+0000 7f152d7fa640 10 client.15849 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f152d7fa640 1 -- 172.21.15.32:0/2316832889 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f634b1d1f0 con 0x55f634c2b440 2024-04-16T05:35:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f152d7fa640 10 client.15849 handle_command_reply: tid=0 2024-04-16T05:35:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 client.15849 shutdown 2024-04-16T05:35:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 -- 172.21.15.32:0/2316832889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 msgr2=0x55f634bb8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb8300 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55f634baa6e0 tx=0x7f153c003eb0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 -- 172.21.15.32:0/2316832889 shutdown_connections 2024-04-16T05:35:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f634bb4110 0x55f634bb8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f634bab530 0x55f634bb8300 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f634c2b440 0x55f634c2d860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 --2- 172.21.15.32:0/2316832889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f634baac10 0x55f634bb7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f152d7fa640 0 client.15849 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 -- 172.21.15.32:0/2316832889 >> 172.21.15.32:0/2316832889 conn(0x55f634ba5340 msgr2=0x55f634ba4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 -- 172.21.15.32:0/2316832889 shutdown_connections 2024-04-16T05:35:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 1 -- 172.21.15.32:0/2316832889 wait complete. 2024-04-16T05:35:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.043+0000 7f1552804740 20 client.15849 trim_cache size 0 max 16384 2024-04-16T05:35:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2024-04-16T05:35:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2024-04-16T05:35:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2024-04-16T05:35:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2024-04-16T05:35:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 -- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f15300a3e90 msgr2=0x7f15300a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f15300a3e90 0x7f15300a6350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15380eff00 tx=0x7f15380b3ad0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 -- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 msgr2=0x7f154810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 0x7f154810e580 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1538005e30 tx=0x7f1538032f20 comp rx=0 tx=0).stop 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 -- 172.21.15.32:0/1089368528 shutdown_connections 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154810f000 0x7f15481a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1548106220 0x7f154810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f15300a3e90 0x7f15300a6350 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 --2- 172.21.15.32:0/1089368528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15481011a0 0x7f154810e580 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 -- 172.21.15.32:0/1089368528 >> 172.21.15.32:0/1089368528 conn(0x7f1548078080 msgr2=0x7f15480fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 -- 172.21.15.32:0/1089368528 shutdown_connections 2024-04-16T05:35:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:43.044+0000 7f154fb3d640 1 -- 172.21.15.32:0/1089368528 wait complete. 2024-04-16T05:35:43.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:43 smithi082 ceph-mon[36195]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:35:43.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:43 smithi032 ceph-mon[32351]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:35:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:43 smithi134 ceph-mon[36153]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-16T05:35:45.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:45 smithi032 ceph-mon[32351]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-16T05:35:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:45 smithi134 ceph-mon[36153]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-16T05:35:46.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:45 smithi082 ceph-mon[36195]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-16T05:35:47.952 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:47 smithi082 ceph-mon[36195]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:35:47.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:47 smithi032 ceph-mon[32351]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:35:48.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:47 smithi134 ceph-mon[36153]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:35:49.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:49 smithi032 ceph-mon[32351]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-16T05:35:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:49 smithi134 ceph-mon[36153]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-16T05:35:50.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:49 smithi082 ceph-mon[36195]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-16T05:35:51.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:51 smithi032 ceph-mon[32351]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T05:35:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:51 smithi134 ceph-mon[36153]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T05:35:52.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:51 smithi082 ceph-mon[36195]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T05:35:53.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:53 smithi032 ceph-mon[32351]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T05:35:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:53 smithi134 ceph-mon[36153]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T05:35:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:53 smithi082 ceph-mon[36195]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T05:35:55.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:55 smithi032 ceph-mon[32351]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-16T05:35:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:55 smithi134 ceph-mon[36153]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 -- 172.21.15.32:0/3716253838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4101b50 msgr2=0x7facc4105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 --2- 172.21.15.32:0/3716253838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4101b50 0x7facc4105b40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7facb0009900 tx=0x7facb002bc30 comp rx=0 tx=0).stop 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 -- 172.21.15.32:0/3716253838 shutdown_connections 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 --2- 172.21.15.32:0/3716253838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facc4106220 0x7facc410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 --2- 172.21.15.32:0/3716253838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4101b50 0x7facc4105b40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 --2- 172.21.15.32:0/3716253838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc41011a0 0x7facc4101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 -- 172.21.15.32:0/3716253838 >> 172.21.15.32:0/3716253838 conn(0x7facc4078080 msgr2=0x7facc4078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 -- 172.21.15.32:0/3716253838 shutdown_connections 2024-04-16T05:35:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.137+0000 7faccb167640 1 -- 172.21.15.32:0/3716253838 wait complete. 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7faccb167640 1 Processor -- start 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7faccb167640 1 -- start start 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7faccb167640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc41011a0 0x7facc4199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7faccb167640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4106220 0x7facc4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7faccb167640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facc419a160 0x7facc41a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7faccb167640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7facc4111290 con 0x7facc4106220 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7faccb167640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7facc4111110 con 0x7facc419a160 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7faccb167640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7facc4111410 con 0x7facc41011a0 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7facc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4106220 0x7facc4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7facc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4106220 0x7facc4199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43742/0 (socket says 172.21.15.32:43742) 2024-04-16T05:35:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7facc3fff640 1 -- 172.21.15.32:0/3335589613 learned_addr learned my addr 172.21.15.32:0/3335589613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.138+0000 7facc8f03640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc41011a0 0x7facc4199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc9704640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facc419a160 0x7facc41a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc3fff640 1 -- 172.21.15.32:0/3335589613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc41011a0 msgr2=0x7facc4199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc3fff640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc41011a0 0x7facc4199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc3fff640 1 -- 172.21.15.32:0/3335589613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facc419a160 msgr2=0x7facc41a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc3fff640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facc419a160 0x7facc41a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc3fff640 1 -- 172.21.15.32:0/3335589613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc41a4f10 con 0x7facc4106220 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc9704640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facc419a160 0x7facc41a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc3fff640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4106220 0x7facc4199bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7facb0000c00 tx=0x7facb002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc1ffb640 1 -- 172.21.15.32:0/3335589613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0041070 con 0x7facc4106220 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc1ffb640 1 -- 172.21.15.32:0/3335589613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb0004450 con 0x7facc4106220 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7faccb167640 1 -- 172.21.15.32:0/3335589613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc41a51a0 con 0x7facc4106220 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7facc1ffb640 1 -- 172.21.15.32:0/3335589613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0034440 con 0x7facc4106220 2024-04-16T05:35:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.139+0000 7faccb167640 1 -- 172.21.15.32:0/3335589613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc41a5690 con 0x7facc4106220 2024-04-16T05:35:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.142+0000 7facc1ffb640 1 -- 172.21.15.32:0/3335589613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7facb0044070 con 0x7facc4106220 2024-04-16T05:35:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.142+0000 7faccde2f740 1 Processor -- start 2024-04-16T05:35:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.142+0000 7faccde2f740 1 -- start start 2024-04-16T05:35:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7faccde2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7faccde2f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 0x55e1021d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7faccde2f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021d9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7faccde2f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e1021327d0 con 0x55e1021d7110 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7faccde2f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e1020cff70 con 0x55e1021ce530 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7faccde2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e10212e790 con 0x55e1021cdc10 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021d9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021d9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43758/0 (socket says 172.21.15.32:43758) 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc3fff640 1 -- 172.21.15.32:0/2706331756 learned_addr learned my addr 172.21.15.32:0/2706331756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc8f03640 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 0x55e1021d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc9704640 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc1ffb640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7facb40a3e60 0x7facb40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc3fff640 1 -- 172.21.15.32:0/2706331756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 msgr2=0x55e1021cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc3fff640 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc3fff640 1 -- 172.21.15.32:0/2706331756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 msgr2=0x55e1021d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc3fff640 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 0x55e1021d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc3fff640 1 -- 172.21.15.32:0/2706331756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e101e45210 con 0x55e1021d7110 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc9704640 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc9704640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7facb40a3e60 0x7facb40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.143+0000 7facc1ffb640 1 -- 172.21.15.32:0/3335589613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7facb00f2950 con 0x7facc4106220 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.144+0000 7facc3fff640 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021d9500 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7facb00efd50 tx=0x7facb004d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f6e29eed583d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.144+0000 7facc9704640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7facb40a3e60 0x7facb40a6320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7facbc005fd0 tx=0x7facbc004a40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.144+0000 7facaeffd640 1 -- 172.21.15.32:0/2706331756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00be090 con 0x55e1021d7110 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.144+0000 7facaeffd640 1 -- 172.21.15.32:0/2706331756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb00bec50 con 0x55e1021d7110 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.144+0000 7facaeffd640 1 -- 172.21.15.32:0/2706331756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00bdeb0 con 0x55e1021d7110 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 -- 172.21.15.32:0/2706331756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 msgr2=0x55e1021d9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021d9500 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7facb00efd50 tx=0x7facb004d000 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 -- 172.21.15.32:0/2706331756 shutdown_connections 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021d9500 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 0x55e1021d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 --2- 172.21.15.32:0/2706331756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 -- 172.21.15.32:0/2706331756 >> 172.21.15.32:0/2706331756 conn(0x55e1021c8340 msgr2=0x55e1021c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 -- 172.21.15.32:0/2706331756 shutdown_connections 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 -- 172.21.15.32:0/2706331756 wait complete. 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 -- start start 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 Processor -- start 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.145+0000 7faccde2f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021cdc10 0x55e1021d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faccde2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021ce530 0x55e1021d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faccde2f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021d4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faccde2f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e1021d9cc0 con 0x55e1021cdc10 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faccde2f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e1020bae90 con 0x55e1021d7110 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faccde2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e1021d9e40 con 0x55e1021ce530 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faccde2f740 10 client.0 _ll_register_callbacks cb 0x55e1020eecc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021cdc10 0x55e1021d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021cdc10 0x55e1021d4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43764/0 (socket says 172.21.15.32:43764) 2024-04-16T05:35:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 -- 172.21.15.32:0/268328553 learned_addr learned my addr 172.21.15.32:0/268328553 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc9704640 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021d4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 -- 172.21.15.32:0/268328553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021ce530 msgr2=0x55e1021d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021ce530 0x55e1021d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 -- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 msgr2=0x55e1021d4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021d4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 -- 172.21.15.32:0/268328553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e101f11620 con 0x55e1021cdc10 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc9704640 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021d4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7facc8f03640 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021cdc10 0x55e1021d4180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7facb800ce30 tx=0x7facb80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736ca63f767c4b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faca3fff640 1 -- 172.21.15.32:0/268328553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8011970 con 0x55e1021cdc10 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faca3fff640 1 -- 172.21.15.32:0/268328553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7facb8011b10 con 0x55e1021cdc10 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faccde2f740 1 -- 172.21.15.32:0/268328553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb00bec50 con 0x55e1021cdc10 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.146+0000 7faca3fff640 1 -- 172.21.15.32:0/268328553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facb8011d10 con 0x55e1021cdc10 2024-04-16T05:35:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faccde2f740 10 client.15861 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faccde2f740 1 -- 172.21.15.32:0/268328553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1021e5f00 con 0x55e1021cdc10 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faca3fff640 1 -- 172.21.15.32:0/268328553 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb00bec50 con 0x55e1021cdc10 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faca3fff640 1 -- 172.21.15.32:0/268328553 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facb8018070 con 0x55e1021cdc10 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faccde2f740 10 client.15861 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faccde2f740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faccde2f740 20 client.15861 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faccde2f740 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e1021e7410 0x55e1021e9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faccde2f740 4 client.15861 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7faccde2f740 1 -- 172.21.15.32:0/268328553 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1020d2380 con 0x55e1021e7410 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.147+0000 7facc3fff640 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e1021e7410 0x55e1021e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.148+0000 7facc3fff640 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e1021e7410 0x55e1021e9840 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a1d751c7fa9e4d server_cookie=c7d54fe4eb8ea4aa in_seq=0 out_seq=0 2024-04-16T05:35:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.148+0000 7faca3fff640 10 client.15861 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:56.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:55 smithi082 ceph-mon[36195]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.148+0000 7faca3fff640 1 -- 172.21.15.32:0/268328553 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1020d2380 con 0x55e1021e7410 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.148+0000 7faca3fff640 10 client.15861 handle_command_reply: tid=0 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 client.15861 shutdown 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 -- 172.21.15.32:0/268328553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021cdc10 msgr2=0x55e1021d4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021cdc10 0x55e1021d4180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7facb800ce30 tx=0x7facb80039c0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 -- 172.21.15.32:0/268328553 shutdown_connections 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021d4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e1021e7410 0x55e1021e9840 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021ce530 0x55e1021d46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 --2- 172.21.15.32:0/268328553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021cdc10 0x55e1021d4180 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faca3fff640 0 client.15861 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 -- 172.21.15.32:0/268328553 >> 172.21.15.32:0/268328553 conn(0x55e1021c8340 msgr2=0x55e1021ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 -- 172.21.15.32:0/268328553 shutdown_connections 2024-04-16T05:35:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 1 -- 172.21.15.32:0/268328553 wait complete. 2024-04-16T05:35:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.149+0000 7faccde2f740 20 client.15861 trim_cache size 0 max 16384 2024-04-16T05:35:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7faccde2f740 1 Processor -- start 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7faccde2f740 1 -- start start 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7faccde2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7faccde2f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 0x55e1021e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7faccde2f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021ef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7faccde2f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7facc41b1b10 con 0x55e1021d7110 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7faccde2f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7facc41af3e0 con 0x55e1021ce530 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7faccde2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7facc41ad230 con 0x55e1021cdc10 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7facc9704640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7facc9704640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021ef150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43770/0 (socket says 172.21.15.32:43770) 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7facc9704640 1 -- 172.21.15.32:0/2025601397 learned_addr learned my addr 172.21.15.32:0/2025601397 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7facc3fff640 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 0x55e1021e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.164+0000 7facc8f03640 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facc9704640 1 -- 172.21.15.32:0/2025601397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 msgr2=0x55e1021d68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facc9704640 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021d68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facc9704640 1 -- 172.21.15.32:0/2025601397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 msgr2=0x55e1021e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facc9704640 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 0x55e1021e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facc9704640 1 -- 172.21.15.32:0/2025601397 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1021d26d0 con 0x55e1021d7110 2024-04-16T05:35:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facc9704640 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021ef150 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7facbc01d620 tx=0x7facbc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602ddcf6c555ada7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facae7fc640 1 -- 172.21.15.32:0/2025601397 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc011a50 con 0x55e1021d7110 2024-04-16T05:35:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facae7fc640 1 -- 172.21.15.32:0/2025601397 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facbc011bf0 con 0x55e1021d7110 2024-04-16T05:35:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.165+0000 7facae7fc640 1 -- 172.21.15.32:0/2025601397 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc01a750 con 0x55e1021d7110 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 -- 172.21.15.32:0/2025601397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 msgr2=0x55e1021ef150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021ef150 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7facbc01d620 tx=0x7facbc012000 comp rx=0 tx=0).stop 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 -- 172.21.15.32:0/2025601397 shutdown_connections 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021d7110 0x55e1021ef150 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021ce530 0x55e1021e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 --2- 172.21.15.32:0/2025601397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021d68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 -- 172.21.15.32:0/2025601397 >> 172.21.15.32:0/2025601397 conn(0x55e1021c8340 msgr2=0x55e1021ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 -- 172.21.15.32:0/2025601397 shutdown_connections 2024-04-16T05:35:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 -- 172.21.15.32:0/2025601397 wait complete. 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 -- start start 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.166+0000 7faccde2f740 1 Processor -- start 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faccde2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021daab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faccde2f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021ce530 0x55e1021daff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faccde2f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021db530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faccde2f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7facc41ad8a0 con 0x55e1021ce530 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faccde2f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7facc41aed10 con 0x55e1021d7110 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faccde2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7facc41adf70 con 0x55e1021cdc10 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faccde2f740 10 client.0 _ll_register_callbacks cb 0x7facb8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021ce530 0x55e1021daff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021ce530 0x55e1021daff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43776/0 (socket says 172.21.15.32:43776) 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 -- 172.21.15.32:0/3570924265 learned_addr learned my addr 172.21.15.32:0/3570924265 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc8f03640 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021daab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc9704640 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021db530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 -- 172.21.15.32:0/3570924265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 msgr2=0x55e1021daab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021daab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 -- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 msgr2=0x55e1021db530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021db530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 -- 172.21.15.32:0/3570924265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e10224caf0 con 0x55e1021ce530 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc9704640 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021db530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7facc3fff640 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021ce530 0x55e1021daff0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7facb00be4e0 tx=0x7facb0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eea0b339363eeee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faca3fff640 1 -- 172.21.15.32:0/3570924265 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00f8070 con 0x55e1021ce530 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faca3fff640 1 -- 172.21.15.32:0/3570924265 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7facb00f8210 con 0x55e1021ce530 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.167+0000 7faccde2f740 1 -- 172.21.15.32:0/3570924265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facbc011bf0 con 0x55e1021ce530 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faca3fff640 1 -- 172.21.15.32:0/3570924265 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facb00dc050 con 0x55e1021ce530 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faccde2f740 10 client.15867 fetch_fsmap learned FSMap version 25 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faccde2f740 1 -- 172.21.15.32:0/3570924265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e10224de10 con 0x55e1021ce530 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faca3fff640 1 -- 172.21.15.32:0/3570924265 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facbc011bf0 con 0x55e1021ce530 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faca3fff640 1 -- 172.21.15.32:0/3570924265 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facb008fae0 con 0x55e1021ce530 2024-04-16T05:35:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faccde2f740 10 client.15867 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:35:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faccde2f740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:35:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faccde2f740 20 client.15867 populate_metadata read hostname 'smithi032' 2024-04-16T05:35:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faccde2f740 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e10224e400 0x55e102250820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:35:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faccde2f740 4 client.15867 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:35:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7faccde2f740 1 -- 172.21.15.32:0/3570924265 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1021401f0 con 0x55e10224e400 2024-04-16T05:35:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.168+0000 7facc8f03640 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e10224e400 0x55e102250820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:35:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.169+0000 7faca3fff640 10 client.15867 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.169+0000 7facc8f03640 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e10224e400 0x55e102250820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9480a4acc19d3dd server_cookie=d4b3cb68fd12f205 in_seq=0 out_seq=0 2024-04-16T05:35:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.172+0000 7faca3fff640 1 -- 172.21.15.32:0/3570924265 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1021401f0 con 0x55e10224e400 2024-04-16T05:35:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.172+0000 7faca3fff640 10 client.15867 handle_command_reply: tid=0 2024-04-16T05:35:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.172+0000 7faccde2f740 1 client.15867 shutdown 2024-04-16T05:35:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.172+0000 7faccde2f740 1 -- 172.21.15.32:0/3570924265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021ce530 msgr2=0x55e1021daff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.172+0000 7faccde2f740 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021ce530 0x55e1021daff0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7facb00be4e0 tx=0x7facb0009680 comp rx=0 tx=0).stop 2024-04-16T05:35:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 1 -- 172.21.15.32:0/3570924265 shutdown_connections 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1021d7110 0x55e1021db530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e1021ce530 0x55e1021daff0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e10224e400 0x55e102250820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 1 --2- 172.21.15.32:0/3570924265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1021cdc10 0x55e1021daab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 1 -- 172.21.15.32:0/3570924265 >> 172.21.15.32:0/3570924265 conn(0x55e1021c8340 msgr2=0x55e1021d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faca3fff640 0 client.15867 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 1 -- 172.21.15.32:0/3570924265 shutdown_connections 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 1 -- 172.21.15.32:0/3570924265 wait complete. 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccde2f740 20 client.15867 trim_cache size 0 max 16384 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2024-04-16T05:35:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2024-04-16T05:35:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 -- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7facb40a3e60 msgr2=0x7facb40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7facb40a3e60 0x7facb40a6320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7facbc005fd0 tx=0x7facbc004a40 comp rx=0 tx=0).stop 2024-04-16T05:35:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 -- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4106220 msgr2=0x7facc4199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4106220 0x7facc4199bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7facb0000c00 tx=0x7facb002cc70 comp rx=0 tx=0).stop 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 -- 172.21.15.32:0/3335589613 shutdown_connections 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facc419a160 0x7facc41a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7facb40a3e60 0x7facb40a6320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc4106220 0x7facc4199bd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 --2- 172.21.15.32:0/3335589613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc41011a0 0x7facc4199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.173+0000 7faccb167640 1 -- 172.21.15.32:0/3335589613 >> 172.21.15.32:0/3335589613 conn(0x7facc4078080 msgr2=0x7facc40fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.174+0000 7faccb167640 1 -- 172.21.15.32:0/3335589613 shutdown_connections 2024-04-16T05:35:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:35:56.174+0000 7faccb167640 1 -- 172.21.15.32:0/3335589613 wait complete. 2024-04-16T05:35:57.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:57 smithi032 ceph-mon[32351]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-16T05:35:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:57 smithi134 ceph-mon[36153]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-16T05:35:58.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:57 smithi082 ceph-mon[36195]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-16T05:35:59.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:35:59 smithi032 ceph-mon[32351]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-16T05:36:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:35:59 smithi134 ceph-mon[36153]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-16T05:36:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:35:59 smithi082 ceph-mon[36195]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-16T05:36:01.233 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:36:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:36:00.762+0000 7f9649226700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:01.233 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:36:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:36:00.762+0000 7f9649226700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:01.233 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:36:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:36:00.762+0000 7f9649226700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:01.233 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:36:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:36:00.762+0000 7f9649226700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:01.233 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:36:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:36:00.762+0000 7f9649226700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T05:36:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:01 smithi032 ceph-mon[32351]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:36:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:36:01.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:01.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:01.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:01.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:01 smithi134 ceph-mon[36153]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:36:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:36:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:01 smithi082 ceph-mon[36195]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T05:36:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:36:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:02 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:36:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:02 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:36:02.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:02 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:02 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:36:03.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:02 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:36:03.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:02 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:02 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:36:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:02 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:36:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:02 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:36:03.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:03 smithi032 ceph-mon[32351]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:03.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:03 smithi032 ceph-mon[32351]: mgrmap e25: x(active, since 66s), standbys: z, y 2024-04-16T05:36:04.035 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:03 smithi134 ceph-mon[36153]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:04.035 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:03 smithi134 ceph-mon[36153]: mgrmap e25: x(active, since 66s), standbys: z, y 2024-04-16T05:36:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:03 smithi082 ceph-mon[36195]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:03 smithi082 ceph-mon[36195]: mgrmap e25: x(active, since 66s), standbys: z, y 2024-04-16T05:36:04.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:36:04.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 -- 172.21.15.32:0/2484998104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103ba0 msgr2=0x7ff40010c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 --2- 172.21.15.32:0/2484998104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103ba0 0x7ff40010c080 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009900 tx=0x7ff3f402bd60 comp rx=0 tx=0).stop 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 -- 172.21.15.32:0/2484998104 shutdown_connections 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 --2- 172.21.15.32:0/2484998104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff40010c5c0 0x7ff40010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 --2- 172.21.15.32:0/2484998104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103ba0 0x7ff40010c080 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 --2- 172.21.15.32:0/2484998104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff400103280 0x7ff400103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 -- 172.21.15.32:0/2484998104 >> 172.21.15.32:0/2484998104 conn(0x7ff400078000 msgr2=0x7ff400078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 -- 172.21.15.32:0/2484998104 shutdown_connections 2024-04-16T05:36:04.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.664+0000 7ff405014640 1 -- 172.21.15.32:0/2484998104 wait complete. 2024-04-16T05:36:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff405014640 1 Processor -- start 2024-04-16T05:36:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff405014640 1 -- start start 2024-04-16T05:36:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff405014640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103280 0x7ff4001a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff405014640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff400103ba0 0x7ff4001a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff405014640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff40010c5c0 0x7ff40019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff405014640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001123d0 con 0x7ff400103280 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff405014640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff400112250 con 0x7ff40010c5c0 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff405014640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff400112550 con 0x7ff400103ba0 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff3fed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103280 0x7ff4001a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff3fed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103280 0x7ff4001a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43396/0 (socket says 172.21.15.32:43396) 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.665+0000 7ff3fed9d640 1 -- 172.21.15.32:0/3255061318 learned_addr learned my addr 172.21.15.32:0/3255061318 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3ff59e640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff40010c5c0 0x7ff40019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff400103ba0 0x7ff4001a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3fed9d640 1 -- 172.21.15.32:0/3255061318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff400103ba0 msgr2=0x7ff4001a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff400103ba0 0x7ff4001a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3fed9d640 1 -- 172.21.15.32:0/3255061318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff40010c5c0 msgr2=0x7ff40019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff40010c5c0 0x7ff40019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3fed9d640 1 -- 172.21.15.32:0/3255061318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40019b070 con 0x7ff400103280 2024-04-16T05:36:04.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff400103ba0 0x7ff4001a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:04.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3ff59e640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff40010c5c0 0x7ff40019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:04.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103280 0x7ff4001a0740 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000d950 tx=0x7ff3f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:04.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3e7fff640 1 -- 172.21.15.32:0/3255061318 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0017070 con 0x7ff400103280 2024-04-16T05:36:04.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff405014640 1 -- 172.21.15.32:0/3255061318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40019b360 con 0x7ff400103280 2024-04-16T05:36:04.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff405014640 1 -- 172.21.15.32:0/3255061318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001a7600 con 0x7ff400103280 2024-04-16T05:36:04.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3e7fff640 1 -- 172.21.15.32:0/3255061318 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f0009c30 con 0x7ff400103280 2024-04-16T05:36:04.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.666+0000 7ff3e7fff640 1 -- 172.21.15.32:0/3255061318 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f000f6b0 con 0x7ff400103280 2024-04-16T05:36:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.669+0000 7ff3e7fff640 1 -- 172.21.15.32:0/3255061318 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7ff3f000f850 con 0x7ff400103280 2024-04-16T05:36:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.669+0000 7ff3e7fff640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff3e80a3f40 0x7ff3e80a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.669+0000 7ff3e7fff640 1 -- 172.21.15.32:0/3255061318 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff3f00d0d80 con 0x7ff400103280 2024-04-16T05:36:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.669+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff3e80a3f40 0x7ff3e80a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff407c01740 1 Processor -- start 2024-04-16T05:36:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff407c01740 1 -- start start 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff407c01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555ce50 0x5572f555d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff407c01740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555d770 0x5572f5565e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff407c01740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5568740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff3e80a3f40 0x7ff3e80a6400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4006010 tx=0x7ff3f4005a20 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff407c01740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5572f54c2180 con 0x5572f5566350 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff407c01740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5572f545f360 con 0x5572f555d770 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff407c01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5572f54be140 con 0x5572f555ce50 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3ff59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5568740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3ff59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5568740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43408/0 (socket says 172.21.15.32:43408) 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3ff59e640 1 -- 172.21.15.32:0/2139619780 learned_addr learned my addr 172.21.15.32:0/2139619780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555ce50 0x5572f555d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555d770 0x5572f5565e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3ff59e640 1 -- 172.21.15.32:0/2139619780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555ce50 msgr2=0x5572f555d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3ff59e640 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555ce50 0x5572f555d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3ff59e640 1 -- 172.21.15.32:0/2139619780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555d770 msgr2=0x5572f5565e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3ff59e640 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555d770 0x5572f5565e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.670+0000 7ff3ff59e640 1 -- 172.21.15.32:0/2139619780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f529e7d0 con 0x5572f5566350 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.671+0000 7ff3ff59e640 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5568740 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009f30 tx=0x7ff3f8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbedfe511bf34e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.671+0000 7ff3e57fa640 1 -- 172.21.15.32:0/2139619780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8016070 con 0x5572f5566350 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.671+0000 7ff3e57fa640 1 -- 172.21.15.32:0/2139619780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f801bb40 con 0x5572f5566350 2024-04-16T05:36:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.671+0000 7ff3e57fa640 1 -- 172.21.15.32:0/2139619780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8012cb0 con 0x5572f5566350 2024-04-16T05:36:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 -- 172.21.15.32:0/2139619780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 msgr2=0x5572f5568740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5568740 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009f30 tx=0x7ff3f8010800 comp rx=0 tx=0).stop 2024-04-16T05:36:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 -- 172.21.15.32:0/2139619780 shutdown_connections 2024-04-16T05:36:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5568740 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555d770 0x5572f5565e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 --2- 172.21.15.32:0/2139619780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555ce50 0x5572f555d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 -- 172.21.15.32:0/2139619780 >> 172.21.15.32:0/2139619780 conn(0x5572f5557580 msgr2=0x5572f5557960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 -- 172.21.15.32:0/2139619780 shutdown_connections 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 -- 172.21.15.32:0/2139619780 wait complete. 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 -- start start 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.672+0000 7ff407c01740 1 Processor -- start 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff407c01740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5563390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff407c01740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f55638d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff407c01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f5566350 0x5572f5563e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff407c01740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5572f5569140 con 0x5572f555d770 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff407c01740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5572f547b2c0 con 0x5572f555ce50 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff407c01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5572f55692c0 con 0x5572f5566350 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fe59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f55638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff407c01740 10 client.0 _ll_register_callbacks cb 0x5572f5462090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fe59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f55638d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43420/0 (socket says 172.21.15.32:43420) 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fe59c640 1 -- 172.21.15.32:0/2094571595 learned_addr learned my addr 172.21.15.32:0/2094571595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5563390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fed9d640 1 -- 172.21.15.32:0/2094571595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f5566350 msgr2=0x5572f5563e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f5566350 0x5572f5563e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fed9d640 1 -- 172.21.15.32:0/2094571595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 msgr2=0x5572f55638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f55638d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fed9d640 1 -- 172.21.15.32:0/2094571595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f51d43f0 con 0x5572f555ce50 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f55638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5563390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00cdfb0 tx=0x7ff3f009cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18d5f090a82cd19 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.673+0000 7ff3e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/2094571595 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0020070 con 0x5572f555ce50 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 1 -- 172.21.15.32:0/2094571595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f801bb40 con 0x5572f555ce50 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/2094571595 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff3f0098da0 con 0x5572f555ce50 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/2094571595 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff3f00a0ae0 con 0x5572f555ce50 2024-04-16T05:36:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 10 client.25409 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 1 -- 172.21.15.32:0/2094571595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5572f51940f0 con 0x5572f555ce50 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/2094571595 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f801bb40 con 0x5572f555ce50 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/2094571595 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff3f00ce400 con 0x5572f555ce50 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 10 client.25409 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 10 client.25409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 20 client.25409 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5572f55765f0 0x5572f5578ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 4 client.25409 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.674+0000 7ff407c01740 1 -- 172.21.15.32:0/2094571595 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572f54cfba0 con 0x5572f55765f0 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.675+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5572f55765f0 0x5572f5578ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.675+0000 7ff3e2ff5640 10 client.25409 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.675+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5572f55765f0 0x5572f5578ab0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56fca898ef50670d server_cookie=59160b5f08324870 in_seq=0 out_seq=0 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/2094571595 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572f54cfba0 con 0x5572f55765f0 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff3e2ff5640 10 client.25409 handle_command_reply: tid=0 2024-04-16T05:36:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 client.25409 shutdown 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 -- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 msgr2=0x5572f5563390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5563390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00cdfb0 tx=0x7ff3f009cef0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 -- 172.21.15.32:0/2094571595 shutdown_connections 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5572f55765f0 0x5572f5578ab0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff3e2ff5640 0 client.25409 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f5566350 0x5572f5563e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f55638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 --2- 172.21.15.32:0/2094571595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5563390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 -- 172.21.15.32:0/2094571595 >> 172.21.15.32:0/2094571595 conn(0x5572f5557580 msgr2=0x5572f55592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 -- 172.21.15.32:0/2094571595 shutdown_connections 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 1 -- 172.21.15.32:0/2094571595 wait complete. 2024-04-16T05:36:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.676+0000 7ff407c01740 20 client.25409 trim_cache size 0 max 16384 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff407c01740 1 Processor -- start 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff407c01740 1 -- start start 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff407c01740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5572060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff407c01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555d770 0x5572f5562cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff407c01740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5563a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff407c01740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec012630 con 0x5572f5566350 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff407c01740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec00ff00 con 0x5572f555ce50 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff407c01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec00dd50 con 0x5572f555d770 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3ff59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5563a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3ff59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5563a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43436/0 (socket says 172.21.15.32:43436) 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3ff59e640 1 -- 172.21.15.32:0/2935411971 learned_addr learned my addr 172.21.15.32:0/2935411971 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5572060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555d770 0x5572f5562cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3ff59e640 1 -- 172.21.15.32:0/2935411971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555d770 msgr2=0x5572f5562cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3ff59e640 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555d770 0x5572f5562cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3ff59e640 1 -- 172.21.15.32:0/2935411971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 msgr2=0x5572f5572060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3ff59e640 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5572060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3ff59e640 1 -- 172.21.15.32:0/2935411971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f5568740 con 0x5572f5566350 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.696+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5572060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.697+0000 7ff3ff59e640 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5563a60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009740 tx=0x7ff3f8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bbbc4b459f6638 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.697+0000 7ff3e4ff9640 1 -- 172.21.15.32:0/2935411971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8016070 con 0x5572f5566350 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.697+0000 7ff3e4ff9640 1 -- 172.21.15.32:0/2935411971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f8004e10 con 0x5572f5566350 2024-04-16T05:36:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.697+0000 7ff3e4ff9640 1 -- 172.21.15.32:0/2935411971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f801b860 con 0x5572f5566350 2024-04-16T05:36:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 -- 172.21.15.32:0/2935411971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 msgr2=0x5572f5563a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5563a60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009740 tx=0x7ff3f8009800 comp rx=0 tx=0).stop 2024-04-16T05:36:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 -- 172.21.15.32:0/2935411971 shutdown_connections 2024-04-16T05:36:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f5566350 0x5572f5563a60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f555d770 0x5572f5562cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 --2- 172.21.15.32:0/2935411971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f5572060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 -- 172.21.15.32:0/2935411971 >> 172.21.15.32:0/2935411971 conn(0x5572f5557580 msgr2=0x5572f5563350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 -- 172.21.15.32:0/2935411971 shutdown_connections 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 -- 172.21.15.32:0/2935411971 wait complete. 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 -- start start 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.698+0000 7ff407c01740 1 Processor -- start 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff407c01740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f556b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff407c01740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f556bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff407c01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f5566350 0x5572f55dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f556b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f556b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.32:54440/0 (socket says 172.21.15.32:54440) 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff407c01740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec00e3c0 con 0x5572f555d770 2024-04-16T05:36:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff407c01740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec00f830 con 0x5572f555ce50 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff407c01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec00ea90 con 0x5572f5566350 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fe59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f556bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fe59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f556bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43446/0 (socket says 172.21.15.32:43446) 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fe59c640 1 -- 172.21.15.32:0/3937143432 learned_addr learned my addr 172.21.15.32:0/3937143432 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff407c01740 10 client.0 _ll_register_callbacks cb 0x7ff3f009c500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fe59c640 1 -- 172.21.15.32:0/3937143432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f5566350 msgr2=0x5572f55dabf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f5566350 0x5572f55dabf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fe59c640 1 -- 172.21.15.32:0/3937143432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 msgr2=0x5572f556b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f556b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.699+0000 7ff3fe59c640 1 -- 172.21.15.32:0/3937143432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f55db1b0 con 0x5572f555d770 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f556b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff3fe59c640 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f556bcb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff3f402c370 tx=0x7ff3f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a5a479990f254f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff3e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/3937143432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f405b060 con 0x5572f555d770 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff407c01740 1 -- 172.21.15.32:0/3937143432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f8004e10 con 0x5572f555d770 2024-04-16T05:36:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/3937143432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff3f404c450 con 0x5572f555d770 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/3937143432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff3f4062050 con 0x5572f555d770 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff407c01740 10 client.15891 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff407c01740 1 -- 172.21.15.32:0/3937143432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5572f55e4d10 con 0x5572f555d770 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.700+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/3937143432 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f8004e10 con 0x5572f555d770 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.701+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/3937143432 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff3f4049040 con 0x5572f555d770 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.701+0000 7ff407c01740 10 client.15891 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.701+0000 7ff407c01740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.701+0000 7ff407c01740 20 client.15891 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.701+0000 7ff407c01740 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5572f55765f0 0x5572f55e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.701+0000 7ff407c01740 4 client.15891 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.701+0000 7ff407c01740 1 -- 172.21.15.32:0/3937143432 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572f54c9a10 con 0x5572f55765f0 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5572f55765f0 0x5572f55e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff3fed9d640 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5572f55765f0 0x5572f55e7480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1e9d408153e0cd server_cookie=605b6e7323f47153 in_seq=0 out_seq=0 2024-04-16T05:36:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff3e2ff5640 10 client.15891 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff3e2ff5640 1 -- 172.21.15.32:0/3937143432 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5572f54c9a10 con 0x5572f55765f0 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff3e2ff5640 10 client.15891 handle_command_reply: tid=0 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff407c01740 1 client.15891 shutdown 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff407c01740 1 -- 172.21.15.32:0/3937143432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 msgr2=0x5572f556bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff407c01740 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f556bcb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff3f402c370 tx=0x7ff3f4009260 comp rx=0 tx=0).stop 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff407c01740 1 -- 172.21.15.32:0/3937143432 shutdown_connections 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.702+0000 7ff407c01740 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5572f55765f0 0x5572f55e7480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff407c01740 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572f5566350 0x5572f55dabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff407c01740 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572f555d770 0x5572f556bcb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff407c01740 1 --2- 172.21.15.32:0/3937143432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572f555ce50 0x5572f556b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff407c01740 1 -- 172.21.15.32:0/3937143432 >> 172.21.15.32:0/3937143432 conn(0x5572f5557580 msgr2=0x5572f5563be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff3e2ff5640 0 client.15891 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff407c01740 1 -- 172.21.15.32:0/3937143432 shutdown_connections 2024-04-16T05:36:04.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff407c01740 1 -- 172.21.15.32:0/3937143432 wait complete. 2024-04-16T05:36:04.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff407c01740 20 client.15891 trim_cache size 0 max 16384 2024-04-16T05:36:04.707 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:36:04.707 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:36:04.707 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:36:04.707 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff405014640 1 -- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff3e80a3f40 msgr2=0x7ff3e80a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff405014640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff3e80a3f40 0x7ff3e80a6400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4006010 tx=0x7ff3f4005a20 comp rx=0 tx=0).stop 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff405014640 1 -- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103280 msgr2=0x7ff4001a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff405014640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103280 0x7ff4001a0740 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000d950 tx=0x7ff3f000de10 comp rx=0 tx=0).stop 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff405014640 1 -- 172.21.15.32:0/3255061318 shutdown_connections 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.703+0000 7ff405014640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff40010c5c0 0x7ff40019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.704+0000 7ff405014640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff400103ba0 0x7ff4001a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.704+0000 7ff405014640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff3e80a3f40 0x7ff3e80a6400 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.704+0000 7ff405014640 1 --2- 172.21.15.32:0/3255061318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff400103280 0x7ff4001a0740 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.704+0000 7ff405014640 1 -- 172.21.15.32:0/3255061318 >> 172.21.15.32:0/3255061318 conn(0x7ff400078000 msgr2=0x7ff400108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:04.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.704+0000 7ff405014640 1 -- 172.21.15.32:0/3255061318 shutdown_connections 2024-04-16T05:36:04.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.704+0000 7ff405014640 1 -- 172.21.15.32:0/3255061318 wait complete. 2024-04-16T05:36:04.717 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:36:04.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 -- 172.21.15.32:0/563105695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0102c00 msgr2=0x7f76f0106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 --2- 172.21.15.32:0/563105695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0102c00 0x7f76f0106c40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009900 tx=0x7f76e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 -- 172.21.15.32:0/563105695 shutdown_connections 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 --2- 172.21.15.32:0/563105695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76f0107320 0x7f76f010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 --2- 172.21.15.32:0/563105695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0102c00 0x7f76f0106c40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 --2- 172.21.15.32:0/563105695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f0102250 0x7f76f0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 -- 172.21.15.32:0/563105695 >> 172.21.15.32:0/563105695 conn(0x7f76f00fda90 msgr2=0x7f76f00ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 -- 172.21.15.32:0/563105695 shutdown_connections 2024-04-16T05:36:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.843+0000 7f76f5663640 1 -- 172.21.15.32:0/563105695 wait complete. 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76f5663640 1 Processor -- start 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76f5663640 1 -- start start 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76f5663640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76f0102250 0x7f76f0074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76f5663640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0107320 0x7f76f0074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76f5663640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f0074d80 0x7f76f01a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76f5663640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f76f0112390 con 0x7f76f0107320 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76f5663640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f76f0112210 con 0x7f76f0074d80 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76f5663640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f76f0112510 con 0x7f76f0102250 2024-04-16T05:36:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76ee7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0107320 0x7f76f0074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76ee7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0107320 0x7f76f0074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43458/0 (socket says 172.21.15.32:43458) 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76ee7fc640 1 -- 172.21.15.32:0/2509414488 learned_addr learned my addr 172.21.15.32:0/2509414488 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.844+0000 7f76eeffd640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76f0102250 0x7f76f0074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76ef7fe640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f0074d80 0x7f76f01a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76ee7fc640 1 -- 172.21.15.32:0/2509414488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76f0102250 msgr2=0x7f76f0074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76ee7fc640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76f0102250 0x7f76f0074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76ee7fc640 1 -- 172.21.15.32:0/2509414488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f0074d80 msgr2=0x7f76f01a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76ee7fc640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f0074d80 0x7f76f01a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76ee7fc640 1 -- 172.21.15.32:0/2509414488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f01a71e0 con 0x7f76f0107320 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76ef7fe640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f0074d80 0x7f76f01a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76ee7fc640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0107320 0x7f76f0074840 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f76e4036040 tx=0x7f76e402ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76d7fff640 1 -- 172.21.15.32:0/2509414488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4041070 con 0x7f76f0107320 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f01a7470 con 0x7f76f0107320 2024-04-16T05:36:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f01a79b0 con 0x7f76f0107320 2024-04-16T05:36:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.845+0000 7f76d7fff640 1 -- 172.21.15.32:0/2509414488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e40054e0 con 0x7f76f0107320 2024-04-16T05:36:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.846+0000 7f76d7fff640 1 -- 172.21.15.32:0/2509414488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4034440 con 0x7f76f0107320 2024-04-16T05:36:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.846+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f0103720 con 0x7f76f0107320 2024-04-16T05:36:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.848+0000 7f76d7fff640 1 -- 172.21.15.32:0/2509414488 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f76e4044070 con 0x7f76f0107320 2024-04-16T05:36:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.848+0000 7f76d7fff640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f76d80a3d20 0x7f76d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.848+0000 7f76eeffd640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f76d80a3d20 0x7f76d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.849+0000 7f76d7fff640 1 -- 172.21.15.32:0/2509414488 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f76e40f2aa0 con 0x7f76f0107320 2024-04-16T05:36:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.849+0000 7f76eeffd640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f76d80a3d20 0x7f76d80a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76e000bbe0 tx=0x7f76e000b390 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:04.856+0000 7f76d7fff640 1 -- 172.21.15.32:0/2509414488 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76e40bed50 con 0x7f76f0107320 2024-04-16T05:36:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.175+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76f0075830 con 0x7f76f0107320 2024-04-16T05:36:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.176+0000 7f76d7fff640 1 -- 172.21.15.32:0/2509414488 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f76e4004c50 con 0x7f76f0107320 2024-04-16T05:36:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:36:05.176 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:36:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:36:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.178+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f76d80a3d20 msgr2=0x7f76d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.178+0000 7f76f5663640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f76d80a3d20 0x7f76d80a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76e000bbe0 tx=0x7f76e000b390 comp rx=0 tx=0).stop 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.178+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0107320 msgr2=0x7f76f0074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.178+0000 7f76f5663640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0107320 0x7f76f0074840 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f76e4036040 tx=0x7f76e402ce20 comp rx=0 tx=0).stop 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.179+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 shutdown_connections 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.179+0000 7f76f5663640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f0074d80 0x7f76f01a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.179+0000 7f76f5663640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f76d80a3d20 0x7f76d80a61e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.179+0000 7f76f5663640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76f0107320 0x7f76f0074840 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.179+0000 7f76f5663640 1 --2- 172.21.15.32:0/2509414488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76f0102250 0x7f76f0074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.179+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 >> 172.21.15.32:0/2509414488 conn(0x7f76f00fda90 msgr2=0x7f76f00ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.179+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 shutdown_connections 2024-04-16T05:36:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.179+0000 7f76f5663640 1 -- 172.21.15.32:0/2509414488 wait complete. 2024-04-16T05:36:05.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:36:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.300+0000 7f84c78ba640 1 -- 172.21.15.32:0/3138325260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c0102220 msgr2=0x7f84c0102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.300+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3138325260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c0102220 0x7f84c0102600 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009900 tx=0x7f84bc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.300+0000 7f84c78ba640 1 -- 172.21.15.32:0/3138325260 shutdown_connections 2024-04-16T05:36:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.300+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3138325260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84c01072f0 0x7f84c010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.300+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3138325260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84c0102bd0 0x7f84c0106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.300+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3138325260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c0102220 0x7f84c0102600 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.300+0000 7f84c78ba640 1 -- 172.21.15.32:0/3138325260 >> 172.21.15.32:0/3138325260 conn(0x7f84c00fdac0 msgr2=0x7f84c00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.301+0000 7f84c78ba640 1 -- 172.21.15.32:0/3138325260 shutdown_connections 2024-04-16T05:36:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.301+0000 7f84c78ba640 1 -- 172.21.15.32:0/3138325260 wait complete. 2024-04-16T05:36:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.301+0000 7f84c78ba640 1 Processor -- start 2024-04-16T05:36:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.301+0000 7f84c78ba640 1 -- start start 2024-04-16T05:36:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.301+0000 7f84c78ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84c0102220 0x7f84c019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c78ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84c01072f0 0x7f84c019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c78ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c01987e0 0x7f84c0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c78ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0112400 con 0x7f84c01987e0 2024-04-16T05:36:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c78ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0112280 con 0x7f84c0102220 2024-04-16T05:36:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c78ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0112580 con 0x7f84c01072f0 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c01987e0 0x7f84c0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c01987e0 0x7f84c0198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43468/0 (socket says 172.21.15.32:43468) 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 -- 172.21.15.32:0/3595798757 learned_addr learned my addr 172.21.15.32:0/3595798757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c4e55640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84c01072f0 0x7f84c019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5656640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84c0102220 0x7f84c019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 -- 172.21.15.32:0/3595798757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84c01072f0 msgr2=0x7f84c019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84c01072f0 0x7f84c019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 -- 172.21.15.32:0/3595798757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84c0102220 msgr2=0x7f84c019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84c0102220 0x7f84c019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 -- 172.21.15.32:0/3595798757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c0199370 con 0x7f84c01987e0 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5656640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84c0102220 0x7f84c019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.302+0000 7f84c5e57640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c01987e0 0x7f84c0198c90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f84b800eaa0 tx=0x7f84b800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.303+0000 7f84b67fc640 1 -- 172.21.15.32:0/3595798757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004d20 con 0x7f84c01987e0 2024-04-16T05:36:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.303+0000 7f84b67fc640 1 -- 172.21.15.32:0/3595798757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b8005020 con 0x7f84c01987e0 2024-04-16T05:36:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.303+0000 7f84b67fc640 1 -- 172.21.15.32:0/3595798757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80116d0 con 0x7f84c01987e0 2024-04-16T05:36:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.303+0000 7f84c78ba640 1 -- 172.21.15.32:0/3595798757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c0199660 con 0x7f84c01987e0 2024-04-16T05:36:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.303+0000 7f84c78ba640 1 -- 172.21.15.32:0/3595798757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a57f0 con 0x7f84c01987e0 2024-04-16T05:36:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.306+0000 7f84b67fc640 1 -- 172.21.15.32:0/3595798757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f84b80051d0 con 0x7f84c01987e0 2024-04-16T05:36:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.306+0000 7f84ca580740 1 Processor -- start 2024-04-16T05:36:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.306+0000 7f84ca580740 1 -- start start 2024-04-16T05:36:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84b67fc640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84ac0a3f90 0x7f84ac0a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84ca580740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 0x55e68e21e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84ca580740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e226e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84ca580740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e229750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84ca580740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e68e1830e0 con 0x55e68e21e780 2024-04-16T05:36:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84ca580740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e68e120520 con 0x55e68e227360 2024-04-16T05:36:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84ca580740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e68e17f0a0 con 0x55e68e21de60 2024-04-16T05:36:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84b67fc640 1 -- 172.21.15.32:0/3595798757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f84b80d0f60 con 0x7f84c01987e0 2024-04-16T05:36:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e226e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84ac0a3f90 0x7f84ac0a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e226e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43474/0 (socket says 172.21.15.32:43474) 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 -- 172.21.15.32:0/2205992468 learned_addr learned my addr 172.21.15.32:0/2205992468 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c5656640 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 0x55e68e21e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 -- 172.21.15.32:0/2205992468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 msgr2=0x55e68e21e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 0x55e68e21e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 -- 172.21.15.32:0/2205992468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 msgr2=0x55e68e229750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c5e57640 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e229750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e229750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 -- 172.21.15.32:0/2205992468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68e1488d0 con 0x55e68e21e780 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c5e57640 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e229750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.307+0000 7f84c4e55640 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e226e20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f84b0015870 tx=0x7f84b0015d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e566da86f191fd78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.308+0000 7f84c4e55640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84ac0a3f90 0x7f84ac0a6450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84b00177b0 tx=0x7f84b001b000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.308+0000 7f84ab7fe640 1 -- 172.21.15.32:0/2205992468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0025070 con 0x55e68e21e780 2024-04-16T05:36:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.308+0000 7f84ab7fe640 1 -- 172.21.15.32:0/2205992468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b001c070 con 0x55e68e21e780 2024-04-16T05:36:05.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.308+0000 7f84ab7fe640 1 -- 172.21.15.32:0/2205992468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0006290 con 0x55e68e21e780 2024-04-16T05:36:05.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.308+0000 7f84ca580740 1 -- 172.21.15.32:0/2205992468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 msgr2=0x55e68e226e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.308+0000 7f84ca580740 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e226e20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f84b0015870 tx=0x7f84b0015d30 comp rx=0 tx=0).stop 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 -- 172.21.15.32:0/2205992468 shutdown_connections 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e229750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e226e20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 --2- 172.21.15.32:0/2205992468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 0x55e68e21e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 -- 172.21.15.32:0/2205992468 >> 172.21.15.32:0/2205992468 conn(0x55e68e218590 msgr2=0x55e68e218970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 -- 172.21.15.32:0/2205992468 shutdown_connections 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 -- 172.21.15.32:0/2205992468 wait complete. 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 -- start start 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.309+0000 7f84ca580740 1 Processor -- start 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84ca580740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 0x55e68e224460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84ca580740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e2249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84ca580740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e224ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84ca580740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e68e22e500 con 0x55e68e21e780 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c4e55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e2249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c4e55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e2249a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43488/0 (socket says 172.21.15.32:43488) 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c4e55640 1 -- 172.21.15.32:0/448559284 learned_addr learned my addr 172.21.15.32:0/448559284 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e68e13c4d0 con 0x55e68e227360 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e68e22e680 con 0x55e68e21de60 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84ca580740 10 client.0 _ll_register_callbacks cb 0x55e68e1212c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c5656640 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 0x55e68e224460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c5e57640 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e224ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c4e55640 1 -- 172.21.15.32:0/448559284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 msgr2=0x55e68e224460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c4e55640 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 0x55e68e224460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c4e55640 1 -- 172.21.15.32:0/448559284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 msgr2=0x55e68e224ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c4e55640 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e224ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.310+0000 7f84c4e55640 1 -- 172.21.15.32:0/448559284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68de953f0 con 0x55e68e21e780 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84c4e55640 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e2249a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009150 tx=0x7f84b0018c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfc2b7975268db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84a8ff9640 1 -- 172.21.15.32:0/448559284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0025040 con 0x55e68e21e780 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84a8ff9640 1 -- 172.21.15.32:0/448559284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f84b0005bd0 con 0x55e68e21e780 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b001c070 con 0x55e68e21e780 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84a8ff9640 1 -- 172.21.15.32:0/448559284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84b0036050 con 0x55e68e21e780 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84ca580740 10 client.15909 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e68df5f7d0 con 0x55e68e21e780 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.311+0000 7f84a8ff9640 1 -- 172.21.15.32:0/448559284 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b001c070 con 0x55e68e21e780 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84a8ff9640 1 -- 172.21.15.32:0/448559284 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f84b0020460 con 0x55e68e21e780 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84ca580740 10 client.15909 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84ca580740 10 client.15909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84ca580740 20 client.15909 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84ca580740 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55e68e23b6f0 0x55e68e23dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84ca580740 4 client.15909 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e68e190b00 con 0x55e68e23b6f0 2024-04-16T05:36:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84c5656640 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55e68e23b6f0 0x55e68e23dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.312+0000 7f84a8ff9640 10 client.15909 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:36:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.318+0000 7f84c5656640 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55e68e23b6f0 0x55e68e23dae0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6af03867d6520e3 server_cookie=bdb840e2a079ba6a in_seq=0 out_seq=0 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84a8ff9640 1 -- 172.21.15.32:0/448559284 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e68e190b00 con 0x55e68e23b6f0 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84a8ff9640 10 client.15909 handle_command_reply: tid=0 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 client.15909 shutdown 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 msgr2=0x55e68e2249a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e2249a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009150 tx=0x7f84b0018c70 comp rx=0 tx=0).stop 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 shutdown_connections 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55e68e23b6f0 0x55e68e23dae0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84a8ff9640 0 client.15909 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e227360 0x55e68e224ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21e780 0x55e68e2249a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 --2- 172.21.15.32:0/448559284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21de60 0x55e68e224460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 >> 172.21.15.32:0/448559284 conn(0x55e68e218590 msgr2=0x55e68e21a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 shutdown_connections 2024-04-16T05:36:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 1 -- 172.21.15.32:0/448559284 wait complete. 2024-04-16T05:36:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.319+0000 7f84ca580740 20 client.15909 trim_cache size 0 max 16384 2024-04-16T05:36:05.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.331+0000 7f84ca580740 1 Processor -- start 2024-04-16T05:36:05.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84ca580740 1 -- start start 2024-04-16T05:36:05.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84ca580740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21de60 0x55e68e226b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84ca580740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21e780 0x55e68e2373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84ca580740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e227360 0x55e68e2175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84ca580740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01afa20 con 0x55e68e21de60 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84ca580740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01ad2f0 con 0x55e68e21e780 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84ca580740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01ab140 con 0x55e68e227360 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5656640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21de60 0x55e68e226b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5656640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21de60 0x55e68e226b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43498/0 (socket says 172.21.15.32:43498) 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5656640 1 -- 172.21.15.32:0/3338882006 learned_addr learned my addr 172.21.15.32:0/3338882006 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c4e55640 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21e780 0x55e68e2373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5e57640 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e227360 0x55e68e2175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5656640 1 -- 172.21.15.32:0/3338882006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e227360 msgr2=0x55e68e2175b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5656640 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e227360 0x55e68e2175b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5656640 1 -- 172.21.15.32:0/3338882006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21e780 msgr2=0x55e68e2373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5656640 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21e780 0x55e68e2373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5656640 1 -- 172.21.15.32:0/3338882006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68e21e5b0 con 0x55e68e21de60 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.332+0000 7f84c5e57640 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e227360 0x55e68e2175b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84c5656640 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21de60 0x55e68e226b90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f84bc005940 tx=0x7f84bc009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5c5bdd75b54545 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84aaffd640 1 -- 172.21.15.32:0/3338882006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc045bc0 con 0x55e68e21de60 2024-04-16T05:36:05.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84aaffd640 1 -- 172.21.15.32:0/3338882006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84bc045d60 con 0x55e68e21de60 2024-04-16T05:36:05.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84aaffd640 1 -- 172.21.15.32:0/3338882006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc005080 con 0x55e68e21de60 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84ca580740 1 -- 172.21.15.32:0/3338882006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21de60 msgr2=0x55e68e226b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84ca580740 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21de60 0x55e68e226b90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f84bc005940 tx=0x7f84bc009640 comp rx=0 tx=0).stop 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84ca580740 1 -- 172.21.15.32:0/3338882006 shutdown_connections 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84ca580740 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e227360 0x55e68e2175b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84ca580740 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21e780 0x55e68e2373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84ca580740 1 --2- 172.21.15.32:0/3338882006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e21de60 0x55e68e226b90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.333+0000 7f84ca580740 1 -- 172.21.15.32:0/3338882006 >> 172.21.15.32:0/3338882006 conn(0x55e68e218590 msgr2=0x55e68e216ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 -- 172.21.15.32:0/3338882006 shutdown_connections 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 -- 172.21.15.32:0/3338882006 wait complete. 2024-04-16T05:36:05.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 -- start start 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 Processor -- start 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21de60 0x55e68e2338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21e780 0x55e68e233e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e227360 0x55e68e220b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01ab7b0 con 0x55e68e227360 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01acc20 con 0x55e68e21de60 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.334+0000 7f84ca580740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01abe80 con 0x55e68e21e780 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84ca580740 10 client.0 _ll_register_callbacks cb 0x7f84b0023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e227360 0x55e68e220b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e227360 0x55e68e220b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43502/0 (socket says 172.21.15.32:43502) 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 -- 172.21.15.32:0/1343447757 learned_addr learned my addr 172.21.15.32:0/1343447757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c4e55640 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21e780 0x55e68e233e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5656640 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21de60 0x55e68e2338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 -- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21e780 msgr2=0x55e68e233e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21e780 0x55e68e233e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 -- 172.21.15.32:0/1343447757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21de60 msgr2=0x55e68e2338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21de60 0x55e68e2338d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 -- 172.21.15.32:0/1343447757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68e221400 con 0x55e68e227360 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c4e55640 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21e780 0x55e68e233e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84c5e57640 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e227360 0x55e68e220b80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009270 tx=0x7f84b809c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adebb157e1ed466 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84a8ff9640 1 -- 172.21.15.32:0/1343447757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80b5070 con 0x55e68e227360 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84a8ff9640 1 -- 172.21.15.32:0/1343447757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f84b80ce6d0 con 0x55e68e227360 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.335+0000 7f84ca580740 1 -- 172.21.15.32:0/1343447757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84bc045d60 con 0x55e68e227360 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.336+0000 7f84a8ff9640 1 -- 172.21.15.32:0/1343447757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84b801f040 con 0x55e68e227360 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.336+0000 7f84ca580740 10 client.15915 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.336+0000 7f84ca580740 1 -- 172.21.15.32:0/1343447757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e68e2a14b0 con 0x55e68e227360 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.336+0000 7f84a8ff9640 1 -- 172.21.15.32:0/1343447757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84bc045d60 con 0x55e68e227360 2024-04-16T05:36:05.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.337+0000 7f84a8ff9640 1 -- 172.21.15.32:0/1343447757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f84b800f040 con 0x55e68e227360 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.337+0000 7f84ca580740 10 client.15915 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.337+0000 7f84ca580740 10 client.15915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.337+0000 7f84ca580740 20 client.15915 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.337+0000 7f84ca580740 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55e68e23b6f0 0x55e68e2a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.337+0000 7f84ca580740 4 client.15915 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.337+0000 7f84ca580740 1 -- 172.21.15.32:0/1343447757 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e68e18a970 con 0x55e68e23b6f0 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.337+0000 7f84c5656640 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55e68e23b6f0 0x55e68e2a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84c5656640 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55e68e23b6f0 0x55e68e2a3c10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cc8a5643ea4468 server_cookie=b145a615e2220763 in_seq=0 out_seq=0 2024-04-16T05:36:05.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84a8ff9640 10 client.15915 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:36:05.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84a8ff9640 1 -- 172.21.15.32:0/1343447757 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e68e18a970 con 0x55e68e23b6f0 2024-04-16T05:36:05.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84a8ff9640 10 client.15915 handle_command_reply: tid=0 2024-04-16T05:36:05.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 client.15915 shutdown 2024-04-16T05:36:05.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 -- 172.21.15.32:0/1343447757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e227360 msgr2=0x55e68e220b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e227360 0x55e68e220b80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009270 tx=0x7f84b809c390 comp rx=0 tx=0).stop 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 -- 172.21.15.32:0/1343447757 shutdown_connections 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55e68e23b6f0 0x55e68e2a3c10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e68e227360 0x55e68e220b80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84a8ff9640 0 client.15915 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68e21e780 0x55e68e233e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 --2- 172.21.15.32:0/1343447757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68e21de60 0x55e68e2338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 -- 172.21.15.32:0/1343447757 >> 172.21.15.32:0/1343447757 conn(0x55e68e218590 msgr2=0x55e68e217730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 -- 172.21.15.32:0/1343447757 shutdown_connections 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 1 -- 172.21.15.32:0/1343447757 wait complete. 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.338+0000 7f84ca580740 20 client.15915 trim_cache size 0 max 16384 2024-04-16T05:36:05.342 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:36:05.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 -- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84ac0a3f90 msgr2=0x7f84ac0a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84ac0a3f90 0x7f84ac0a6450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84b00177b0 tx=0x7f84b001b000 comp rx=0 tx=0).stop 2024-04-16T05:36:05.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 -- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c01987e0 msgr2=0x7f84c0198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c01987e0 0x7f84c0198c90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f84b800eaa0 tx=0x7f84b800eea0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 -- 172.21.15.32:0/3595798757 shutdown_connections 2024-04-16T05:36:05.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84ac0a3f90 0x7f84ac0a6450 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c01987e0 0x7f84c0198c90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84c01072f0 0x7f84c019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 --2- 172.21.15.32:0/3595798757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84c0102220 0x7f84c019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 -- 172.21.15.32:0/3595798757 >> 172.21.15.32:0/3595798757 conn(0x7f84c00fdac0 msgr2=0x7f84c00feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 -- 172.21.15.32:0/3595798757 shutdown_connections 2024-04-16T05:36:05.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.339+0000 7f84c78ba640 1 -- 172.21.15.32:0/3595798757 wait complete. 2024-04-16T05:36:05.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:36:05.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/1741663305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c0103bc0 msgr2=0x7fa9c010c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/1741663305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c0103bc0 0x7fa9c010c0a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009900 tx=0x7fa9a802bd60 comp rx=0 tx=0).stop 2024-04-16T05:36:05.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/1741663305 shutdown_connections 2024-04-16T05:36:05.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/1741663305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c010c5e0 0x7fa9c010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/1741663305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c0103bc0 0x7fa9c010c0a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/1741663305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c01032a0 0x7fa9c0103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/1741663305 >> 172.21.15.32:0/1741663305 conn(0x7fa9c0077280 msgr2=0x7fa9c0077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/1741663305 shutdown_connections 2024-04-16T05:36:05.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.465+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/1741663305 wait complete. 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9c4f3d640 1 Processor -- start 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9c4f3d640 1 -- start start 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9c4f3d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c01032a0 0x7fa9c019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9c4f3d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0103bc0 0x7fa9c019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9c4f3d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c010c5e0 0x7fa9c01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9c4f3d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0112410 con 0x7fa9c01032a0 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9c4f3d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0112290 con 0x7fa9c010c5e0 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9c4f3d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0112590 con 0x7fa9c0103bc0 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c01032a0 0x7fa9c019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c01032a0 0x7fa9c019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43520/0 (socket says 172.21.15.32:43520) 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9be59c640 1 -- 172.21.15.32:0/2520583612 learned_addr learned my addr 172.21.15.32:0/2520583612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0103bc0 0x7fa9c019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.466+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c010c5e0 0x7fa9c01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9be59c640 1 -- 172.21.15.32:0/2520583612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0103bc0 msgr2=0x7fa9c019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0103bc0 0x7fa9c019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9be59c640 1 -- 172.21.15.32:0/2520583612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c010c5e0 msgr2=0x7fa9c01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c010c5e0 0x7fa9c01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9be59c640 1 -- 172.21.15.32:0/2520583612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c0198f10 con 0x7fa9c01032a0 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0103bc0 0x7fa9c019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c010c5e0 0x7fa9c01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c01032a0 0x7fa9c019e610 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000d7a0 tx=0x7fa9b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9b77fe640 1 -- 172.21.15.32:0/2520583612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0017070 con 0x7fa9c01032a0 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9b77fe640 1 -- 172.21.15.32:0/2520583612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b0005500 con 0x7fa9c01032a0 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9b77fe640 1 -- 172.21.15.32:0/2520583612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0004e00 con 0x7fa9c01032a0 2024-04-16T05:36:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/2520583612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c0199200 con 0x7fa9c01032a0 2024-04-16T05:36:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.467+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/2520583612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c0100e20 con 0x7fa9c01032a0 2024-04-16T05:36:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.469+0000 7fa9b77fe640 1 -- 172.21.15.32:0/2520583612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fa9b0009c30 con 0x7fa9c01032a0 2024-04-16T05:36:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.470+0000 7fa9b77fe640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9ac0a3e40 0x7fa9ac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.470+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9ac0a3e40 0x7fa9ac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.470+0000 7fa9b77fe640 1 -- 172.21.15.32:0/2520583612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fa9b00d0910 con 0x7fa9c01032a0 2024-04-16T05:36:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.470+0000 7fa9c7bff740 1 Processor -- start 2024-04-16T05:36:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9c7bff740 1 -- start start 2024-04-16T05:36:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9ac0a3e40 0x7fa9ac0a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8006010 tx=0x7fa9a8005a20 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f74240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 0x55c394f7ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f7d360 0x55c394f7f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c394ed90e0 con 0x55c394f7d360 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c394e76520 con 0x55c394f73e60 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c394ed50a0 con 0x55c394f74780 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9bed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f7d360 0x55c394f7f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9bed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f7d360 0x55c394f7f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43524/0 (socket says 172.21.15.32:43524) 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9bed9d640 1 -- 172.21.15.32:0/2695439077 learned_addr learned my addr 172.21.15.32:0/2695439077 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.471+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 0x55c394f7ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f74240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9bed9d640 1 -- 172.21.15.32:0/2695439077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 msgr2=0x55c394f7ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 0x55c394f7ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9bed9d640 1 -- 172.21.15.32:0/2695439077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 msgr2=0x55c394f74240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f74240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9bed9d640 1 -- 172.21.15.32:0/2695439077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c394e9e8d0 con 0x55c394f7d360 2024-04-16T05:36:05.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f7d360 0x55c394f7f750 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009fd0 tx=0x7fa9b800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8f485c73749bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9b4ff9640 1 -- 172.21.15.32:0/2695439077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8015070 con 0x55c394f7d360 2024-04-16T05:36:05.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9b4ff9640 1 -- 172.21.15.32:0/2695439077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b8011be0 con 0x55c394f7d360 2024-04-16T05:36:05.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.472+0000 7fa9b4ff9640 1 -- 172.21.15.32:0/2695439077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8010c30 con 0x55c394f7d360 2024-04-16T05:36:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2695439077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f7d360 msgr2=0x55c394f7f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f7d360 0x55c394f7f750 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009fd0 tx=0x7fa9b800f7f0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2695439077 shutdown_connections 2024-04-16T05:36:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f7d360 0x55c394f7f750 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 0x55c394f7ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2695439077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f74240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2695439077 >> 172.21.15.32:0/2695439077 conn(0x55c394f6e590 msgr2=0x55c394f6e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2695439077 shutdown_connections 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.473+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2695439077 wait complete. 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 1 -- start start 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 1 Processor -- start 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f7a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f7a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 0x55c394f7ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c394f7ffa0 con 0x55c394f74780 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c394e924d0 con 0x55c394f73e60 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c394f80120 con 0x55c394f7d360 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f7a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9c7bff740 10 client.0 _ll_register_callbacks cb 0x55c394e772c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f7a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43530/0 (socket says 172.21.15.32:43530) 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.474+0000 7fa9b7fff640 1 -- 172.21.15.32:0/2601132467 learned_addr learned my addr 172.21.15.32:0/2601132467 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f7a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 0x55c394f7ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9b7fff640 1 -- 172.21.15.32:0/2601132467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 msgr2=0x55c394f7ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 0x55c394f7ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9b7fff640 1 -- 172.21.15.32:0/2601132467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 msgr2=0x55c394f7a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f7a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9b7fff640 1 -- 172.21.15.32:0/2601132467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c394beb3f0 con 0x55c394f74780 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f7a940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8000c00 tx=0x7fa9a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b1995e66d57ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2601132467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a804e070 con 0x55c394f74780 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2601132467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fa9a8004ad0 con 0x55c394f74780 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2601132467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b8011be0 con 0x55c394f74780 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2601132467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9a805a050 con 0x55c394f74780 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9c7bff740 10 client.15927 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.475+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2601132467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c394cb57d0 con 0x55c394f74780 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2601132467 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b8011be0 con 0x55c394f74780 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2601132467 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9a8049070 con 0x55c394f74780 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9c7bff740 10 client.15927 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9c7bff740 10 client.15927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9c7bff740 20 client.15927 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c394f8d240 0x55c394f8f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9c7bff740 4 client.15927 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2601132467 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c394ee6b00 con 0x55c394f8d240 2024-04-16T05:36:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.476+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c394f8d240 0x55c394f8f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9a5ffb640 10 client.15927 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:36:05.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c394f8d240 0x55c394f8f6d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95f88fbb8b68e413 server_cookie=ef1a0609eea0a449 in_seq=0 out_seq=0 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2601132467 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c394ee6b00 con 0x55c394f8d240 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9a5ffb640 10 client.15927 handle_command_reply: tid=0 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 client.15927 shutdown 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2601132467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 msgr2=0x55c394f7a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f7a940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8000c00 tx=0x7fa9a8009260 comp rx=0 tx=0).stop 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2601132467 shutdown_connections 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c394f8d240 0x55c394f8f6d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 0x55c394f7ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9a5ffb640 0 client.15927 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f7a940 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2601132467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f7a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.477+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2601132467 >> 172.21.15.32:0/2601132467 conn(0x55c394f6e590 msgr2=0x55c394f702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.478+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2601132467 shutdown_connections 2024-04-16T05:36:05.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.478+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2601132467 wait complete. 2024-04-16T05:36:05.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.478+0000 7fa9c7bff740 20 client.15927 trim_cache size 0 max 16384 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9c7bff740 1 Processor -- start 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9c7bff740 1 -- start start 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f73e60 0x55c394f88fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 0x55c394f695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f7d360 0x55c394f94fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b3e00 con 0x55c394f73e60 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b16d0 con 0x55c394f7d360 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01af520 con 0x55c394f74780 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f73e60 0x55c394f88fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f73e60 0x55c394f88fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43544/0 (socket says 172.21.15.32:43544) 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9be59c640 1 -- 172.21.15.32:0/2092572223 learned_addr learned my addr 172.21.15.32:0/2092572223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.503+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 0x55c394f695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f7d360 0x55c394f94fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9be59c640 1 -- 172.21.15.32:0/2092572223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 msgr2=0x55c394f695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 0x55c394f695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9be59c640 1 -- 172.21.15.32:0/2092572223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f7d360 msgr2=0x55c394f94fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f7d360 0x55c394f94fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9be59c640 1 -- 172.21.15.32:0/2092572223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c394f793d0 con 0x55c394f73e60 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f7d360 0x55c394f94fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f73e60 0x55c394f88fd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0098cf0 tx=0x7fa9b00048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e95a9f12d75f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9a7fff640 1 -- 172.21.15.32:0/2092572223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0020040 con 0x55c394f73e60 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9a7fff640 1 -- 172.21.15.32:0/2092572223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b00a06d0 con 0x55c394f73e60 2024-04-16T05:36:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.504+0000 7fa9a7fff640 1 -- 172.21.15.32:0/2092572223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00ce0e0 con 0x55c394f73e60 2024-04-16T05:36:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2092572223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f73e60 msgr2=0x55c394f88fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f73e60 0x55c394f88fd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0098cf0 tx=0x7fa9b00048a0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2092572223 shutdown_connections 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f7d360 0x55c394f94fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f74780 0x55c394f695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2092572223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f73e60 0x55c394f88fd0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2092572223 >> 172.21.15.32:0/2092572223 conn(0x55c394f6e590 msgr2=0x55c394f6d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2092572223 shutdown_connections 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2092572223 wait complete. 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 -- start start 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.505+0000 7fa9c7bff740 1 Processor -- start 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f824b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9c7bff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 0x55c394f829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01afb90 con 0x55c394f74780 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b1000 con 0x55c394f73e60 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9c7bff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b0260 con 0x55c394f7d360 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f824b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f824b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43548/0 (socket says 172.21.15.32:43548) 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 -- 172.21.15.32:0/2159250444 learned_addr learned my addr 172.21.15.32:0/2159250444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9c7bff740 10 client.0 _ll_register_callbacks cb 0x7fa9a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9bed9d640 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 0x55c394f829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 -- 172.21.15.32:0/2159250444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 msgr2=0x55c394f829f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 0x55c394f829f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 -- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 msgr2=0x55c394f856d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f856d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 -- 172.21.15.32:0/2159250444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c394f82f60 con 0x55c394f74780 2024-04-16T05:36:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9b7fff640 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f824b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8003f00 tx=0x7fa9a8005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4557cf837a93943d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.506+0000 7fa9a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2159250444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8062070 con 0x55c394f74780 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2159250444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fa9a802cdc0 con 0x55c394f74780 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2159250444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b00a06d0 con 0x55c394f74780 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2159250444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9a806e050 con 0x55c394f74780 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 10 client.15933 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2159250444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c394ff44f0 con 0x55c394f74780 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2159250444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9a8044070 con 0x55c394f74780 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2159250444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9a8049040 con 0x55c394f74780 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 10 client.15933 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 10 client.15933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 20 client.15933 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c394ff4c10 0x55c394ff7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 4 client.15933 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.507+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2159250444 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa9b00a06d0 con 0x55c394ff4c10 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c394ff4c10 0x55c394ff7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9be59c640 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c394ff4c10 0x55c394ff7030 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e49e9dcbc09c7bf7 server_cookie=82e0abc14cd7ba94 in_seq=0 out_seq=0 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9a5ffb640 10 client.15933 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9a5ffb640 1 -- 172.21.15.32:0/2159250444 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa9b00a06d0 con 0x55c394ff4c10 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9a5ffb640 10 client.15933 handle_command_reply: tid=0 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 client.15933 shutdown 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2159250444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 msgr2=0x55c394f824b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f824b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8003f00 tx=0x7fa9a8005410 comp rx=0 tx=0).stop 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2159250444 shutdown_connections 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c394ff4c10 0x55c394ff7030 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c394f7d360 0x55c394f829f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c394f74780 0x55c394f824b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 --2- 172.21.15.32:0/2159250444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c394f73e60 0x55c394f856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2159250444 >> 172.21.15.32:0/2159250444 conn(0x55c394f6e590 msgr2=0x55c394f7a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.508+0000 7fa9a5ffb640 0 client.15933 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:36:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2159250444 shutdown_connections 2024-04-16T05:36:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c7bff740 1 -- 172.21.15.32:0/2159250444 wait complete. 2024-04-16T05:36:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c7bff740 20 client.15933 trim_cache size 0 max 16384 2024-04-16T05:36:05.513 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:36:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9ac0a3e40 msgr2=0x7fa9ac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9ac0a3e40 0x7fa9ac0a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8006010 tx=0x7fa9a8005a20 comp rx=0 tx=0).stop 2024-04-16T05:36:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c01032a0 msgr2=0x7fa9c019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c01032a0 0x7fa9c019e610 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000d7a0 tx=0x7fa9b000dc60 comp rx=0 tx=0).stop 2024-04-16T05:36:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/2520583612 shutdown_connections 2024-04-16T05:36:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c010c5e0 0x7fa9c01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0103bc0 0x7fa9c019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9ac0a3e40 0x7fa9ac0a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 --2- 172.21.15.32:0/2520583612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c01032a0 0x7fa9c019e610 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.509+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/2520583612 >> 172.21.15.32:0/2520583612 conn(0x7fa9c0077280 msgr2=0x7fa9c00fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.510+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/2520583612 shutdown_connections 2024-04-16T05:36:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.510+0000 7fa9c4f3d640 1 -- 172.21.15.32:0/2520583612 wait complete. 2024-04-16T05:36:05.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:36:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.647+0000 7f2a9e435640 1 -- 172.21.15.32:0/2396496567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a98103b90 msgr2=0x7f2a9810c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.647+0000 7f2a9e435640 1 --2- 172.21.15.32:0/2396496567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a98103b90 0x7f2a9810c070 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a8002bd60 comp rx=0 tx=0).stop 2024-04-16T05:36:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.647+0000 7f2a9e435640 1 -- 172.21.15.32:0/2396496567 shutdown_connections 2024-04-16T05:36:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.647+0000 7f2a9e435640 1 --2- 172.21.15.32:0/2396496567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a9810c5b0 0x7f2a9810ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.647+0000 7f2a9e435640 1 --2- 172.21.15.32:0/2396496567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a98103b90 0x7f2a9810c070 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.647+0000 7f2a9e435640 1 --2- 172.21.15.32:0/2396496567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98103270 0x7f2a98103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.647+0000 7f2a9e435640 1 -- 172.21.15.32:0/2396496567 >> 172.21.15.32:0/2396496567 conn(0x7f2a980780b0 msgr2=0x7f2a980784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.648+0000 7f2a9e435640 1 -- 172.21.15.32:0/2396496567 shutdown_connections 2024-04-16T05:36:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.648+0000 7f2a9e435640 1 -- 172.21.15.32:0/2396496567 wait complete. 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.648+0000 7f2a9e435640 1 Processor -- start 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.648+0000 7f2a9e435640 1 -- start start 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9e435640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a98103270 0x7f2a9819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9e435640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98103b90 0x7f2a9819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9e435640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9810c5b0 0x7f2a981986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9e435640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981123c0 con 0x7f2a9810c5b0 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9e435640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98112240 con 0x7f2a98103b90 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9e435640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98112540 con 0x7f2a98103270 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9c9d2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9810c5b0 0x7f2a981986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9c9d2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9810c5b0 0x7f2a981986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43558/0 (socket says 172.21.15.32:43558) 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9c9d2640 1 -- 172.21.15.32:0/292305574 learned_addr learned my addr 172.21.15.32:0/292305574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a97fff640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a98103270 0x7f2a9819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98103b90 0x7f2a9819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9c9d2640 1 -- 172.21.15.32:0/292305574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a98103270 msgr2=0x7f2a9819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9c9d2640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a98103270 0x7f2a9819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9c9d2640 1 -- 172.21.15.32:0/292305574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98103b90 msgr2=0x7f2a9819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9c9d2640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98103b90 0x7f2a9819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.649+0000 7f2a9c9d2640 1 -- 172.21.15.32:0/292305574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a98198f60 con 0x7f2a9810c5b0 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.650+0000 7f2a9c9d2640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9810c5b0 0x7f2a981986a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000cce0 tx=0x7f2a9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.650+0000 7f2a95ffb640 1 -- 172.21.15.32:0/292305574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9000f450 con 0x7f2a9810c5b0 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.650+0000 7f2a95ffb640 1 -- 172.21.15.32:0/292305574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a9000fad0 con 0x7f2a9810c5b0 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.650+0000 7f2a9e435640 1 -- 172.21.15.32:0/292305574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a98199250 con 0x7f2a9810c5b0 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.650+0000 7f2a95ffb640 1 -- 172.21.15.32:0/292305574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90018b60 con 0x7f2a9810c5b0 2024-04-16T05:36:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.650+0000 7f2a9e435640 1 -- 172.21.15.32:0/292305574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a981a5560 con 0x7f2a9810c5b0 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.653+0000 7f2a95ffb640 1 -- 172.21.15.32:0/292305574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f2a90018d00 con 0x7f2a9810c5b0 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.653+0000 7f2aa101d740 1 Processor -- start 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.653+0000 7f2aa101d740 1 -- start start 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 0x563b4dc10ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc11430 0x563b4dc1e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc1e770 0x563b4dc20b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2aa101d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563b4db760e0 con 0x563b4dc11430 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2aa101d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563b4db13520 con 0x563b4dc10b10 2024-04-16T05:36:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2aa101d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563b4db720a0 con 0x563b4dc1e770 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc11430 0x563b4dc1e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc11430 0x563b4dc1e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43564/0 (socket says 172.21.15.32:43564) 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a97fff640 1 -- 172.21.15.32:0/303618662 learned_addr learned my addr 172.21.15.32:0/303618662 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 0x563b4dc10ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc1e770 0x563b4dc20b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a95ffb640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2a840a3ec0 0x7f2a840a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a97fff640 1 -- 172.21.15.32:0/303618662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc1e770 msgr2=0x563b4dc20b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a97fff640 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc1e770 0x563b4dc20b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a97fff640 1 -- 172.21.15.32:0/303618662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 msgr2=0x563b4dc10ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a97fff640 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 0x563b4dc10ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a97fff640 1 -- 172.21.15.32:0/303618662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4db3b8d0 con 0x563b4dc11430 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a9c9d2640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2a840a3ec0 0x7f2a840a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.654+0000 7f2a95ffb640 1 -- 172.21.15.32:0/292305574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2a900d11b0 con 0x7f2a9810c5b0 2024-04-16T05:36:05.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.655+0000 7f2a97fff640 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc11430 0x563b4dc1e230 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800ad00 tx=0x7f2a8800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bc3403eb08e568 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.655+0000 7f2a8e7fc640 1 -- 172.21.15.32:0/303618662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8801d070 con 0x563b4dc11430 2024-04-16T05:36:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.655+0000 7f2a8e7fc640 1 -- 172.21.15.32:0/303618662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a88018b20 con 0x563b4dc11430 2024-04-16T05:36:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.655+0000 7f2a8e7fc640 1 -- 172.21.15.32:0/303618662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88010b60 con 0x563b4dc11430 2024-04-16T05:36:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2a9c9d2640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2a840a3ec0 0x7f2a840a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a900ce4a0 tx=0x7f2a9000cf10 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 -- 172.21.15.32:0/303618662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc11430 msgr2=0x563b4dc1e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc11430 0x563b4dc1e230 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800ad00 tx=0x7f2a8800f790 comp rx=0 tx=0).stop 2024-04-16T05:36:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 -- 172.21.15.32:0/303618662 shutdown_connections 2024-04-16T05:36:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc1e770 0x563b4dc20b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc11430 0x563b4dc1e230 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 --2- 172.21.15.32:0/303618662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 0x563b4dc10ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 -- 172.21.15.32:0/303618662 >> 172.21.15.32:0/303618662 conn(0x563b4dc0b590 msgr2=0x563b4dc0b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 -- 172.21.15.32:0/303618662 shutdown_connections 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 -- 172.21.15.32:0/303618662 wait complete. 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.656+0000 7f2aa101d740 1 -- start start 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2aa101d740 1 Processor -- start 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc10b10 0x563b4dc1b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc11430 0x563b4dc1bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2aa101d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563b4dc215b0 con 0x563b4dc1e770 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2aa101d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563b4db2f4d0 con 0x563b4dc11430 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2aa101d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563b4dc21730 con 0x563b4dc10b10 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2aa101d740 10 client.0 _ll_register_callbacks cb 0x563b4db142c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43580/0 (socket says 172.21.15.32:43580) 2024-04-16T05:36:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2891501652 learned_addr learned my addr 172.21.15.32:0/2891501652 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc11430 0x563b4dc1bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc10b10 0x563b4dc1b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2891501652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc10b10 msgr2=0x563b4dc1b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc10b10 0x563b4dc1b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc11430 msgr2=0x563b4dc1bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc11430 0x563b4dc1bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2891501652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4d8883f0 con 0x563b4dc1e770 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.657+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc11430 0x563b4dc1bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.658+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1c160 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2a8002c4d0 tx=0x7f2a8002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daaf0442c5cf1cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.658+0000 7f2a777fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.658+0000 7f2a777fe640 1 -- 172.21.15.32:0/2891501652 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80004b70 con 0x563b4dc1e770 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.658+0000 7f2a777fe640 1 -- 172.21.15.32:0/2891501652 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2a80004f80 con 0x563b4dc1e770 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.658+0000 7f2aa101d740 1 -- 172.21.15.32:0/2891501652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a88018b20 con 0x563b4dc1e770 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.658+0000 7f2a777fe640 1 -- 172.21.15.32:0/2891501652 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a80005180 con 0x563b4dc1e770 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.658+0000 7f2aa101d740 10 client.15945 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.658+0000 7f2aa101d740 1 -- 172.21.15.32:0/2891501652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b4d9527d0 con 0x563b4dc1e770 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2a777fe640 1 -- 172.21.15.32:0/2891501652 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a88018b20 con 0x563b4dc1e770 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2a777fe640 1 -- 172.21.15.32:0/2891501652 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2a80003d00 con 0x563b4dc1e770 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2aa101d740 10 client.15945 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2aa101d740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2aa101d740 20 client.15945 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563b4dc2e750 0x563b4dc30b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2aa101d740 4 client.15945 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2aa101d740 1 -- 172.21.15.32:0/2891501652 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b4db83b00 con 0x563b4dc2e750 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.659+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563b4dc2e750 0x563b4dc30b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.660+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563b4dc2e750 0x563b4dc30b40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d02e0e49bfca4e server_cookie=ea84eb414ed2067b in_seq=0 out_seq=0 2024-04-16T05:36:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.660+0000 7f2a777fe640 10 client.15945 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.660+0000 7f2a777fe640 1 -- 172.21.15.32:0/2891501652 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b4db83b00 con 0x563b4dc2e750 2024-04-16T05:36:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.660+0000 7f2a777fe640 10 client.15945 handle_command_reply: tid=0 2024-04-16T05:36:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.660+0000 7f2aa101d740 1 client.15945 shutdown 2024-04-16T05:36:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 -- 172.21.15.32:0/2891501652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 msgr2=0x563b4dc1c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1c160 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2a8002c4d0 tx=0x7f2a8002c850 comp rx=0 tx=0).stop 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 -- 172.21.15.32:0/2891501652 shutdown_connections 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1c160 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc11430 0x563b4dc1bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563b4dc2e750 0x563b4dc30b40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2891501652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc10b10 0x563b4dc1b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 -- 172.21.15.32:0/2891501652 >> 172.21.15.32:0/2891501652 conn(0x563b4dc0b590 msgr2=0x563b4dc0d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2a777fe640 0 client.15945 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 -- 172.21.15.32:0/2891501652 shutdown_connections 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 1 -- 172.21.15.32:0/2891501652 wait complete. 2024-04-16T05:36:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.661+0000 7f2aa101d740 20 client.15945 trim_cache size 0 max 16384 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.674+0000 7f2aa101d740 1 Processor -- start 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.674+0000 7f2aa101d740 1 -- start start 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.674+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 0x563b4dc1de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.674+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc2a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.674+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.674+0000 7f2aa101d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981af9c0 con 0x563b4dc1e770 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.674+0000 7f2aa101d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981ad290 con 0x563b4dc10b10 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.674+0000 7f2aa101d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981ab0e0 con 0x563b4dc11430 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43588/0 (socket says 172.21.15.32:43588) 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2405428381 learned_addr learned my addr 172.21.15.32:0/2405428381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc2a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 0x563b4dc1de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2405428381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 msgr2=0x563b4dc2a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc2a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2405428381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 msgr2=0x563b4dc1de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 0x563b4dc1de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2405428381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4dc20b60 con 0x563b4dc1e770 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1a6c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2a800434d0 tx=0x7f2a8002bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c44e877eeec41e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8dffb640 1 -- 172.21.15.32:0/2405428381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80004b70 con 0x563b4dc1e770 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8dffb640 1 -- 172.21.15.32:0/2405428381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a800436c0 con 0x563b4dc1e770 2024-04-16T05:36:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.675+0000 7f2a8dffb640 1 -- 172.21.15.32:0/2405428381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8002cc10 con 0x563b4dc1e770 2024-04-16T05:36:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 -- 172.21.15.32:0/2405428381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 msgr2=0x563b4dc1a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1a6c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2a800434d0 tx=0x7f2a8002bd60 comp rx=0 tx=0).stop 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 -- 172.21.15.32:0/2405428381 shutdown_connections 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc1e770 0x563b4dc1a6c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc2a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2405428381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc10b10 0x563b4dc1de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 -- 172.21.15.32:0/2405428381 >> 172.21.15.32:0/2405428381 conn(0x563b4dc0b590 msgr2=0x563b4dc19fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 -- 172.21.15.32:0/2405428381 shutdown_connections 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.676+0000 7f2aa101d740 1 -- 172.21.15.32:0/2405428381 wait complete. 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.677+0000 7f2aa101d740 1 -- start start 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.677+0000 7f2aa101d740 1 Processor -- start 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.677+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc10b10 0x563b4dc26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.677+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc18e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.677+0000 7f2aa101d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc1e770 0x563b4dc19400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.677+0000 7f2aa101d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981ab750 con 0x563b4dc10b10 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.677+0000 7f2aa101d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981acbc0 con 0x563b4dc1e770 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.677+0000 7f2aa101d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981abe20 con 0x563b4dc11430 2024-04-16T05:36:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2aa101d740 10 client.0 _ll_register_callbacks cb 0x7f2a80009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc10b10 0x563b4dc26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc10b10 0x563b4dc26f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43600/0 (socket says 172.21.15.32:43600) 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2859098525 learned_addr learned my addr 172.21.15.32:0/2859098525 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc18e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2859098525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 msgr2=0x563b4dc18e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc18e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2859098525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc1e770 msgr2=0x563b4dc19400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc1e770 0x563b4dc19400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 -- 172.21.15.32:0/2859098525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4dc172e0 con 0x563b4dc10b10 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc18e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a8ffff640 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc10b10 0x563b4dc26f10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2a80043fc0 tx=0x7f2a80043ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3669ea89ee5f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a777fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a777fe640 1 -- 172.21.15.32:0/2859098525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80004b70 con 0x563b4dc10b10 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a777fe640 1 -- 172.21.15.32:0/2859098525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2a8002c9b0 con 0x563b4dc10b10 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2aa101d740 1 -- 172.21.15.32:0/2859098525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a800436c0 con 0x563b4dc10b10 2024-04-16T05:36:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2a777fe640 1 -- 172.21.15.32:0/2859098525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a8002cbb0 con 0x563b4dc10b10 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2aa101d740 10 client.15951 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.678+0000 7f2aa101d740 1 -- 172.21.15.32:0/2859098525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b4dc94070 con 0x563b4dc10b10 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2a777fe640 1 -- 172.21.15.32:0/2859098525 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a800436c0 con 0x563b4dc10b10 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2a777fe640 1 -- 172.21.15.32:0/2859098525 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2a8002d030 con 0x563b4dc10b10 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2aa101d740 10 client.15951 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2aa101d740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2aa101d740 20 client.15951 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563b4dc94570 0x563b4dc96990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2aa101d740 4 client.15951 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2aa101d740 1 -- 172.21.15.32:0/2859098525 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b4db7d970 con 0x563b4dc94570 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.679+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563b4dc94570 0x563b4dc96990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2a97fff640 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563b4dc94570 0x563b4dc96990 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fdf3883f0734b18 server_cookie=695883ee3f1aa119 in_seq=0 out_seq=0 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2a777fe640 10 client.15951 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2a777fe640 1 -- 172.21.15.32:0/2859098525 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b4db7d970 con 0x563b4dc94570 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2a777fe640 10 client.15951 handle_command_reply: tid=0 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 client.15951 shutdown 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 -- 172.21.15.32:0/2859098525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc10b10 msgr2=0x563b4dc26f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc10b10 0x563b4dc26f10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2a80043fc0 tx=0x7f2a80043ba0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 -- 172.21.15.32:0/2859098525 shutdown_connections 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563b4dc1e770 0x563b4dc19400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563b4dc94570 0x563b4dc96990 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b4dc11430 0x563b4dc18e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 --2- 172.21.15.32:0/2859098525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b4dc10b10 0x563b4dc26f10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 -- 172.21.15.32:0/2859098525 >> 172.21.15.32:0/2859098525 conn(0x563b4dc0b590 msgr2=0x563b4dc1a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2a777fe640 0 client.15951 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 -- 172.21.15.32:0/2859098525 shutdown_connections 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.680+0000 7f2aa101d740 1 -- 172.21.15.32:0/2859098525 wait complete. 2024-04-16T05:36:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2aa101d740 20 client.15951 trim_cache size 0 max 16384 2024-04-16T05:36:05.684 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:36:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 -- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2a840a3ec0 msgr2=0x7f2a840a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2a840a3ec0 0x7f2a840a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a900ce4a0 tx=0x7f2a9000cf10 comp rx=0 tx=0).stop 2024-04-16T05:36:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 -- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9810c5b0 msgr2=0x7f2a981986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9810c5b0 0x7f2a981986a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000cce0 tx=0x7f2a9000a620 comp rx=0 tx=0).stop 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 -- 172.21.15.32:0/292305574 shutdown_connections 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2a840a3ec0 0x7f2a840a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9810c5b0 0x7f2a981986a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98103b90 0x7f2a9819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 --2- 172.21.15.32:0/292305574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a98103270 0x7f2a9819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 -- 172.21.15.32:0/292305574 >> 172.21.15.32:0/292305574 conn(0x7f2a980780b0 msgr2=0x7f2a980ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 -- 172.21.15.32:0/292305574 shutdown_connections 2024-04-16T05:36:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:05.681+0000 7f2a9e435640 1 -- 172.21.15.32:0/292305574 wait complete. 2024-04-16T05:36:05.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:05 smithi032 ceph-mon[32351]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:05.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:05 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2509414488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:06.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:05 smithi134 ceph-mon[36153]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:06.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:05 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2509414488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:06.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:05 smithi082 ceph-mon[36195]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:06.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:05 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2509414488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:06.692 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:36:06.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:36:06.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 -- 172.21.15.32:0/1546536263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8101af0 msgr2=0x7f95d8105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 --2- 172.21.15.32:0/1546536263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8101af0 0x7f95d8105b30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009900 tx=0x7f95cc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 -- 172.21.15.32:0/1546536263 shutdown_connections 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 --2- 172.21.15.32:0/1546536263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95d8106210 0x7f95d810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 --2- 172.21.15.32:0/1546536263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8101af0 0x7f95d8105b30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 --2- 172.21.15.32:0/1546536263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d8101140 0x7f95d8101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 -- 172.21.15.32:0/1546536263 >> 172.21.15.32:0/1546536263 conn(0x7f95d8078000 msgr2=0x7f95d8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 -- 172.21.15.32:0/1546536263 shutdown_connections 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 -- 172.21.15.32:0/1546536263 wait complete. 2024-04-16T05:36:06.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.819+0000 7f95dd900640 1 Processor -- start 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95dd900640 1 -- start start 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95dd900640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95d8101140 0x7f95d819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95dd900640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d8106210 0x7f95d819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95dd900640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8198720 0x7f95d8198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95dd900640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81123e0 con 0x7f95d8198720 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95dd900640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8112260 con 0x7f95d8101140 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95dd900640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8112560 con 0x7f95d8106210 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8198720 0x7f95d8198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8198720 0x7f95d8198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43620/0 (socket says 172.21.15.32:43620) 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95d77fe640 1 -- 172.21.15.32:0/2053719814 learned_addr learned my addr 172.21.15.32:0/2053719814 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:06.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95d67fc640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d8106210 0x7f95d819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.820+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95d8101140 0x7f95d819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95d77fe640 1 -- 172.21.15.32:0/2053719814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d8106210 msgr2=0x7f95d819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95d77fe640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d8106210 0x7f95d819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95d77fe640 1 -- 172.21.15.32:0/2053719814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95d8101140 msgr2=0x7f95d819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95d77fe640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95d8101140 0x7f95d819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95d77fe640 1 -- 172.21.15.32:0/2053719814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d8199320 con 0x7f95d8198720 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95d8101140 0x7f95d819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95d77fe640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8198720 0x7f95d8198ba0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f95d000eb10 tx=0x7f95d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95dc8fe640 1 -- 172.21.15.32:0/2053719814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0004b20 con 0x7f95d8198720 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95dc8fe640 1 -- 172.21.15.32:0/2053719814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d0004cc0 con 0x7f95d8198720 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95dd900640 1 -- 172.21.15.32:0/2053719814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d81a5400 con 0x7f95d8198720 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95dc8fe640 1 -- 172.21.15.32:0/2053719814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0010760 con 0x7f95d8198720 2024-04-16T05:36:06.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.821+0000 7f95dd900640 1 -- 172.21.15.32:0/2053719814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81a57b0 con 0x7f95d8198720 2024-04-16T05:36:06.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.824+0000 7f95e05be740 1 Processor -- start 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.824+0000 7f95dc8fe640 1 -- 172.21.15.32:0/2053719814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f95d0005020 con 0x7f95d8198720 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.824+0000 7f95e05be740 1 -- start start 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95e05be740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 0x55de8d465b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95e05be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d472e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95e05be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d4757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95e05be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55de8d3cae90 con 0x55de8d4733c0 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95e05be740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55de8d3684c0 con 0x55de8d465760 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95e05be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55de8d3c6e50 con 0x55de8d466080 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d4757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d4757b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43632/0 (socket says 172.21.15.32:43632) 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 -- 172.21.15.32:0/3314510285 learned_addr learned my addr 172.21.15.32:0/3314510285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d472e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 0x55de8d465b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95dc8fe640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95c00a3ec0 0x7f95c00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 -- 172.21.15.32:0/3314510285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 msgr2=0x55de8d472e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d472e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 -- 172.21.15.32:0/3314510285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 msgr2=0x55de8d465b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 0x55de8d465b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.825+0000 7f95d67fc640 1 -- 172.21.15.32:0/3314510285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de8d1a57e0 con 0x55de8d4733c0 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.826+0000 7f95d77fe640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95c00a3ec0 0x7f95c00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.826+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d4757b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95cc005ec0 tx=0x7f95cc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8b9b907cf842ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.826+0000 7f95dc8fe640 1 -- 172.21.15.32:0/2053719814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f95d00d1350 con 0x7f95d8198720 2024-04-16T05:36:06.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.826+0000 7f95bdffb640 1 -- 172.21.15.32:0/3314510285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc004030 con 0x55de8d4733c0 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.826+0000 7f95bdffb640 1 -- 172.21.15.32:0/3314510285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc004ba0 con 0x55de8d4733c0 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.826+0000 7f95bdffb640 1 -- 172.21.15.32:0/3314510285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc005380 con 0x55de8d4733c0 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 -- 172.21.15.32:0/3314510285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 msgr2=0x55de8d4757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95d77fe640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95c00a3ec0 0x7f95c00a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d00ce640 tx=0x7f95d0080040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d4757b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95cc005ec0 tx=0x7f95cc005dd0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 -- 172.21.15.32:0/3314510285 shutdown_connections 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d4757b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d472e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 --2- 172.21.15.32:0/3314510285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 0x55de8d465b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 -- 172.21.15.32:0/3314510285 >> 172.21.15.32:0/3314510285 conn(0x55de8d4601e0 msgr2=0x55de8d4605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 -- 172.21.15.32:0/3314510285 shutdown_connections 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 -- 172.21.15.32:0/3314510285 wait complete. 2024-04-16T05:36:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.827+0000 7f95e05be740 1 -- start start 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95e05be740 1 Processor -- start 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95e05be740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 0x55de8d470310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95e05be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d470850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95e05be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d470d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95e05be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55de8d4761f0 con 0x55de8d4733c0 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95e05be740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55de8d353410 con 0x55de8d465760 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95e05be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55de8d476370 con 0x55de8d466080 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95e05be740 10 client.0 _ll_register_callbacks cb 0x55de8d34eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95d67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d470d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95d67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d470d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43638/0 (socket says 172.21.15.32:43638) 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95d67fc640 1 -- 172.21.15.32:0/3230927516 learned_addr learned my addr 172.21.15.32:0/3230927516 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d470850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.828+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 0x55de8d470310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95d67fc640 1 -- 172.21.15.32:0/3230927516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 msgr2=0x55de8d470850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d470850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95d67fc640 1 -- 172.21.15.32:0/3230927516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 msgr2=0x55de8d470310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 0x55de8d470310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95d67fc640 1 -- 172.21.15.32:0/3230927516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de8d205f70 con 0x55de8d4733c0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d470d90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f95cc02e040 tx=0x7f95cc043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b2e87006ab3995 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95b6ffd640 1 -- 172.21.15.32:0/3230927516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc058060 con 0x55de8d4733c0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95b6ffd640 1 -- 172.21.15.32:0/3230927516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f95cc0046c0 con 0x55de8d4733c0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95e05be740 1 -- 172.21.15.32:0/3230927516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95cc004ba0 con 0x55de8d4733c0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95b6ffd640 1 -- 172.21.15.32:0/3230927516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f95cc05f050 con 0x55de8d4733c0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95e05be740 10 client.15963 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.829+0000 7f95e05be740 1 -- 172.21.15.32:0/3230927516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55de8d09d170 con 0x55de8d4733c0 2024-04-16T05:36:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95b6ffd640 1 -- 172.21.15.32:0/3230927516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc004ba0 con 0x55de8d4733c0 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95b6ffd640 1 -- 172.21.15.32:0/3230927516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f95cc03f410 con 0x55de8d4733c0 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95e05be740 10 client.15963 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95e05be740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95e05be740 20 client.15963 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95e05be740 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55de8d483650 0x55de8d485a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95e05be740 4 client.15963 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95e05be740 1 -- 172.21.15.32:0/3230927516 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de8d3d88b0 con 0x55de8d483650 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.830+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55de8d483650 0x55de8d485a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.831+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55de8d483650 0x55de8d485a40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc7ac896902ee81 server_cookie=95327a9af5572663 in_seq=0 out_seq=0 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.831+0000 7f95b6ffd640 10 client.15963 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.831+0000 7f95b6ffd640 1 -- 172.21.15.32:0/3230927516 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de8d3d88b0 con 0x55de8d483650 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.831+0000 7f95b6ffd640 10 client.15963 handle_command_reply: tid=0 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.831+0000 7f95e05be740 1 client.15963 shutdown 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 -- 172.21.15.32:0/3230927516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 msgr2=0x55de8d470d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d470d90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f95cc02e040 tx=0x7f95cc043040 comp rx=0 tx=0).stop 2024-04-16T05:36:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 -- 172.21.15.32:0/3230927516 shutdown_connections 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d4733c0 0x55de8d470d90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55de8d483650 0x55de8d485a40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d470850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 --2- 172.21.15.32:0/3230927516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d465760 0x55de8d470310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 -- 172.21.15.32:0/3230927516 >> 172.21.15.32:0/3230927516 conn(0x55de8d4601e0 msgr2=0x55de8d461f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95b6ffd640 0 client.15963 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 -- 172.21.15.32:0/3230927516 shutdown_connections 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 1 -- 172.21.15.32:0/3230927516 wait complete. 2024-04-16T05:36:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.832+0000 7f95e05be740 20 client.15963 trim_cache size 0 max 16384 2024-04-16T05:36:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.864+0000 7f95e05be740 1 Processor -- start 2024-04-16T05:36:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95e05be740 1 -- start start 2024-04-16T05:36:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95e05be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d465760 0x55de8d47f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95e05be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d46f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95e05be740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d46ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95e05be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81af9d0 con 0x55de8d465760 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95e05be740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ad2a0 con 0x55de8d4733c0 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95e05be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ab0f0 con 0x55de8d466080 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d465760 0x55de8d47f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d465760 0x55de8d47f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43646/0 (socket says 172.21.15.32:43646) 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 -- 172.21.15.32:0/3588736803 learned_addr learned my addr 172.21.15.32:0/3588736803 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d46f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d46ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 -- 172.21.15.32:0/3588736803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 msgr2=0x55de8d46f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d46f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 -- 172.21.15.32:0/3588736803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 msgr2=0x55de8d46ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d46ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d67fc640 1 -- 172.21.15.32:0/3588736803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de8d465eb0 con 0x55de8d465760 2024-04-16T05:36:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.865+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d46f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.866+0000 7f95d67fc640 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d465760 0x55de8d47f1d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009900 tx=0x7f95cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74afe717be1ea71 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.866+0000 7f95bd7fa640 1 -- 172.21.15.32:0/3588736803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc058040 con 0x55de8d465760 2024-04-16T05:36:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.866+0000 7f95bd7fa640 1 -- 172.21.15.32:0/3588736803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc009520 con 0x55de8d465760 2024-04-16T05:36:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.866+0000 7f95bd7fa640 1 -- 172.21.15.32:0/3588736803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc03f410 con 0x55de8d465760 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 -- 172.21.15.32:0/3588736803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d465760 msgr2=0x55de8d47f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d465760 0x55de8d47f1d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009900 tx=0x7f95cc005fd0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 -- 172.21.15.32:0/3588736803 shutdown_connections 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d46ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d466080 0x55de8d46f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 --2- 172.21.15.32:0/3588736803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d465760 0x55de8d47f1d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 -- 172.21.15.32:0/3588736803 >> 172.21.15.32:0/3588736803 conn(0x55de8d4601e0 msgr2=0x55de8d461630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 -- 172.21.15.32:0/3588736803 shutdown_connections 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 -- 172.21.15.32:0/3588736803 wait complete. 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 -- start start 2024-04-16T05:36:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.867+0000 7f95e05be740 1 Processor -- start 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95e05be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d465760 0x55de8d46d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95e05be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d466080 0x55de8d46db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95e05be740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d4e88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95e05be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ab760 con 0x55de8d466080 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95e05be740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81acbd0 con 0x55de8d4733c0 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95e05be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81abe30 con 0x55de8d465760 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95e05be740 10 client.0 _ll_register_callbacks cb 0x7f95cc02c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d466080 0x55de8d46db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d466080 0x55de8d46db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43648/0 (socket says 172.21.15.32:43648) 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 -- 172.21.15.32:0/4167156193 learned_addr learned my addr 172.21.15.32:0/4167156193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d67fc640 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d465760 0x55de8d46d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d67fc640 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d4e88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 -- 172.21.15.32:0/4167156193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d465760 msgr2=0x55de8d46d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d465760 0x55de8d46d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 -- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 msgr2=0x55de8d4e88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d4e88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 -- 172.21.15.32:0/4167156193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de8d4e8df0 con 0x55de8d466080 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d67fc640 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d4e88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95d6ffd640 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d466080 0x55de8d46db80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55de8d46fb70 tx=0x7f95c80139e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f26a2a46665e4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.868+0000 7f95b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95b6ffd640 1 -- 172.21.15.32:0/4167156193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c801d620 con 0x55de8d466080 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95b6ffd640 1 -- 172.21.15.32:0/4167156193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f95c801d7c0 con 0x55de8d466080 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 1 -- 172.21.15.32:0/4167156193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95cc009520 con 0x55de8d466080 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95b6ffd640 1 -- 172.21.15.32:0/4167156193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f95c802d050 con 0x55de8d466080 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 10 client.15969 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 1 -- 172.21.15.32:0/4167156193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55de8d4ea0c0 con 0x55de8d466080 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95b6ffd640 1 -- 172.21.15.32:0/4167156193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc009520 con 0x55de8d466080 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95b6ffd640 1 -- 172.21.15.32:0/4167156193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f95c8011600 con 0x55de8d466080 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 10 client.15969 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 20 client.15969 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55de8d4ea870 0x55de8d4ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 4 client.15969 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:36:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.869+0000 7f95e05be740 1 -- 172.21.15.32:0/4167156193 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55de8d3d2720 con 0x55de8d4ea870 2024-04-16T05:36:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.870+0000 7f95d67fc640 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55de8d4ea870 0x55de8d4ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.870+0000 7f95b6ffd640 10 client.15969 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.870+0000 7f95d67fc640 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55de8d4ea870 0x55de8d4ecc90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f88a8b04bd4df95 server_cookie=aa35e1f146345e72 in_seq=0 out_seq=0 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.871+0000 7f95b6ffd640 1 -- 172.21.15.32:0/4167156193 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55de8d3d2720 con 0x55de8d4ea870 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.871+0000 7f95b6ffd640 10 client.15969 handle_command_reply: tid=0 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.871+0000 7f95e05be740 1 client.15969 shutdown 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 -- 172.21.15.32:0/4167156193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d466080 msgr2=0x55de8d46db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d466080 0x55de8d46db80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55de8d46fb70 tx=0x7f95c80139e0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 -- 172.21.15.32:0/4167156193 shutdown_connections 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8d4733c0 0x55de8d4e88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de8d466080 0x55de8d46db80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55de8d4ea870 0x55de8d4ecc90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 --2- 172.21.15.32:0/4167156193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de8d465760 0x55de8d46d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95b6ffd640 0 client.15969 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 -- 172.21.15.32:0/4167156193 >> 172.21.15.32:0/4167156193 conn(0x55de8d4601e0 msgr2=0x55de8d461ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 -- 172.21.15.32:0/4167156193 shutdown_connections 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 1 -- 172.21.15.32:0/4167156193 wait complete. 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.872+0000 7f95e05be740 20 client.15969 trim_cache size 0 max 16384 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "79b225b3-8e79-4ad9-b9a8-e8af1691fa06", 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:36:06.874 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:36:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 -- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95c00a3ec0 msgr2=0x7f95c00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95c00a3ec0 0x7f95c00a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d00ce640 tx=0x7f95d0080040 comp rx=0 tx=0).stop 2024-04-16T05:36:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 -- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8198720 msgr2=0x7f95d8198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8198720 0x7f95d8198ba0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f95d000eb10 tx=0x7f95d000c2d0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 -- 172.21.15.32:0/2053719814 shutdown_connections 2024-04-16T05:36:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95c00a3ec0 0x7f95c00a6380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95d8198720 0x7f95d8198ba0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d8106210 0x7f95d819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 --2- 172.21.15.32:0/2053719814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95d8101140 0x7f95d819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 -- 172.21.15.32:0/2053719814 >> 172.21.15.32:0/2053719814 conn(0x7f95d8078000 msgr2=0x7f95d80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 -- 172.21.15.32:0/2053719814 shutdown_connections 2024-04-16T05:36:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:06.873+0000 7f95dd900640 1 -- 172.21.15.32:0/2053719814 wait complete. 2024-04-16T05:36:06.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:36:06.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:36:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 -- 172.21.15.32:0/2109128062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410102c00 msgr2=0x7f5410106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 --2- 172.21.15.32:0/2109128062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410102c00 0x7f5410106c40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009900 tx=0x7f53f802bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 -- 172.21.15.32:0/2109128062 shutdown_connections 2024-04-16T05:36:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 --2- 172.21.15.32:0/2109128062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410107320 0x7f541010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 --2- 172.21.15.32:0/2109128062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410102c00 0x7f5410106c40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 --2- 172.21.15.32:0/2109128062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5410102250 0x7f5410102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 -- 172.21.15.32:0/2109128062 >> 172.21.15.32:0/2109128062 conn(0x7f5410079400 msgr2=0x7f5410079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 -- 172.21.15.32:0/2109128062 shutdown_connections 2024-04-16T05:36:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.013+0000 7f54149a8640 1 -- 172.21.15.32:0/2109128062 wait complete. 2024-04-16T05:36:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.014+0000 7f54149a8640 1 Processor -- start 2024-04-16T05:36:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f54149a8640 1 -- start start 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f54149a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410102250 0x7f5410074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f54149a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5410107320 0x7f5410074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f54149a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410074d80 0x7f54101a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f54149a8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54101123b0 con 0x7f5410074d80 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f54149a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5410112230 con 0x7f5410102250 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f54149a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5410112530 con 0x7f5410107320 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410074d80 0x7f54101a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410074d80 0x7f54101a6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43660/0 (socket says 172.21.15.32:43660) 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540ed9d640 1 -- 172.21.15.32:0/3326148876 learned_addr learned my addr 172.21.15.32:0/3326148876 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f5405d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5410107320 0x7f5410074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540e59c640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410102250 0x7f5410074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540ed9d640 1 -- 172.21.15.32:0/3326148876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5410107320 msgr2=0x7f5410074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540ed9d640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5410107320 0x7f5410074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540ed9d640 1 -- 172.21.15.32:0/3326148876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410102250 msgr2=0x7f5410074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540ed9d640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410102250 0x7f5410074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540ed9d640 1 -- 172.21.15.32:0/3326148876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54101a7400 con 0x7f5410074d80 2024-04-16T05:36:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.015+0000 7f540e59c640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410102250 0x7f5410074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.016+0000 7f540ed9d640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410074d80 0x7f54101a6d60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f540800eaa0 tx=0x7f540800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.016+0000 7f5407fff640 1 -- 172.21.15.32:0/3326148876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540800cdb0 con 0x7f5410074d80 2024-04-16T05:36:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.016+0000 7f5407fff640 1 -- 172.21.15.32:0/3326148876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5408005020 con 0x7f5410074d80 2024-04-16T05:36:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.016+0000 7f5407fff640 1 -- 172.21.15.32:0/3326148876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54080116b0 con 0x7f5410074d80 2024-04-16T05:36:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.016+0000 7f54149a8640 1 -- 172.21.15.32:0/3326148876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54101a76f0 con 0x7f5410074d80 2024-04-16T05:36:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.016+0000 7f54149a8640 1 -- 172.21.15.32:0/3326148876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54101a7bc0 con 0x7f5410074d80 2024-04-16T05:36:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.019+0000 7f5407fff640 1 -- 172.21.15.32:0/3326148876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f5408005560 con 0x7f5410074d80 2024-04-16T05:36:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.019+0000 7f5417598740 1 Processor -- start 2024-04-16T05:36:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5417598740 1 -- start start 2024-04-16T05:36:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5417598740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c004230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5417598740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49c00ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5417598740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c00d350 0x55f49c00f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5417598740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f49bf69180 con 0x55f49c004770 2024-04-16T05:36:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5417598740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f49bf06360 con 0x55f49c003e50 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5417598740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f49bf65140 con 0x55f49c00d350 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f540e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49c00ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f540e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49c00ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43668/0 (socket says 172.21.15.32:43668) 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f540e59c640 1 -- 172.21.15.32:0/2530193172 learned_addr learned my addr 172.21.15.32:0/2530193172 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c004230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5407fff640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53fc0a3e40 0x7f53fc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.020+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c00d350 0x55f49c00f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f540ed9d640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53fc0a3e40 0x7f53fc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5405d9b640 1 -- 172.21.15.32:0/2530193172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c00d350 msgr2=0x55f49c00f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c00d350 0x55f49c00f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5405d9b640 1 -- 172.21.15.32:0/2530193172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 msgr2=0x55f49c00ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49c00ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5407fff640 1 -- 172.21.15.32:0/3326148876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f54080d0f20 con 0x7f5410074d80 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5405d9b640 1 -- 172.21.15.32:0/2530193172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49bd457d0 con 0x55f49c003e50 2024-04-16T05:36:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f540e59c640 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49c00ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c004230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5410103150 tx=0x7f53f8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d54314bcfa2d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5404d99640 1 -- 172.21.15.32:0/2530193172 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8004020 con 0x55f49c003e50 2024-04-16T05:36:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5404d99640 1 -- 172.21.15.32:0/2530193172 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f8004ba0 con 0x55f49c003e50 2024-04-16T05:36:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.021+0000 7f5404d99640 1 -- 172.21.15.32:0/2530193172 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8005250 con 0x55f49c003e50 2024-04-16T05:36:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 -- 172.21.15.32:0/2530193172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 msgr2=0x55f49c004230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f540ed9d640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53fc0a3e40 0x7f53fc0a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f54080ce210 tx=0x7f5408087040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c004230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5410103150 tx=0x7f53f8005fd0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 -- 172.21.15.32:0/2530193172 shutdown_connections 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c00d350 0x55f49c00f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49c00ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 --2- 172.21.15.32:0/2530193172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c004230 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 -- 172.21.15.32:0/2530193172 >> 172.21.15.32:0/2530193172 conn(0x55f49bffe580 msgr2=0x55f49bffe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 -- 172.21.15.32:0/2530193172 shutdown_connections 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.022+0000 7f5417598740 1 -- 172.21.15.32:0/2530193172 wait complete. 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 1 -- start start 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 1 Processor -- start 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c00a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c004770 0x55f49c00a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c00ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f49c010080 con 0x55f49c00d350 2024-04-16T05:36:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f49bf222c0 con 0x55f49c003e50 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f49c010200 con 0x55f49c004770 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5417598740 10 client.0 _ll_register_callbacks cb 0x55f49bf09090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5405d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c00ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5405d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c00ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43684/0 (socket says 172.21.15.32:43684) 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5405d9b640 1 -- 172.21.15.32:0/3513718873 learned_addr learned my addr 172.21.15.32:0/3513718873 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f5405d9b640 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c00a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.023+0000 7f540e59c640 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c004770 0x55f49c00a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5405d9b640 1 -- 172.21.15.32:0/3513718873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c004770 msgr2=0x55f49c00a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5405d9b640 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c004770 0x55f49c00a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5405d9b640 1 -- 172.21.15.32:0/3513718873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 msgr2=0x55f49c00a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5405d9b640 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c00a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5405d9b640 1 -- 172.21.15.32:0/3513718873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49bc7b3f0 con 0x55f49c00d350 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5405d9b640 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c00ad80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f53f8005930 tx=0x7f53f8043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11edb2bfd15a197 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f53f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f53f5ffb640 1 -- 172.21.15.32:0/3513718873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8054060 con 0x55f49c00d350 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f53f5ffb640 1 -- 172.21.15.32:0/3513718873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f53f8004fd0 con 0x55f49c00d350 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5417598740 1 -- 172.21.15.32:0/3513718873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f8004ba0 con 0x55f49c00d350 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f53f5ffb640 1 -- 172.21.15.32:0/3513718873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f53f8004d50 con 0x55f49c00d350 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5417598740 10 client.15981 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f5417598740 1 -- 172.21.15.32:0/3513718873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f49bc3b0f0 con 0x55f49c00d350 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.024+0000 7f53f5ffb640 1 -- 172.21.15.32:0/3513718873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f8004ba0 con 0x55f49c00d350 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.025+0000 7f53f5ffb640 1 -- 172.21.15.32:0/3513718873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f53f8003b80 con 0x55f49c00d350 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.025+0000 7f5417598740 10 client.15981 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.025+0000 7f5417598740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.025+0000 7f5417598740 20 client.15981 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.025+0000 7f5417598740 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f49c01d420 0x55f49c01f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.025+0000 7f5417598740 4 client.15981 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.025+0000 7f5417598740 1 -- 172.21.15.32:0/3513718873 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f49bf76ba0 con 0x55f49c01d420 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.025+0000 7f540e59c640 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f49c01d420 0x55f49c01f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f53f5ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f540e59c640 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f49c01d420 0x55f49c01f810 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61772954269a5f8 server_cookie=22769e1a2931a82a in_seq=0 out_seq=0 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f53f5ffb640 1 -- 172.21.15.32:0/3513718873 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f49bf76ba0 con 0x55f49c01d420 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f53f5ffb640 10 client.15981 handle_command_reply: tid=0 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f5417598740 1 client.15981 shutdown 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f5417598740 1 -- 172.21.15.32:0/3513718873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 msgr2=0x55f49c00ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f5417598740 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c00ad80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f53f8005930 tx=0x7f53f8043040 comp rx=0 tx=0).stop 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f5417598740 1 -- 172.21.15.32:0/3513718873 shutdown_connections 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f5417598740 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c00ad80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f5417598740 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f49c01d420 0x55f49c01f810 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f5417598740 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c004770 0x55f49c00a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f53f5ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.026+0000 7f5417598740 1 --2- 172.21.15.32:0/3513718873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c003e50 0x55f49c00a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.027+0000 7f5417598740 1 -- 172.21.15.32:0/3513718873 >> 172.21.15.32:0/3513718873 conn(0x55f49bffe580 msgr2=0x55f49c0002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.027+0000 7f5417598740 1 -- 172.21.15.32:0/3513718873 shutdown_connections 2024-04-16T05:36:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.027+0000 7f5417598740 1 -- 172.21.15.32:0/3513718873 wait complete. 2024-04-16T05:36:07.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.027+0000 7f5417598740 20 client.15981 trim_cache size 0 max 16384 2024-04-16T05:36:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.061+0000 7f5417598740 1 Processor -- start 2024-04-16T05:36:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.061+0000 7f5417598740 1 -- start start 2024-04-16T05:36:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5417598740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 0x55f49c00ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5417598740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49bffd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5417598740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c00d350 0x55f49c018b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5417598740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54101b4470 con 0x55f49c004770 2024-04-16T05:36:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5417598740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f54101b1d40 con 0x55f49c00d350 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5417598740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54101afb90 con 0x55f49c003e50 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49bffd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49bffd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43692/0 (socket says 172.21.15.32:43692) 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 -- 172.21.15.32:0/3392768223 learned_addr learned my addr 172.21.15.32:0/3392768223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5405d9b640 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 0x55f49c00ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5405d9b640 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c00d350 0x55f49c018b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 -- 172.21.15.32:0/3392768223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 msgr2=0x55f49c00ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 0x55f49c00ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 -- 172.21.15.32:0/3392768223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c00d350 msgr2=0x55f49c018b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c00d350 0x55f49c018b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 -- 172.21.15.32:0/3392768223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49bffe3b0 con 0x55f49c004770 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f5405d9b640 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c00d350 0x55f49c018b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.062+0000 7f540e59c640 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49bffd760 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55f49c009b60 tx=0x7f540000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb485f342bcef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.063+0000 7f53f7fff640 1 -- 172.21.15.32:0/3392768223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400023070 con 0x55f49c004770 2024-04-16T05:36:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.063+0000 7f53f7fff640 1 -- 172.21.15.32:0/3392768223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5400004490 con 0x55f49c004770 2024-04-16T05:36:07.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.063+0000 7f53f7fff640 1 -- 172.21.15.32:0/3392768223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540001e400 con 0x55f49c004770 2024-04-16T05:36:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.063+0000 7f5417598740 1 -- 172.21.15.32:0/3392768223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 msgr2=0x55f49bffd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.063+0000 7f5417598740 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49bffd760 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55f49c009b60 tx=0x7f540000b390 comp rx=0 tx=0).stop 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 -- 172.21.15.32:0/3392768223 shutdown_connections 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c00d350 0x55f49c018b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c004770 0x55f49bffd760 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 --2- 172.21.15.32:0/3392768223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 0x55f49c00ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 -- 172.21.15.32:0/3392768223 >> 172.21.15.32:0/3392768223 conn(0x55f49bffe580 msgr2=0x55f49c018450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 -- 172.21.15.32:0/3392768223 shutdown_connections 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 -- 172.21.15.32:0/3392768223 wait complete. 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 -- start start 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 Processor -- start 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 0x55f49c07bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c004770 0x55f49c07c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c0220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54101b0200 con 0x55f49c00d350 2024-04-16T05:36:07.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f54101b1670 con 0x55f49c004770 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.064+0000 7f5417598740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54101b08d0 con 0x55f49c003e50 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5417598740 10 client.0 _ll_register_callbacks cb 0x7f540000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c0220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c0220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43704/0 (socket says 172.21.15.32:43704) 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 -- 172.21.15.32:0/2330807001 learned_addr learned my addr 172.21.15.32:0/2330807001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 0x55f49c07bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f540e59c640 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c004770 0x55f49c07c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 -- 172.21.15.32:0/2330807001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 msgr2=0x55f49c07bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 0x55f49c07bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 -- 172.21.15.32:0/2330807001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c004770 msgr2=0x55f49c07c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c004770 0x55f49c07c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 -- 172.21.15.32:0/2330807001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49c0227d0 con 0x55f49c00d350 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5405d9b640 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c0220e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f53f8004570 tx=0x7f53f8043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8683b8d949accf3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f53f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f53f5ffb640 1 -- 172.21.15.32:0/2330807001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f805f070 con 0x55f49c00d350 2024-04-16T05:36:07.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f53f5ffb640 1 -- 172.21.15.32:0/2330807001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f53f80035f0 con 0x55f49c00d350 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.065+0000 7f5417598740 1 -- 172.21.15.32:0/2330807001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5400004490 con 0x55f49c00d350 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f53f5ffb640 1 -- 172.21.15.32:0/2330807001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f53f8054040 con 0x55f49c00d350 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f5417598740 10 client.15987 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f5417598740 1 -- 172.21.15.32:0/2330807001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f49c0836f0 con 0x55f49c00d350 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f53f5ffb640 1 -- 172.21.15.32:0/2330807001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5400004490 con 0x55f49c00d350 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f53f5ffb640 1 -- 172.21.15.32:0/2330807001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f53f8005340 con 0x55f49c00d350 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f5417598740 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f5417598740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f5417598740 20 client.15987 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f5417598740 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f49c083d80 0x55f49c0861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f5417598740 4 client.15987 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f5417598740 1 -- 172.21.15.32:0/2330807001 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f49bf70a10 con 0x55f49c083d80 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.066+0000 7f540e59c640 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f49c083d80 0x55f49c0861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f540e59c640 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f49c083d80 0x55f49c0861a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a738435b20bdc205 server_cookie=e81d93903b1317da in_seq=0 out_seq=0 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f53f5ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:07.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f53f5ffb640 1 -- 172.21.15.32:0/2330807001 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f49bf70a10 con 0x55f49c083d80 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f53f5ffb640 10 client.15987 handle_command_reply: tid=0 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 client.15987 shutdown 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 -- 172.21.15.32:0/2330807001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 msgr2=0x55f49c0220e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c0220e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f53f8004570 tx=0x7f53f8043040 comp rx=0 tx=0).stop 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 -- 172.21.15.32:0/2330807001 shutdown_connections 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f49c00d350 0x55f49c0220e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f49c004770 0x55f49c07c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f49c083d80 0x55f49c0861a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 --2- 172.21.15.32:0/2330807001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f49c003e50 0x55f49c07bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f53f5ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 -- 172.21.15.32:0/2330807001 >> 172.21.15.32:0/2330807001 conn(0x55f49bffe580 msgr2=0x55f49c00ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 -- 172.21.15.32:0/2330807001 shutdown_connections 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 1 -- 172.21.15.32:0/2330807001 wait complete. 2024-04-16T05:36:07.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.067+0000 7f5417598740 20 client.15987 trim_cache size 0 max 16384 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "79b225b3-8e79-4ad9-b9a8-e8af1691fa06": { 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "79b225b3-8e79-4ad9-b9a8-e8af1691fa06", 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:36:07.071 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 -- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53fc0a3e40 msgr2=0x7f53fc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53fc0a3e40 0x7f53fc0a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f54080ce210 tx=0x7f5408087040 comp rx=0 tx=0).stop 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 -- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410074d80 msgr2=0x7f54101a6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410074d80 0x7f54101a6d60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f540800eaa0 tx=0x7f540800ef60 comp rx=0 tx=0).stop 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 -- 172.21.15.32:0/3326148876 shutdown_connections 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f53fc0a3e40 0x7f53fc0a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5410074d80 0x7f54101a6d60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5410107320 0x7f5410074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 --2- 172.21.15.32:0/3326148876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410102250 0x7f5410074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 -- 172.21.15.32:0/3326148876 >> 172.21.15.32:0/3326148876 conn(0x7f5410079400 msgr2=0x7f5410104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 -- 172.21.15.32:0/3326148876 shutdown_connections 2024-04-16T05:36:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:07.068+0000 7f54149a8640 1 -- 172.21.15.32:0/3326148876 wait complete. 2024-04-16T05:36:07.081 INFO:tasks.cephfs.filesystem:scrub status for tag:79b225b3-8e79-4ad9-b9a8-e8af1691fa06 - {'path': '/', 'tag': '79b225b3-8e79-4ad9-b9a8-e8af1691fa06', 'options': 'recursive,force'} 2024-04-16T05:36:07.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:07 smithi032 ceph-mon[32351]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:36:07.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:07 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:36:07.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:07 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:07.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:07 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:36:07.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:07 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:08.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:07 smithi134 ceph-mon[36153]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:36:08.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:07 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:36:08.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:07 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:08.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:07 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:36:08.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:07 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:07 smithi082 ceph-mon[36195]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:36:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:07 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:36:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:07 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:07 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:36:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:07 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/4072253607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee0106220 msgr2=0x7f2ee010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/4072253607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee0106220 0x7f2ee010ddf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009ca0 tx=0x7f2ed802c100 comp rx=0 tx=0).stop 2024-04-16T05:36:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/4072253607 shutdown_connections 2024-04-16T05:36:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/4072253607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee0106220 0x7f2ee010ddf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/4072253607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee0101b50 0x7f2ee0105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/4072253607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ee01011a0 0x7f2ee0101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/4072253607 >> 172.21.15.32:0/4072253607 conn(0x7f2ee0078080 msgr2=0x7f2ee0078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/4072253607 shutdown_connections 2024-04-16T05:36:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.310+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/4072253607 wait complete. 2024-04-16T05:36:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.311+0000 7f2ee7e7e640 1 Processor -- start 2024-04-16T05:36:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.311+0000 7f2ee7e7e640 1 -- start start 2024-04-16T05:36:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.311+0000 7f2ee7e7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ee01011a0 0x7f2ee01965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.311+0000 7f2ee7e7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee0106220 0x7f2ee0196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.311+0000 7f2ee7e7e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee019aeb0 0x7f2ee019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.311+0000 7f2ee7e7e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01101d0 con 0x7f2ee019aeb0 2024-04-16T05:36:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.311+0000 7f2ee7e7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0110050 con 0x7f2ee01011a0 2024-04-16T05:36:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.311+0000 7f2ee7e7e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0110350 con 0x7f2ee0106220 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee641b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee019aeb0 0x7f2ee019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee641b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee019aeb0 0x7f2ee019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46630/0 (socket says 172.21.15.32:46630) 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee641b640 1 -- 172.21.15.32:0/1524407055 learned_addr learned my addr 172.21.15.32:0/1524407055 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5419640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee0106220 0x7f2ee0196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ee01011a0 0x7f2ee01965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5419640 1 -- 172.21.15.32:0/1524407055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ee01011a0 msgr2=0x7f2ee01965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5419640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ee01011a0 0x7f2ee01965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5419640 1 -- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee019aeb0 msgr2=0x7f2ee019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5419640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee019aeb0 0x7f2ee019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5419640 1 -- 172.21.15.32:0/1524407055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee019b9a0 con 0x7f2ee0106220 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ee01011a0 0x7f2ee01965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee641b640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee019aeb0 0x7f2ee019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.312+0000 7f2ee5419640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee0106220 0x7f2ee0196b30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0007830 tx=0x7f2ed0007c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.313+0000 7f2ed6ffd640 1 -- 172.21.15.32:0/1524407055 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000f710 con 0x7f2ee0106220 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.313+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/1524407055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee019bc90 con 0x7f2ee0106220 2024-04-16T05:36:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.313+0000 7f2ed6ffd640 1 -- 172.21.15.32:0/1524407055 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed001a070 con 0x7f2ee0106220 2024-04-16T05:36:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.313+0000 7f2ed6ffd640 1 -- 172.21.15.32:0/1524407055 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0015d20 con 0x7f2ee0106220 2024-04-16T05:36:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.313+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/1524407055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee01a34d0 con 0x7f2ee0106220 2024-04-16T05:36:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.315+0000 7f2ed6ffd640 1 -- 172.21.15.32:0/1524407055 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f2ed0015470 con 0x7f2ee0106220 2024-04-16T05:36:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.316+0000 7f2ed6ffd640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2ecc0a3ee0 0x7f2ecc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.316+0000 7f2ed6ffd640 1 -- 172.21.15.32:0/1524407055 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2ed00d0df0 con 0x7f2ee0106220 2024-04-16T05:36:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.316+0000 7f2eeaa62740 1 Processor -- start 2024-04-16T05:36:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.316+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2ecc0a3ee0 0x7f2ecc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.316+0000 7f2eeaa62740 1 -- start start 2024-04-16T05:36:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1be800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5639cd11a7d0 con 0x5639cd1c7920 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5639cd0b7f70 con 0x5639cd1be420 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5639cd116790 con 0x5639cd1bed40 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee641b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee641b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46632/0 (socket says 172.21.15.32:46632) 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee641b640 1 -- 172.21.15.32:0/3383554082 learned_addr learned my addr 172.21.15.32:0/3383554082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2ecc0a3ee0 0x7f2ecc0a63a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009640 tx=0x7f2edc005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee5419640 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee641b640 1 -- 172.21.15.32:0/3383554082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 msgr2=0x5639cd1c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee641b640 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee641b640 1 -- 172.21.15.32:0/3383554082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 msgr2=0x5639cd1be800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee641b640 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1be800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee641b640 1 -- 172.21.15.32:0/3383554082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cce2d210 con 0x5639cd1c7920 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.317+0000 7f2ee5419640 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.318+0000 7f2ee641b640 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c9d10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005f80 tx=0x7f2ed80098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a4e5e2014f8172 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.318+0000 7f2ecbfff640 1 -- 172.21.15.32:0/3383554082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8043070 con 0x5639cd1c7920 2024-04-16T05:36:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.318+0000 7f2ecbfff640 1 -- 172.21.15.32:0/3383554082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8003970 con 0x5639cd1c7920 2024-04-16T05:36:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.318+0000 7f2ecbfff640 1 -- 172.21.15.32:0/3383554082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8039040 con 0x5639cd1c7920 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3383554082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 msgr2=0x5639cd1c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c9d10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005f80 tx=0x7f2ed80098c0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3383554082 shutdown_connections 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c9d10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/3383554082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1be800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3383554082 >> 172.21.15.32:0/3383554082 conn(0x5639cd1b8b50 msgr2=0x5639cd1b8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3383554082 shutdown_connections 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.319+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3383554082 wait complete. 2024-04-16T05:36:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 1 -- start start 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 1 Processor -- start 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1c5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5639cd1ca6b0 con 0x5639cd1bed40 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5639cd0a2e90 con 0x5639cd1c7920 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5639cd1ca830 con 0x5639cd1be420 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2eeaa62740 10 client.0 _ll_register_callbacks cb 0x5639cd0d6cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2ee5419640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2ee5419640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46634/0 (socket says 172.21.15.32:46634) 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.320+0000 7f2ee5419640 1 -- 172.21.15.32:0/1536428712 learned_addr learned my addr 172.21.15.32:0/1536428712 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ee641b640 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ee5419640 1 -- 172.21.15.32:0/1536428712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 msgr2=0x5639cd1c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ee5419640 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ee5419640 1 -- 172.21.15.32:0/1536428712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 msgr2=0x5639cd1c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ee5419640 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ee5419640 1 -- 172.21.15.32:0/1536428712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ccef9620 con 0x5639cd1bed40 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ee5419640 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1c4ee0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2ed009c790 tx=0x7f2ed009c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e92f6526ab1ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ec97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ec97fa640 1 -- 172.21.15.32:0/1536428712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00ce560 con 0x5639cd1bed40 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2ec97fa640 1 -- 172.21.15.32:0/1536428712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2ed009db30 con 0x5639cd1bed40 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.321+0000 7f2eeaa62740 1 -- 172.21.15.32:0/1536428712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed8003970 con 0x5639cd1bed40 2024-04-16T05:36:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.322+0000 7f2ec97fa640 1 -- 172.21.15.32:0/1536428712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ed00a5470 con 0x5639cd1bed40 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.322+0000 7f2eeaa62740 10 client.15999 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.322+0000 7f2eeaa62740 1 -- 172.21.15.32:0/1536428712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639cd1d6970 con 0x5639cd1bed40 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.322+0000 7f2ec97fa640 1 -- 172.21.15.32:0/1536428712 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8003970 con 0x5639cd1bed40 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.322+0000 7f2ec97fa640 1 -- 172.21.15.32:0/1536428712 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ed00b3030 con 0x5639cd1bed40 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.323+0000 7f2eeaa62740 10 client.15999 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.323+0000 7f2eeaa62740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.323+0000 7f2eeaa62740 20 client.15999 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.323+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5639cd1d8330 0x5639cd1da750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.323+0000 7f2eeaa62740 4 client.15999 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.323+0000 7f2eeaa62740 1 -- 172.21.15.32:0/1536428712 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639cd0ba380 con 0x5639cd1d8330 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.323+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5639cd1d8330 0x5639cd1da750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.324+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5639cd1d8330 0x5639cd1da750 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914447d66e81e80e server_cookie=1f7c4e2435eba600 in_seq=0 out_seq=0 2024-04-16T05:36:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.324+0000 7f2ec97fa640 10 client.15999 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.324+0000 7f2ec97fa640 1 -- 172.21.15.32:0/1536428712 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639cd0ba380 con 0x5639cd1d8330 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.324+0000 7f2ec97fa640 10 client.15999 handle_command_reply: tid=0 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.324+0000 7f2eeaa62740 1 client.15999 shutdown 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 -- 172.21.15.32:0/1536428712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 msgr2=0x5639cd1c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1c4ee0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2ed009c790 tx=0x7f2ed009c250 comp rx=0 tx=0).stop 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 -- 172.21.15.32:0/1536428712 shutdown_connections 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1c4ee0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5639cd1d8330 0x5639cd1da750 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/1536428712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 -- 172.21.15.32:0/1536428712 >> 172.21.15.32:0/1536428712 conn(0x5639cd1b8b50 msgr2=0x5639cd1af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2ec97fa640 0 client.15999 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 -- 172.21.15.32:0/1536428712 shutdown_connections 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 1 -- 172.21.15.32:0/1536428712 wait complete. 2024-04-16T05:36:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.325+0000 7f2eeaa62740 20 client.15999 trim_cache size 0 max 16384 2024-04-16T05:36:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.340+0000 7f2eeaa62740 1 Processor -- start 2024-04-16T05:36:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.340+0000 7f2eeaa62740 1 -- start start 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.340+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.340+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.340+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.340+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01af930 con 0x5639cd1bed40 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.340+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01ad200 con 0x5639cd1c7920 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.340+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01ab050 con 0x5639cd1be420 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee5419640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee5419640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1d3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46642/0 (socket says 172.21.15.32:46642) 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee5419640 1 -- 172.21.15.32:0/3492624631 learned_addr learned my addr 172.21.15.32:0/3492624631 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee641b640 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee641b640 1 -- 172.21.15.32:0/3492624631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 msgr2=0x5639cd1c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee641b640 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee641b640 1 -- 172.21.15.32:0/3492624631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 msgr2=0x5639cd1d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee641b640 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1d3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee641b640 1 -- 172.21.15.32:0/3492624631 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cd1dadd0 con 0x5639cd1c7920 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee5419640 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.341+0000 7f2ee641b640 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1e0060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005e30 tx=0x7f2ed8009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2e69968b72cdea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2ecb7fe640 1 -- 172.21.15.32:0/3492624631 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8039040 con 0x5639cd1c7920 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2ecb7fe640 1 -- 172.21.15.32:0/3492624631 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8032c90 con 0x5639cd1c7920 2024-04-16T05:36:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2ecb7fe640 1 -- 172.21.15.32:0/3492624631 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8043040 con 0x5639cd1c7920 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3492624631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 msgr2=0x5639cd1e0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1e0060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005e30 tx=0x7f2ed8009800 comp rx=0 tx=0).stop 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3492624631 shutdown_connections 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1c7920 0x5639cd1e0060 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1bed40 0x5639cd1d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/3492624631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1be420 0x5639cd1c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3492624631 >> 172.21.15.32:0/3492624631 conn(0x5639cd1b8b50 msgr2=0x5639cd1af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3492624631 shutdown_connections 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.342+0000 7f2eeaa62740 1 -- 172.21.15.32:0/3492624631 wait complete. 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.343+0000 7f2eeaa62740 1 -- start start 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.343+0000 7f2eeaa62740 1 Processor -- start 2024-04-16T05:36:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.343+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1c1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2eeaa62740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01ab6c0 con 0x5639cd1c7920 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01acb30 con 0x5639cd1be420 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2eeaa62740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01abd90 con 0x5639cd1bed40 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2eeaa62740 10 client.0 _ll_register_callbacks cb 0x7f2ed0005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee5c1a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee5c1a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1c1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.32:37060/0 (socket says 172.21.15.32:37060) 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee5c1a640 1 -- 172.21.15.32:0/2891045723 learned_addr learned my addr 172.21.15.32:0/2891045723 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee641b640 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee5419640 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee641b640 1 -- 172.21.15.32:0/2891045723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 msgr2=0x5639cd1c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee641b640 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee641b640 1 -- 172.21.15.32:0/2891045723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 msgr2=0x5639cd1c1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee641b640 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1c1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee641b640 1 -- 172.21.15.32:0/2891045723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639cd23d8d0 con 0x5639cd1c7920 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.344+0000 7f2ee5419640 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.345+0000 7f2ee641b640 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c2090 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8000f80 tx=0x7f2ed8032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7515827d59ffb11f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.345+0000 7f2ec97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.345+0000 7f2ec97fa640 1 -- 172.21.15.32:0/2891045723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8039040 con 0x5639cd1c7920 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.345+0000 7f2eeaa62740 1 -- 172.21.15.32:0/2891045723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed8032c90 con 0x5639cd1c7920 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.345+0000 7f2ec97fa640 1 -- 172.21.15.32:0/2891045723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2ed8003970 con 0x5639cd1c7920 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.345+0000 7f2ec97fa640 1 -- 172.21.15.32:0/2891045723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ed8043040 con 0x5639cd1c7920 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.345+0000 7f2eeaa62740 10 client.16005 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.345+0000 7f2eeaa62740 1 -- 172.21.15.32:0/2891045723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639cd23eae0 con 0x5639cd1c7920 2024-04-16T05:36:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.346+0000 7f2ec97fa640 1 -- 172.21.15.32:0/2891045723 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8032c90 con 0x5639cd1c7920 2024-04-16T05:36:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.346+0000 7f2ec97fa640 1 -- 172.21.15.32:0/2891045723 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ed8004000 con 0x5639cd1c7920 2024-04-16T05:36:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.346+0000 7f2eeaa62740 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.346+0000 7f2eeaa62740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.346+0000 7f2eeaa62740 20 client.16005 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.346+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5639cd23ed10 0x5639cd241100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.346+0000 7f2eeaa62740 4 client.16005 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:36:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.346+0000 7f2eeaa62740 1 -- 172.21.15.32:0/2891045723 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639cd1281f0 con 0x5639cd23ed10 2024-04-16T05:36:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.347+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5639cd23ed10 0x5639cd241100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.347+0000 7f2ec97fa640 10 client.16005 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.347+0000 7f2ee5c1a640 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5639cd23ed10 0x5639cd241100 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154ccb617446062f server_cookie=f2d765ffeb79473b in_seq=0 out_seq=0 2024-04-16T05:36:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.353+0000 7f2ec97fa640 1 -- 172.21.15.32:0/2891045723 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639cd1281f0 con 0x5639cd23ed10 2024-04-16T05:36:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.353+0000 7f2ec97fa640 10 client.16005 handle_command_reply: tid=0 2024-04-16T05:36:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.353+0000 7f2eeaa62740 1 client.16005 shutdown 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.353+0000 7f2eeaa62740 1 -- 172.21.15.32:0/2891045723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 msgr2=0x5639cd1c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c2090 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8000f80 tx=0x7f2ed8032b00 comp rx=0 tx=0).stop 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 -- 172.21.15.32:0/2891045723 shutdown_connections 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639cd1c7920 0x5639cd1c2090 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5639cd23ed10 0x5639cd241100 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639cd1bed40 0x5639cd1c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2ec97fa640 0 client.16005 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 --2- 172.21.15.32:0/2891045723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639cd1be420 0x5639cd1c1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 -- 172.21.15.32:0/2891045723 >> 172.21.15.32:0/2891045723 conn(0x5639cd1b8b50 msgr2=0x5639cd1c4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 -- 172.21.15.32:0/2891045723 shutdown_connections 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 1 -- 172.21.15.32:0/2891045723 wait complete. 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.354+0000 7f2eeaa62740 20 client.16005 trim_cache size 0 max 16384 2024-04-16T05:36:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2024-04-16T05:36:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2024-04-16T05:36:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2024-04-16T05:36:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2024-04-16T05:36:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2ecc0a3ee0 msgr2=0x7f2ecc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2ecc0a3ee0 0x7f2ecc0a63a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009640 tx=0x7f2edc005f90 comp rx=0 tx=0).stop 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/1524407055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee0106220 msgr2=0x7f2ee0196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee0106220 0x7f2ee0196b30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0007830 tx=0x7f2ed0007c30 comp rx=0 tx=0).stop 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/1524407055 shutdown_connections 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2ecc0a3ee0 0x7f2ecc0a63a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee019aeb0 0x7f2ee019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee0106220 0x7f2ee0196b30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 --2- 172.21.15.32:0/1524407055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ee01011a0 0x7f2ee01965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/1524407055 >> 172.21.15.32:0/1524407055 conn(0x7f2ee0078080 msgr2=0x7f2ee00feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/1524407055 shutdown_connections 2024-04-16T05:36:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:09.355+0000 7f2ee7e7e640 1 -- 172.21.15.32:0/1524407055 wait complete. 2024-04-16T05:36:09.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:09 smithi032 ceph-mon[32351]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-16T05:36:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:09 smithi134 ceph-mon[36153]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-16T05:36:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:09 smithi082 ceph-mon[36195]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-16T05:36:10.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:10 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:36:11.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:10 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:36:11.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:10 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:36:11.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:11 smithi032 ceph-mon[32351]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:11 smithi134 ceph-mon[36153]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:11 smithi082 ceph-mon[36195]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:36:13.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:13 smithi032 ceph-mon[32351]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:36:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:13 smithi134 ceph-mon[36153]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:36:14.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:13 smithi082 ceph-mon[36195]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:36:15.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:15 smithi032 ceph-mon[32351]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-16T05:36:16.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:15 smithi134 ceph-mon[36153]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-16T05:36:16.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:15 smithi082 ceph-mon[36195]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-16T05:36:17.953 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:17 smithi082 ceph-mon[36195]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:17.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:17 smithi032 ceph-mon[32351]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:18.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:17 smithi134 ceph-mon[36153]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:19.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:19 smithi032 ceph-mon[32351]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-16T05:36:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:19 smithi134 ceph-mon[36153]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-16T05:36:20.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:19 smithi082 ceph-mon[36195]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-16T05:36:21.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:21 smithi032 ceph-mon[32351]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-16T05:36:22.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:21 smithi134 ceph-mon[36153]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-16T05:36:22.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:21 smithi082 ceph-mon[36195]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 -- 172.21.15.32:0/1930998820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106220 msgr2=0x7f459010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 --2- 172.21.15.32:0/1930998820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106220 0x7f459010ddf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4588009ca0 tx=0x7f458802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 -- 172.21.15.32:0/1930998820 shutdown_connections 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 --2- 172.21.15.32:0/1930998820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106220 0x7f459010ddf0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 --2- 172.21.15.32:0/1930998820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4590101b50 0x7f4590105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 --2- 172.21.15.32:0/1930998820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45901011a0 0x7f4590101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 -- 172.21.15.32:0/1930998820 >> 172.21.15.32:0/1930998820 conn(0x7f4590078080 msgr2=0x7f4590078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 -- 172.21.15.32:0/1930998820 shutdown_connections 2024-04-16T05:36:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.491+0000 7f4596bb5640 1 -- 172.21.15.32:0/1930998820 wait complete. 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4596bb5640 1 Processor -- start 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4596bb5640 1 -- start start 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4596bb5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45901011a0 0x7f4590199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4596bb5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4590106220 0x7f4590199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4596bb5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459019a0e0 0x7f45901a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4596bb5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45901112c0 con 0x7f459019a0e0 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4596bb5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4590111140 con 0x7f4590106220 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4594951640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45901011a0 0x7f4590199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4596bb5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4590111440 con 0x7f45901011a0 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4594951640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45901011a0 0x7f4590199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:35012/0 (socket says 172.21.15.32:35012) 2024-04-16T05:36:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4594951640 1 -- 172.21.15.32:0/22971716 learned_addr learned my addr 172.21.15.32:0/22971716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f458ffff640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4590106220 0x7f4590199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.492+0000 7f4595152640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459019a0e0 0x7f45901a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f4594951640 1 -- 172.21.15.32:0/22971716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4590106220 msgr2=0x7f4590199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f4594951640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4590106220 0x7f4590199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f4594951640 1 -- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459019a0e0 msgr2=0x7f45901a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f4594951640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459019a0e0 0x7f45901a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f4594951640 1 -- 172.21.15.32:0/22971716 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45901a4e60 con 0x7f45901011a0 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f4595152640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459019a0e0 0x7f45901a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f4594951640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45901011a0 0x7f4590199660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f458400e9f0 tx=0x7f458400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f458dffb640 1 -- 172.21.15.32:0/22971716 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458400ccc0 con 0x7f45901011a0 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f4596bb5640 1 -- 172.21.15.32:0/22971716 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45901974b0 con 0x7f45901011a0 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.493+0000 7f458dffb640 1 -- 172.21.15.32:0/22971716 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4584010910 con 0x7f45901011a0 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.494+0000 7f458dffb640 1 -- 172.21.15.32:0/22971716 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840188a0 con 0x7f45901011a0 2024-04-16T05:36:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.494+0000 7f4596bb5640 1 -- 172.21.15.32:0/22971716 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4590197a50 con 0x7f45901011a0 2024-04-16T05:36:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.497+0000 7f458dffb640 1 -- 172.21.15.32:0/22971716 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f4584010ac0 con 0x7f45901011a0 2024-04-16T05:36:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.497+0000 7f459979c740 1 Processor -- start 2024-04-16T05:36:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f459979c740 1 -- start start 2024-04-16T05:36:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f459979c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2af420 0x5575bb2af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f459979c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f459979c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2b8920 0x5575bb2bad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f459979c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5575bb20b7d0 con 0x5575bb2b8920 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f459979c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5575bb1a8f70 con 0x5575bb2afd40 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f459979c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5575bb207790 con 0x5575bb2af420 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f458dffb640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f45800a3f60 0x7f45800a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f458ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2b8920 0x5575bb2bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f4595152640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2af420 0x5575bb2af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f4595152640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2af420 0x5575bb2af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:35014/0 (socket says 172.21.15.32:35014) 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f4595152640 1 -- 172.21.15.32:0/3277543174 learned_addr learned my addr 172.21.15.32:0/3277543174 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f4594951640 1 --2- 172.21.15.32:0/3277543174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.498+0000 7f4595152640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f45800a3f60 0x7f45800a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f458dffb640 1 -- 172.21.15.32:0/22971716 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f45840d1230 con 0x7f45901011a0 2024-04-16T05:36:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f458ffff640 1 -- 172.21.15.32:0/3277543174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2af420 msgr2=0x5575bb2af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f458ffff640 1 --2- 172.21.15.32:0/3277543174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2af420 0x5575bb2af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f458ffff640 1 -- 172.21.15.32:0/3277543174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 msgr2=0x5575bb2b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f458ffff640 1 --2- 172.21.15.32:0/3277543174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2b83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f458ffff640 1 -- 172.21.15.32:0/3277543174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575baf1e210 con 0x5575bb2b8920 2024-04-16T05:36:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f4595152640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f45800a3f60 0x7f45800a6420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4588032530 tx=0x7f4588005f70 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f458ffff640 1 --2- 172.21.15.32:0/3277543174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2b8920 0x5575bb2bad10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f459006b7e0 tx=0x7f457c00e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660f27fdc1d9f8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.499+0000 7f457affd640 1 -- 172.21.15.32:0/3277543174 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c005020 con 0x5575bb2b8920 2024-04-16T05:36:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.500+0000 7f457affd640 1 -- 172.21.15.32:0/3277543174 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c0056a0 con 0x5575bb2b8920 2024-04-16T05:36:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.500+0000 7f457affd640 1 -- 172.21.15.32:0/3277543174 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c017680 con 0x5575bb2b8920 2024-04-16T05:36:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.500+0000 7f459979c740 1 -- 172.21.15.32:0/3277543174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2b8920 msgr2=0x5575bb2bad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.500+0000 7f459979c740 1 --2- 172.21.15.32:0/3277543174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2b8920 0x5575bb2bad10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f459006b7e0 tx=0x7f457c00e6e0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.500+0000 7f459979c740 1 -- 172.21.15.32:0/3277543174 shutdown_connections 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.500+0000 7f459979c740 1 --2- 172.21.15.32:0/3277543174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2b8920 0x5575bb2bad10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.501+0000 7f459979c740 1 --2- 172.21.15.32:0/3277543174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2b83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.501+0000 7f459979c740 1 --2- 172.21.15.32:0/3277543174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2af420 0x5575bb2af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.501+0000 7f459979c740 1 -- 172.21.15.32:0/3277543174 >> 172.21.15.32:0/3277543174 conn(0x5575bb2a9b50 msgr2=0x5575bb2a9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.501+0000 7f459979c740 1 -- 172.21.15.32:0/3277543174 shutdown_connections 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.501+0000 7f459979c740 1 -- 172.21.15.32:0/3277543174 wait complete. 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.501+0000 7f459979c740 1 -- start start 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.501+0000 7f459979c740 1 Processor -- start 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f459979c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f459979c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f459979c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f459979c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5575bb2bb710 con 0x5575bb2af420 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f459979c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5575bb193e90 con 0x5575bb2afd40 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f459979c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5575bb2bb890 con 0x5575bb2b6540 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f459979c740 10 client.0 _ll_register_callbacks cb 0x5575bb1c7cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f4595152640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f458ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f4595152640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:35028/0 (socket says 172.21.15.32:35028) 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f4595152640 1 -- 172.21.15.32:0/2534081610 learned_addr learned my addr 172.21.15.32:0/2534081610 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f4594951640 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f4595152640 1 -- 172.21.15.32:0/2534081610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 msgr2=0x5575bb2b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f4595152640 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.502+0000 7f4595152640 1 -- 172.21.15.32:0/2534081610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 msgr2=0x5575bb2b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.503+0000 7f4595152640 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.503+0000 7f4595152640 1 -- 172.21.15.32:0/2534081610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575bafea620 con 0x5575bb2b6540 2024-04-16T05:36:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.503+0000 7f4594951640 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.503+0000 7f4595152640 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2c4bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4588004af0 tx=0x7f458802e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09934bdd248e933 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.503+0000 7f456ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-16T05:36:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.503+0000 7f456ffff640 1 -- 172.21.15.32:0/2534081610 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458804f030 con 0x5575bb2b6540 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.503+0000 7f456ffff640 1 -- 172.21.15.32:0/2534081610 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f45880329b0 con 0x5575bb2b6540 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.503+0000 7f459979c740 1 -- 172.21.15.32:0/2534081610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f457c0056a0 con 0x5575bb2b6540 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.504+0000 7f456ffff640 1 -- 172.21.15.32:0/2534081610 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f458805c050 con 0x5575bb2b6540 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.504+0000 7f459979c740 10 client.25471 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.504+0000 7f459979c740 1 -- 172.21.15.32:0/2534081610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575bb2c8040 con 0x5575bb2b6540 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.504+0000 7f456ffff640 1 -- 172.21.15.32:0/2534081610 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c0056a0 con 0x5575bb2b6540 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.504+0000 7f456ffff640 1 -- 172.21.15.32:0/2534081610 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f458804a040 con 0x5575bb2b6540 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.504+0000 7f459979c740 10 client.25471 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.504+0000 7f459979c740 10 client.25471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.504+0000 7f459979c740 20 client.25471 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.505+0000 7f459979c740 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5575bb2c9500 0x5575bb2cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.505+0000 7f459979c740 4 client.25471 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.505+0000 7f459979c740 1 -- 172.21.15.32:0/2534081610 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575bb1ab380 con 0x5575bb2c9500 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.505+0000 7f4594951640 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5575bb2c9500 0x5575bb2cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.505+0000 7f4594951640 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5575bb2c9500 0x5575bb2cb930 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e2bf6b2582c47c server_cookie=c68db10966906567 in_seq=0 out_seq=0 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.505+0000 7f456ffff640 10 client.25471 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f456ffff640 1 -- 172.21.15.32:0/2534081610 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575bb1ab380 con 0x5575bb2c9500 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f456ffff640 10 client.25471 handle_command_reply: tid=0 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 client.25471 shutdown 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 -- 172.21.15.32:0/2534081610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 msgr2=0x5575bb2c4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2c4bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4588004af0 tx=0x7f458802e040 comp rx=0 tx=0).stop 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 -- 172.21.15.32:0/2534081610 shutdown_connections 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5575bb2c9500 0x5575bb2cb930 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f456ffff640 0 client.25471 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2c4bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 --2- 172.21.15.32:0/2534081610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.506+0000 7f459979c740 1 -- 172.21.15.32:0/2534081610 >> 172.21.15.32:0/2534081610 conn(0x5575bb2a9b50 msgr2=0x5575bb2ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.507+0000 7f459979c740 1 -- 172.21.15.32:0/2534081610 shutdown_connections 2024-04-16T05:36:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.507+0000 7f459979c740 1 -- 172.21.15.32:0/2534081610 wait complete. 2024-04-16T05:36:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.507+0000 7f459979c740 20 client.25471 trim_cache size 0 max 16384 2024-04-16T05:36:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f459979c740 1 Processor -- start 2024-04-16T05:36:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f459979c740 1 -- start start 2024-04-16T05:36:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f459979c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f459979c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2afd40 0x5575bb2d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f459979c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2b6540 0x5575bb2d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f459979c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45901af8e0 con 0x5575bb2af420 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f459979c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f45901ad1b0 con 0x5575bb2b6540 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f459979c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45901ab000 con 0x5575bb2afd40 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f4594951640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f4594951640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57440/0 (socket says 172.21.15.32:57440) 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.536+0000 7f4594951640 1 -- 172.21.15.32:0/270030113 learned_addr learned my addr 172.21.15.32:0/270030113 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f458ffff640 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2afd40 0x5575bb2d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f4595152640 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2b6540 0x5575bb2d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f4594951640 1 -- 172.21.15.32:0/270030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2afd40 msgr2=0x5575bb2d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f4594951640 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2afd40 0x5575bb2d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f4594951640 1 -- 172.21.15.32:0/270030113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2b6540 msgr2=0x5575bb2d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f4594951640 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2b6540 0x5575bb2d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f4594951640 1 -- 172.21.15.32:0/270030113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575bb2b4010 con 0x5575bb2af420 2024-04-16T05:36:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f4595152640 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2b6540 0x5575bb2d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f4594951640 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2c4b40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4584099560 tx=0x7f45840bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1db78a8b832e46d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f457a7fc640 1 -- 172.21.15.32:0/270030113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840b5070 con 0x5575bb2af420 2024-04-16T05:36:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f457a7fc640 1 -- 172.21.15.32:0/270030113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45840a0f90 con 0x5575bb2af420 2024-04-16T05:36:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.537+0000 7f457a7fc640 1 -- 172.21.15.32:0/270030113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458409c030 con 0x5575bb2af420 2024-04-16T05:36:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 -- 172.21.15.32:0/270030113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 msgr2=0x5575bb2c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2c4b40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4584099560 tx=0x7f45840bd040 comp rx=0 tx=0).stop 2024-04-16T05:36:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 -- 172.21.15.32:0/270030113 shutdown_connections 2024-04-16T05:36:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2b6540 0x5575bb2d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2afd40 0x5575bb2d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 --2- 172.21.15.32:0/270030113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2af420 0x5575bb2c4b40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 -- 172.21.15.32:0/270030113 >> 172.21.15.32:0/270030113 conn(0x5575bb2a9b50 msgr2=0x5575bb2b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 -- 172.21.15.32:0/270030113 shutdown_connections 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.538+0000 7f459979c740 1 -- 172.21.15.32:0/270030113 wait complete. 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.539+0000 7f459979c740 1 -- start start 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.539+0000 7f459979c740 1 Processor -- start 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.539+0000 7f459979c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.539+0000 7f459979c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f459979c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2c9500 0x5575bb2ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f459979c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45901ab670 con 0x5575bb2c9500 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f459979c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f45901acae0 con 0x5575bb2afd40 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f459979c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45901abd40 con 0x5575bb2b6540 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f459979c740 10 client.0 _ll_register_callbacks cb 0x7f4584005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2c9500 0x5575bb2ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2c9500 0x5575bb2ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57456/0 (socket says 172.21.15.32:57456) 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 -- 172.21.15.32:0/1627147549 learned_addr learned my addr 172.21.15.32:0/1627147549 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4594951640 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f458ffff640 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 -- 172.21.15.32:0/1627147549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 msgr2=0x5575bb2ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 -- 172.21.15.32:0/1627147549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 msgr2=0x5575bb2d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 -- 172.21.15.32:0/1627147549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575bb2cf5b0 con 0x5575bb2c9500 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f4595152640 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2c9500 0x5575bb2ced80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f458802c700 tx=0x7f4588032ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8511b5d48769260 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.540+0000 7f456ffff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.541+0000 7f456ffff640 1 -- 172.21.15.32:0/1627147549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458804f070 con 0x5575bb2c9500 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.541+0000 7f456ffff640 1 -- 172.21.15.32:0/1627147549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4588003d40 con 0x5575bb2c9500 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.541+0000 7f459979c740 1 -- 172.21.15.32:0/1627147549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45840a0f90 con 0x5575bb2c9500 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.541+0000 7f456ffff640 1 -- 172.21.15.32:0/1627147549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4588039040 con 0x5575bb2c9500 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.541+0000 7f459979c740 10 client.16023 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.541+0000 7f459979c740 1 -- 172.21.15.32:0/1627147549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575bb32faa0 con 0x5575bb2c9500 2024-04-16T05:36:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.541+0000 7f456ffff640 1 -- 172.21.15.32:0/1627147549 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45840a0f90 con 0x5575bb2c9500 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.541+0000 7f456ffff640 1 -- 172.21.15.32:0/1627147549 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f458804a040 con 0x5575bb2c9500 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f459979c740 10 client.16023 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f459979c740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f459979c740 20 client.16023 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f459979c740 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5575bb330220 0x5575bb332640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f459979c740 4 client.16023 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f459979c740 1 -- 172.21.15.32:0/1627147549 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575bb2191f0 con 0x5575bb330220 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f4594951640 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5575bb330220 0x5575bb332640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f456ffff640 10 client.16023 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.542+0000 7f4594951640 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5575bb330220 0x5575bb332640 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29436a026cbc99d9 server_cookie=187582c7b3e4808e in_seq=0 out_seq=0 2024-04-16T05:36:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f456ffff640 1 -- 172.21.15.32:0/1627147549 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575bb2191f0 con 0x5575bb330220 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f456ffff640 10 client.16023 handle_command_reply: tid=0 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 client.16023 shutdown 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 -- 172.21.15.32:0/1627147549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2c9500 msgr2=0x5575bb2ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2c9500 0x5575bb2ced80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f458802c700 tx=0x7f4588032ab0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 -- 172.21.15.32:0/1627147549 shutdown_connections 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5575bb2c9500 0x5575bb2ced80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5575bb330220 0x5575bb332640 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575bb2b6540 0x5575bb2ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f456ffff640 0 client.16023 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 --2- 172.21.15.32:0/1627147549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bb2afd40 0x5575bb2d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.546+0000 7f459979c740 1 -- 172.21.15.32:0/1627147549 >> 172.21.15.32:0/1627147549 conn(0x5575bb2a9b50 msgr2=0x5575bb2b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.547+0000 7f459979c740 1 -- 172.21.15.32:0/1627147549 shutdown_connections 2024-04-16T05:36:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.547+0000 7f459979c740 1 -- 172.21.15.32:0/1627147549 wait complete. 2024-04-16T05:36:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.547+0000 7f459979c740 20 client.16023 trim_cache size 0 max 16384 2024-04-16T05:36:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2024-04-16T05:36:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2024-04-16T05:36:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2024-04-16T05:36:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 -- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f45800a3f60 msgr2=0x7f45800a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f45800a3f60 0x7f45800a6420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4588032530 tx=0x7f4588005f70 comp rx=0 tx=0).stop 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 -- 172.21.15.32:0/22971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45901011a0 msgr2=0x7f4590199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45901011a0 0x7f4590199660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f458400e9f0 tx=0x7f458400edf0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 -- 172.21.15.32:0/22971716 shutdown_connections 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f45800a3f60 0x7f45800a6420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459019a0e0 0x7f45901a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4590106220 0x7f4590199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 --2- 172.21.15.32:0/22971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45901011a0 0x7f4590199660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 -- 172.21.15.32:0/22971716 >> 172.21.15.32:0/22971716 conn(0x7f4590078080 msgr2=0x7f45900fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 -- 172.21.15.32:0/22971716 shutdown_connections 2024-04-16T05:36:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2024-04-16T05:36:22.548+0000 7f4596bb5640 1 -- 172.21.15.32:0/22971716 wait complete. 2024-04-16T05:36:23.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:23 smithi032 ceph-mon[32351]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-16T05:36:24.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:23 smithi134 ceph-mon[36153]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-16T05:36:24.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:23 smithi082 ceph-mon[36195]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-16T05:36:25.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:25 smithi032 ceph-mon[32351]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-16T05:36:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:25 smithi134 ceph-mon[36153]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-16T05:36:26.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:25 smithi082 ceph-mon[36195]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-16T05:36:27.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:27 smithi032 ceph-mon[32351]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:36:28.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:27 smithi134 ceph-mon[36153]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:36:28.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:27 smithi082 ceph-mon[36195]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:36:29.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:29 smithi032 ceph-mon[32351]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:36:29.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:29 smithi032 ceph-mon[32351]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T05:36:30.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:29 smithi134 ceph-mon[36153]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:36:30.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:29 smithi134 ceph-mon[36153]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T05:36:30.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:29 smithi082 ceph-mon[36195]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:36:30.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:29 smithi082 ceph-mon[36195]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T05:36:31.754 INFO:tasks.workunit.client.0.smithi032.stderr:Updating files: 94% (11594/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-16T05:36:31.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:31 smithi032 ceph-mon[32351]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:32.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:31 smithi134 ceph-mon[36153]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:31 smithi082 ceph-mon[36195]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:32.564 INFO:tasks.workunit.client.0.smithi032.stderr:Note: switching to 'a87fcff6340baeedd43af126e793f3241b6400ea'. 2024-04-16T05:36:32.564 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-04-16T05:36:32.564 INFO:tasks.workunit.client.0.smithi032.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr:state without impacting any branches by switching back to a branch. 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr: git switch -c 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr:Or undo this operation with: 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr: git switch - 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-16T05:36:32.565 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-04-16T05:36:32.566 INFO:tasks.workunit.client.0.smithi032.stderr:HEAD is now at a87fcff6340 qa: enable dynamic debug for dbench workunit 2024-04-16T05:36:32.571 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-16T05:36:32.632 INFO:tasks.workunit.client.0.smithi032.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-16T05:36:32.634 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T05:36:32.634 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-16T05:36:33.376 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-16T05:36:33.429 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-16T05:36:33.470 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T05:36:33.472 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T05:36:33.472 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-16T05:36:33.511 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T05:36:33.514 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:36:33.514 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-16T05:36:33.570 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-16T05:36:33.571 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-16T05:36:33.572 DEBUG:teuthology.orchestra.run.smithi032:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a87fcff6340baeedd43af126e793f3241b6400ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-16T05:36:33.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:36:33.632 INFO:tasks.workunit.client.0.smithi032.stderr:+ echo -n 'module ceph +p' 2024-04-16T05:36:33.632 INFO:tasks.workunit.client.0.smithi032.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh: line 4: /sys/kernel/debug/dynamic_debug/control: Permission denied 2024-04-16T05:36:33.632 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-16T05:36:33.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-16T05:36:33.923 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a87fcff6340baeedd43af126e793f3241b6400ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-16T05:36:34.139 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=dc6265b9adf940ae882a92366928e7bb Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a87fcff6340baeedd43af126e793f3241b6400ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-16T05:36:34.142 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-16T05:36:34.151 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:36:34.156 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:33 smithi082 ceph-mon[36195]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:36:34.156 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:33 smithi134 ceph-mon[36153]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:36:34.156 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:33 smithi032 ceph-mon[32351]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-16T05:36:34.164 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-16T05:36:34.189 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-16T05:36:34.189 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T05:36:34.189 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:36:34.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-16T05:36:34.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-16T05:36:35.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:35 smithi032 ceph-mon[32351]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:35 smithi134 ceph-mon[36153]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:36.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:35 smithi082 ceph-mon[36195]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:37.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:36:37.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:36:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 -- 172.21.15.32:0/4020970487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006afc0 msgr2=0x7fe76010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 --2- 172.21.15.32:0/4020970487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006afc0 0x7fe76010eea0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe758009cd0 tx=0x7fe75802c180 comp rx=0 tx=0).stop 2024-04-16T05:36:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 -- 172.21.15.32:0/4020970487 shutdown_connections 2024-04-16T05:36:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 --2- 172.21.15.32:0/4020970487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006afc0 0x7fe76010eea0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 --2- 172.21.15.32:0/4020970487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe76006a600 0x7fe76006aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 --2- 172.21.15.32:0/4020970487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe760108200 0x7fe7601085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 -- 172.21.15.32:0/4020970487 >> 172.21.15.32:0/4020970487 conn(0x7fe760076d00 msgr2=0x7fe760077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 -- 172.21.15.32:0/4020970487 shutdown_connections 2024-04-16T05:36:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.204+0000 7fe7681c8640 1 -- 172.21.15.32:0/4020970487 wait complete. 2024-04-16T05:36:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe7681c8640 1 Processor -- start 2024-04-16T05:36:37.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe7681c8640 1 -- start start 2024-04-16T05:36:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe7681c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006a600 0x7fe76019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe7681c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe76006afc0 0x7fe76019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe7681c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe760108200 0x7fe7601985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe7681c8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601123b0 con 0x7fe76006a600 2024-04-16T05:36:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe7681c8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe760112230 con 0x7fe76006afc0 2024-04-16T05:36:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe7681c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe760112530 con 0x7fe760108200 2024-04-16T05:36:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe765f64640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006a600 0x7fe76019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe765f64640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006a600 0x7fe76019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60636/0 (socket says 172.21.15.32:60636) 2024-04-16T05:36:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe765f64640 1 -- 172.21.15.32:0/2264942682 learned_addr learned my addr 172.21.15.32:0/2264942682 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe765763640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe76006afc0 0x7fe76019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe766765640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe760108200 0x7fe7601985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe765f64640 1 -- 172.21.15.32:0/2264942682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe760108200 msgr2=0x7fe7601985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe765f64640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe760108200 0x7fe7601985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe765f64640 1 -- 172.21.15.32:0/2264942682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe76006afc0 msgr2=0x7fe76019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.205+0000 7fe765f64640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe76006afc0 0x7fe76019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.206+0000 7fe765f64640 1 -- 172.21.15.32:0/2264942682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe760198d30 con 0x7fe76006a600 2024-04-16T05:36:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.206+0000 7fe765763640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe76006afc0 0x7fe76019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.206+0000 7fe765f64640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006a600 0x7fe76019e400 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c970 tx=0x7fe75c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.206+0000 7fe756ffd640 1 -- 172.21.15.32:0/2264942682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c007c10 con 0x7fe76006a600 2024-04-16T05:36:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.206+0000 7fe756ffd640 1 -- 172.21.15.32:0/2264942682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75c007db0 con 0x7fe76006a600 2024-04-16T05:36:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.206+0000 7fe7681c8640 1 -- 172.21.15.32:0/2264942682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe760199020 con 0x7fe76006a600 2024-04-16T05:36:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.206+0000 7fe756ffd640 1 -- 172.21.15.32:0/2264942682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00f940 con 0x7fe76006a600 2024-04-16T05:36:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.206+0000 7fe7681c8640 1 -- 172.21.15.32:0/2264942682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7601a52c0 con 0x7fe76006a600 2024-04-16T05:36:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.208+0000 7fe756ffd640 1 -- 172.21.15.32:0/2264942682 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe75c00fae0 con 0x7fe76006a600 2024-04-16T05:36:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe756ffd640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe74c0a3da0 0x7fe74c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe76adb3740 1 Processor -- start 2024-04-16T05:36:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe756ffd640 1 -- 172.21.15.32:0/2264942682 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fe75c0d0f70 con 0x7fe76006a600 2024-04-16T05:36:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe765763640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe74c0a3da0 0x7fe74c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe76adb3740 1 -- start start 2024-04-16T05:36:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea270b00 0x555dea270ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 0x555dea280b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe76adb3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555dea1d6180 con 0x555dea271420 2024-04-16T05:36:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe76adb3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555dea173360 con 0x555dea27e760 2024-04-16T05:36:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.209+0000 7fe76adb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555dea1d2140 con 0x555dea270b00 2024-04-16T05:36:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60646/0 (socket says 172.21.15.32:60646) 2024-04-16T05:36:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 -- 172.21.15.32:0/3466597656 learned_addr learned my addr 172.21.15.32:0/3466597656 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765f64640 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea270b00 0x555dea270ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe74c0a3da0 0x7fe74c0a6260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe760107600 tx=0x7fe750005e50 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe766765640 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 0x555dea280b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 -- 172.21.15.32:0/3466597656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea270b00 msgr2=0x555dea270ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea270b00 0x555dea270ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 -- 172.21.15.32:0/3466597656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 msgr2=0x555dea280b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 0x555dea280b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 -- 172.21.15.32:0/3466597656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de9fb27d0 con 0x555dea271420 2024-04-16T05:36:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe766765640 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 0x555dea280b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe765763640 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27e220 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe75000ef20 tx=0x7fe750011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e529e17c2c815f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe74bfff640 1 -- 172.21.15.32:0/3466597656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75001e640 con 0x555dea271420 2024-04-16T05:36:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe74bfff640 1 -- 172.21.15.32:0/3466597656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe750023070 con 0x555dea271420 2024-04-16T05:36:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.210+0000 7fe74bfff640 1 -- 172.21.15.32:0/3466597656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750010bc0 con 0x555dea271420 2024-04-16T05:36:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 -- 172.21.15.32:0/3466597656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 msgr2=0x555dea27e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27e220 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe75000ef20 tx=0x7fe750011000 comp rx=0 tx=0).stop 2024-04-16T05:36:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 -- 172.21.15.32:0/3466597656 shutdown_connections 2024-04-16T05:36:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 0x555dea280b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27e220 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3466597656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea270b00 0x555dea270ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 -- 172.21.15.32:0/3466597656 >> 172.21.15.32:0/3466597656 conn(0x555dea26b580 msgr2=0x555dea26b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 -- 172.21.15.32:0/3466597656 shutdown_connections 2024-04-16T05:36:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 -- 172.21.15.32:0/3466597656 wait complete. 2024-04-16T05:36:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.211+0000 7fe76adb3740 1 -- start start 2024-04-16T05:36:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe76adb3740 1 Processor -- start 2024-04-16T05:36:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea27b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 0x555dea27c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe76adb3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555dea281540 con 0x555dea271420 2024-04-16T05:36:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe76adb3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555dea18f2c0 con 0x555dea270b00 2024-04-16T05:36:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe76adb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555dea2816c0 con 0x555dea27e760 2024-04-16T05:36:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe76adb3740 10 client.0 _ll_register_callbacks cb 0x555dea176090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765763640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765763640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60658/0 (socket says 172.21.15.32:60658) 2024-04-16T05:36:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765763640 1 -- 172.21.15.32:0/1583950414 learned_addr learned my addr 172.21.15.32:0/1583950414 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765f64640 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea27b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe766765640 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 0x555dea27c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765763640 1 -- 172.21.15.32:0/1583950414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 msgr2=0x555dea27c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765763640 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 0x555dea27c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765763640 1 -- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 msgr2=0x555dea27b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765763640 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea27b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765763640 1 -- 172.21.15.32:0/1583950414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de9ee83f0 con 0x555dea271420 2024-04-16T05:36:37.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.212+0000 7fe765f64640 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea27b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe765763640 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27bc30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe750005820 tx=0x7fe750010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f6e325c39b7bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe7497fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe7497fa640 1 -- 172.21.15.32:0/1583950414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750005250 con 0x555dea271420 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe7497fa640 1 -- 172.21.15.32:0/1583950414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fe7500053f0 con 0x555dea271420 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 1 -- 172.21.15.32:0/1583950414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe750023070 con 0x555dea271420 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe7497fa640 1 -- 172.21.15.32:0/1583950414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe75003a050 con 0x555dea271420 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 10 client.16035 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 1 -- 172.21.15.32:0/1583950414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555de9ea80f0 con 0x555dea271420 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe7497fa640 1 -- 172.21.15.32:0/1583950414 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe750023070 con 0x555dea271420 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe7497fa640 1 -- 172.21.15.32:0/1583950414 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe75000f040 con 0x555dea271420 2024-04-16T05:36:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 10 client.16035 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 20 client.16035 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555dea28ea70 0x555dea290e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 4 client.16035 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.213+0000 7fe76adb3740 1 -- 172.21.15.32:0/1583950414 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dea1e3ba0 con 0x555dea28ea70 2024-04-16T05:36:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.214+0000 7fe765f64640 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555dea28ea70 0x555dea290e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.214+0000 7fe7497fa640 10 client.16035 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.214+0000 7fe765f64640 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555dea28ea70 0x555dea290e60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866a575d45742d4d server_cookie=82d9526807ff5bd3 in_seq=0 out_seq=0 2024-04-16T05:36:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe7497fa640 1 -- 172.21.15.32:0/1583950414 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555dea1e3ba0 con 0x555dea28ea70 2024-04-16T05:36:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe7497fa640 10 client.16035 handle_command_reply: tid=0 2024-04-16T05:36:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 client.16035 shutdown 2024-04-16T05:36:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 -- 172.21.15.32:0/1583950414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 msgr2=0x555dea27bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27bc30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe750005820 tx=0x7fe750010e60 comp rx=0 tx=0).stop 2024-04-16T05:36:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 -- 172.21.15.32:0/1583950414 shutdown_connections 2024-04-16T05:36:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555dea28ea70 0x555dea290e60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 0x555dea27c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea27bc30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe7497fa640 0 client.16035 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:37.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 --2- 172.21.15.32:0/1583950414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea27b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 -- 172.21.15.32:0/1583950414 >> 172.21.15.32:0/1583950414 conn(0x555dea26b580 msgr2=0x555dea26d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 -- 172.21.15.32:0/1583950414 shutdown_connections 2024-04-16T05:36:37.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 1 -- 172.21.15.32:0/1583950414 wait complete. 2024-04-16T05:36:37.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.215+0000 7fe76adb3740 20 client.16035 trim_cache size 0 max 16384 2024-04-16T05:36:37.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe76adb3740 1 Processor -- start 2024-04-16T05:36:37.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe76adb3740 1 -- start start 2024-04-16T05:36:37.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea270b00 0x555dea27de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea271420 0x555dea28a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 0x555dea26b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe76adb3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601af990 con 0x555dea270b00 2024-04-16T05:36:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe76adb3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601ad260 con 0x555dea27e760 2024-04-16T05:36:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe76adb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601ab0b0 con 0x555dea271420 2024-04-16T05:36:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea270b00 0x555dea27de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea270b00 0x555dea27de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60664/0 (socket says 172.21.15.32:60664) 2024-04-16T05:36:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 -- 172.21.15.32:0/715008462 learned_addr learned my addr 172.21.15.32:0/715008462 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765763640 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea271420 0x555dea28a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 -- 172.21.15.32:0/715008462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea271420 msgr2=0x555dea28a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea271420 0x555dea28a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 -- 172.21.15.32:0/715008462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 msgr2=0x555dea26b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 0x555dea26b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 -- 172.21.15.32:0/715008462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dea276530 con 0x555dea270b00 2024-04-16T05:36:37.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765763640 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea271420 0x555dea28a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.233+0000 7fe765f64640 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea270b00 0x555dea27de20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe75c007590 tx=0x7fe75c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92de63624158c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe74b7fe640 1 -- 172.21.15.32:0/715008462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0a57e0 con 0x555dea270b00 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe74b7fe640 1 -- 172.21.15.32:0/715008462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75c09aca0 con 0x555dea270b00 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe74b7fe640 1 -- 172.21.15.32:0/715008462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0ce5a0 con 0x555dea270b00 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 -- 172.21.15.32:0/715008462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea270b00 msgr2=0x555dea27de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea270b00 0x555dea27de20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe75c007590 tx=0x7fe75c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 -- 172.21.15.32:0/715008462 shutdown_connections 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea27e760 0x555dea26b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea271420 0x555dea28a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 --2- 172.21.15.32:0/715008462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea270b00 0x555dea27de20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 -- 172.21.15.32:0/715008462 >> 172.21.15.32:0/715008462 conn(0x555dea26b580 msgr2=0x555dea269ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 -- 172.21.15.32:0/715008462 shutdown_connections 2024-04-16T05:36:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.234+0000 7fe76adb3740 1 -- 172.21.15.32:0/715008462 wait complete. 2024-04-16T05:36:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 1 -- start start 2024-04-16T05:36:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 1 Processor -- start 2024-04-16T05:36:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea2ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea2ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 0x555dea283dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601ab720 con 0x555dea271420 2024-04-16T05:36:37.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601acb90 con 0x555dea270b00 2024-04-16T05:36:37.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601abdf0 con 0x555dea27e760 2024-04-16T05:36:37.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe76adb3740 10 client.0 _ll_register_callbacks cb 0x7fe75c098ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:37.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe765763640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea2ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe765763640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea2ed590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60678/0 (socket says 172.21.15.32:60678) 2024-04-16T05:36:37.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe765763640 1 -- 172.21.15.32:0/3796865424 learned_addr learned my addr 172.21.15.32:0/3796865424 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe766765640 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 0x555dea283dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.235+0000 7fe765f64640 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea2ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe765763640 1 -- 172.21.15.32:0/3796865424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 msgr2=0x555dea283dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe765763640 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 0x555dea283dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe765763640 1 -- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 msgr2=0x555dea2ed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe765763640 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea2ed050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe765763640 1 -- 172.21.15.32:0/3796865424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dea2844b0 con 0x555dea271420 2024-04-16T05:36:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe765f64640 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea2ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe765763640 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea2ed590 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe7500105e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc843ed5aa658104 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe7497fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe7497fa640 1 -- 172.21.15.32:0/3796865424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750043060 con 0x555dea271420 2024-04-16T05:36:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe7497fa640 1 -- 172.21.15.32:0/3796865424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fe750031a30 con 0x555dea271420 2024-04-16T05:36:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 1 -- 172.21.15.32:0/3796865424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c09aca0 con 0x555dea271420 2024-04-16T05:36:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe7497fa640 1 -- 172.21.15.32:0/3796865424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe750011cd0 con 0x555dea271420 2024-04-16T05:36:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 10 client.16041 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 1 -- 172.21.15.32:0/3796865424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555dea2f47a0 con 0x555dea271420 2024-04-16T05:36:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe7497fa640 1 -- 172.21.15.32:0/3796865424 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75c09aca0 con 0x555dea271420 2024-04-16T05:36:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe7497fa640 1 -- 172.21.15.32:0/3796865424 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe750004b00 con 0x555dea271420 2024-04-16T05:36:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 10 client.16041 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 20 client.16041 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555dea2f4f50 0x555dea2f7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 4 client.16041 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:36:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.236+0000 7fe76adb3740 1 -- 172.21.15.32:0/3796865424 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555dea1dda10 con 0x555dea2f4f50 2024-04-16T05:36:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.237+0000 7fe765f64640 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555dea2f4f50 0x555dea2f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.237+0000 7fe765f64640 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555dea2f4f50 0x555dea2f7370 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ac7e7f9297e2828 server_cookie=8b4e5851dbd9d824 in_seq=0 out_seq=0 2024-04-16T05:36:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.237+0000 7fe7497fa640 10 client.16041 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7497fa640 1 -- 172.21.15.32:0/3796865424 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555dea1dda10 con 0x555dea2f4f50 2024-04-16T05:36:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7497fa640 10 client.16041 handle_command_reply: tid=0 2024-04-16T05:36:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 client.16041 shutdown 2024-04-16T05:36:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 -- 172.21.15.32:0/3796865424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 msgr2=0x555dea2ed590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea2ed590 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe7500105e0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 -- 172.21.15.32:0/3796865424 shutdown_connections 2024-04-16T05:36:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555dea2f4f50 0x555dea2f7370 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dea27e760 0x555dea283dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555dea271420 0x555dea2ed590 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 --2- 172.21.15.32:0/3796865424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dea270b00 0x555dea2ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7497fa640 0 client.16041 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 -- 172.21.15.32:0/3796865424 >> 172.21.15.32:0/3796865424 conn(0x555dea26b580 msgr2=0x555dea26a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 -- 172.21.15.32:0/3796865424 shutdown_connections 2024-04-16T05:36:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 1 -- 172.21.15.32:0/3796865424 wait complete. 2024-04-16T05:36:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe76adb3740 20 client.16041 trim_cache size 0 max 16384 2024-04-16T05:36:37.250 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:36:37.250 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:36:37.250 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:36:37.250 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:36:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 -- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe74c0a3da0 msgr2=0x7fe74c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe74c0a3da0 0x7fe74c0a6260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe760107600 tx=0x7fe750005e50 comp rx=0 tx=0).stop 2024-04-16T05:36:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 -- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006a600 msgr2=0x7fe76019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006a600 0x7fe76019e400 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c970 tx=0x7fe75c00ce30 comp rx=0 tx=0).stop 2024-04-16T05:36:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 -- 172.21.15.32:0/2264942682 shutdown_connections 2024-04-16T05:36:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe760108200 0x7fe7601985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe76006afc0 0x7fe76019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe74c0a3da0 0x7fe74c0a6260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.238+0000 7fe7681c8640 1 --2- 172.21.15.32:0/2264942682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76006a600 0x7fe76019e400 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.239+0000 7fe7681c8640 1 -- 172.21.15.32:0/2264942682 >> 172.21.15.32:0/2264942682 conn(0x7fe760076d00 msgr2=0x7fe7600ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.239+0000 7fe7681c8640 1 -- 172.21.15.32:0/2264942682 shutdown_connections 2024-04-16T05:36:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.239+0000 7fe7681c8640 1 -- 172.21.15.32:0/2264942682 wait complete. 2024-04-16T05:36:37.253 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:36:37.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:36:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 -- 172.21.15.32:0/2284536413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 msgr2=0x7f6d341026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 --2- 172.21.15.32:0/2284536413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 0x7f6d341026d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009900 tx=0x7f6d2402bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 -- 172.21.15.32:0/2284536413 shutdown_connections 2024-04-16T05:36:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 --2- 172.21.15.32:0/2284536413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d34107330 0x7f6d3410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 --2- 172.21.15.32:0/2284536413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d34102ca0 0x7f6d34106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 --2- 172.21.15.32:0/2284536413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 0x7f6d341026d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 -- 172.21.15.32:0/2284536413 >> 172.21.15.32:0/2284536413 conn(0x7f6d34079190 msgr2=0x7f6d340795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 -- 172.21.15.32:0/2284536413 shutdown_connections 2024-04-16T05:36:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.360+0000 7f6d38b50640 1 -- 172.21.15.32:0/2284536413 wait complete. 2024-04-16T05:36:37.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d38b50640 1 Processor -- start 2024-04-16T05:36:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d38b50640 1 -- start start 2024-04-16T05:36:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d38b50640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 0x7f6d3419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d38b50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d34107330 0x7f6d3419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d38b50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d341986c0 0x7f6d34198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d38b50640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341123e0 con 0x7f6d341022f0 2024-04-16T05:36:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d38b50640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6d34112260 con 0x7f6d341986c0 2024-04-16T05:36:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d38b50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d34112560 con 0x7f6d34107330 2024-04-16T05:36:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d3259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 0x7f6d3419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d3259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 0x7f6d3419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60688/0 (socket says 172.21.15.32:60688) 2024-04-16T05:36:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.361+0000 7f6d3259c640 1 -- 172.21.15.32:0/213657352 learned_addr learned my addr 172.21.15.32:0/213657352 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d32d9d640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d341986c0 0x7f6d34198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d2bfff640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d34107330 0x7f6d3419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d3259c640 1 -- 172.21.15.32:0/213657352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d34107330 msgr2=0x7f6d3419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d3259c640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d34107330 0x7f6d3419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d3259c640 1 -- 172.21.15.32:0/213657352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d341986c0 msgr2=0x7f6d34198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d3259c640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d341986c0 0x7f6d34198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d3259c640 1 -- 172.21.15.32:0/213657352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d341992c0 con 0x7f6d341022f0 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d32d9d640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d341986c0 0x7f6d34198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d3259c640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 0x7f6d3419e600 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6d24006010 tx=0x7f6d2402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d2b7fe640 1 -- 172.21.15.32:0/213657352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24041070 con 0x7f6d341022f0 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d34199550 con 0x7f6d341022f0 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d2b7fe640 1 -- 172.21.15.32:0/213657352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d24004450 con 0x7f6d341022f0 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d2b7fe640 1 -- 172.21.15.32:0/213657352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24034440 con 0x7f6d341022f0 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.362+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d341a56e0 con 0x7f6d341022f0 2024-04-16T05:36:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.363+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d34103730 con 0x7f6d341022f0 2024-04-16T05:36:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.364+0000 7f6d2b7fe640 1 -- 172.21.15.32:0/213657352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d24044070 con 0x7f6d341022f0 2024-04-16T05:36:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.365+0000 7f6d2b7fe640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6d200a3d20 0x7f6d200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.365+0000 7f6d2bfff640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6d200a3d20 0x7f6d200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.365+0000 7f6d2b7fe640 1 -- 172.21.15.32:0/213657352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6d240f2750 con 0x7f6d341022f0 2024-04-16T05:36:37.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.365+0000 7f6d2bfff640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6d200a3d20 0x7f6d200a61e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6d34199f10 tx=0x7f6d1c00b590 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.369+0000 7f6d2b7fe640 1 -- 172.21.15.32:0/213657352 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d240beb60 con 0x7f6d341022f0 2024-04-16T05:36:37.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.596+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d3410a530 con 0x7f6d341022f0 2024-04-16T05:36:37.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.596+0000 7f6d2b7fe640 1 -- 172.21.15.32:0/213657352 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f6d240be460 con 0x7f6d341022f0 2024-04-16T05:36:37.595 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:36:37.595 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:36:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:36:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6d200a3d20 msgr2=0x7f6d200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6d200a3d20 0x7f6d200a61e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6d34199f10 tx=0x7f6d1c00b590 comp rx=0 tx=0).stop 2024-04-16T05:36:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 msgr2=0x7f6d3419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 0x7f6d3419e600 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6d24006010 tx=0x7f6d2402cc70 comp rx=0 tx=0).stop 2024-04-16T05:36:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 shutdown_connections 2024-04-16T05:36:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d341986c0 0x7f6d34198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d34107330 0x7f6d3419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6d200a3d20 0x7f6d200a61e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 --2- 172.21.15.32:0/213657352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d341022f0 0x7f6d3419e600 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 >> 172.21.15.32:0/213657352 conn(0x7f6d34079190 msgr2=0x7f6d34101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 shutdown_connections 2024-04-16T05:36:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.599+0000 7f6d38b50640 1 -- 172.21.15.32:0/213657352 wait complete. 2024-04-16T05:36:37.609 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:36:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 -- 172.21.15.32:0/2999502772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0410c5e0 msgr2=0x7f1d0410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 --2- 172.21.15.32:0/2999502772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0410c5e0 0x7f1d0410ea80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009900 tx=0x7f1cfc02bd60 comp rx=0 tx=0).stop 2024-04-16T05:36:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 -- 172.21.15.32:0/2999502772 shutdown_connections 2024-04-16T05:36:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 --2- 172.21.15.32:0/2999502772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d0410c5e0 0x7f1d0410ea80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 --2- 172.21.15.32:0/2999502772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d04103bc0 0x7f1d0410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 --2- 172.21.15.32:0/2999502772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d041032a0 0x7f1d04103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 -- 172.21.15.32:0/2999502772 >> 172.21.15.32:0/2999502772 conn(0x7f1d04077280 msgr2=0x7f1d04077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 -- 172.21.15.32:0/2999502772 shutdown_connections 2024-04-16T05:36:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.718+0000 7f1d09a47640 1 -- 172.21.15.32:0/2999502772 wait complete. 2024-04-16T05:36:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d09a47640 1 Processor -- start 2024-04-16T05:36:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d09a47640 1 -- start start 2024-04-16T05:36:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d09a47640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d041032a0 0x7f1d0419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d09a47640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04103bc0 0x7f1d0419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d09a47640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d0410c5e0 0x7f1d04198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d09a47640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04112410 con 0x7f1d041032a0 2024-04-16T05:36:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d09a47640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04112290 con 0x7f1d04103bc0 2024-04-16T05:36:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d09a47640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04112590 con 0x7f1d0410c5e0 2024-04-16T05:36:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d041032a0 0x7f1d0419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d041032a0 0x7f1d0419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60708/0 (socket says 172.21.15.32:60708) 2024-04-16T05:36:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d02ffd640 1 -- 172.21.15.32:0/1825966640 learned_addr learned my addr 172.21.15.32:0/1825966640 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d037fe640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d0410c5e0 0x7f1d04198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.719+0000 7f1d027fc640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04103bc0 0x7f1d0419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d02ffd640 1 -- 172.21.15.32:0/1825966640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d0410c5e0 msgr2=0x7f1d04198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d0410c5e0 0x7f1d04198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d02ffd640 1 -- 172.21.15.32:0/1825966640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04103bc0 msgr2=0x7f1d0419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04103bc0 0x7f1d0419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d02ffd640 1 -- 172.21.15.32:0/1825966640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d04198d90 con 0x7f1d041032a0 2024-04-16T05:36:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d037fe640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d0410c5e0 0x7f1d04198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d027fc640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04103bc0 0x7f1d0419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d041032a0 0x7f1d0419e460 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400eab0 tx=0x7f1cf400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d08a45640 1 -- 172.21.15.32:0/1825966640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400ce40 con 0x7f1d041032a0 2024-04-16T05:36:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d08a45640 1 -- 172.21.15.32:0/1825966640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf4011950 con 0x7f1d041032a0 2024-04-16T05:36:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d08a45640 1 -- 172.21.15.32:0/1825966640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4010ae0 con 0x7f1d041032a0 2024-04-16T05:36:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d09a47640 1 -- 172.21.15.32:0/1825966640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d04199080 con 0x7f1d041032a0 2024-04-16T05:36:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.720+0000 7f1d09a47640 1 -- 172.21.15.32:0/1825966640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d041a5320 con 0x7f1d041032a0 2024-04-16T05:36:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.722+0000 7f1d08a45640 1 -- 172.21.15.32:0/1825966640 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1cf4011b00 con 0x7f1d041032a0 2024-04-16T05:36:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d0c62e740 1 Processor -- start 2024-04-16T05:36:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d0c62e740 1 -- start start 2024-04-16T05:36:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 0x55d8884e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884eb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d8884450e0 con 0x55d8884e0780 2024-04-16T05:36:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d8883e2520 con 0x55d8884e9360 2024-04-16T05:36:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d8884410a0 con 0x55d8884dfe60 2024-04-16T05:36:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d08a45640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1cec0a3d20 0x7f1cec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60716/0 (socket says 172.21.15.32:60716) 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d02ffd640 1 -- 172.21.15.32:0/2223944354 learned_addr learned my addr 172.21.15.32:0/2223944354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d037fe640 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 0x55d8884e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d037fe640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1cec0a3d20 0x7f1cec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d027fc640 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d08a45640 1 -- 172.21.15.32:0/1825966640 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1cf40d1310 con 0x7f1d041032a0 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d02ffd640 1 -- 172.21.15.32:0/2223944354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 msgr2=0x55d8884e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 0x55d8884e0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d02ffd640 1 -- 172.21.15.32:0/2223944354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 msgr2=0x55d8884eb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884eb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.723+0000 7f1d02ffd640 1 -- 172.21.15.32:0/2223944354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88840a8d0 con 0x55d8884e0780 2024-04-16T05:36:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.724+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e8e20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1cf401d000 tx=0x7f1cf401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdb0f66b037fe47 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.724+0000 7f1d037fe640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1cec0a3d20 0x7f1cec0a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc006010 tx=0x7f1cfc0056d0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.724+0000 7f1ce9ffb640 1 -- 172.21.15.32:0/2223944354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf409cf10 con 0x55d8884e0780 2024-04-16T05:36:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.724+0000 7f1ce9ffb640 1 -- 172.21.15.32:0/2223944354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf40a12c0 con 0x55d8884e0780 2024-04-16T05:36:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.724+0000 7f1ce9ffb640 1 -- 172.21.15.32:0/2223944354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40ce940 con 0x55d8884e0780 2024-04-16T05:36:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.724+0000 7f1d0c62e740 1 -- 172.21.15.32:0/2223944354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 msgr2=0x55d8884e8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.724+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e8e20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1cf401d000 tx=0x7f1cf401e000 comp rx=0 tx=0).stop 2024-04-16T05:36:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 -- 172.21.15.32:0/2223944354 shutdown_connections 2024-04-16T05:36:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884eb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e8e20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/2223944354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 0x55d8884e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 -- 172.21.15.32:0/2223944354 >> 172.21.15.32:0/2223944354 conn(0x55d8884da590 msgr2=0x55d8884da970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 -- 172.21.15.32:0/2223944354 shutdown_connections 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 -- 172.21.15.32:0/2223944354 wait complete. 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 -- start start 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 Processor -- start 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884e6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 0x55d8884e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d8884ec000 con 0x55d8884dfe60 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d8883fe4d0 con 0x55d8884e0780 2024-04-16T05:36:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d8884ec180 con 0x55d8884e9360 2024-04-16T05:36:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d0c62e740 10 client.0 _ll_register_callbacks cb 0x55d8883e32c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.725+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884e6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60718/0 (socket says 172.21.15.32:60718) 2024-04-16T05:36:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d02ffd640 1 -- 172.21.15.32:0/4204082894 learned_addr learned my addr 172.21.15.32:0/4204082894 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d037fe640 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d02ffd640 1 -- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 msgr2=0x55d8884e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d02ffd640 1 -- 172.21.15.32:0/4204082894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 msgr2=0x55d8884e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d027fc640 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 0x55d8884e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 0x55d8884e6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d02ffd640 1 -- 172.21.15.32:0/4204082894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8881573f0 con 0x55d8884dfe60 2024-04-16T05:36:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d037fe640 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884e6400 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1cf409cea0 tx=0x7f1cf40094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77aff1e7a1e2a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/4204082894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40b5070 con 0x55d8884dfe60 2024-04-16T05:36:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/4204082894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1cf40b6260 con 0x55d8884dfe60 2024-04-16T05:36:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d0c62e740 1 -- 172.21.15.32:0/4204082894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf40a12c0 con 0x55d8884dfe60 2024-04-16T05:36:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/4204082894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1cf40ceb50 con 0x55d8884dfe60 2024-04-16T05:36:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d0c62e740 10 client.16059 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1d0c62e740 1 -- 172.21.15.32:0/4204082894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d8882217d0 con 0x55d8884dfe60 2024-04-16T05:36:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.726+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/4204082894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf40a12c0 con 0x55d8884dfe60 2024-04-16T05:36:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/4204082894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1cf400f040 con 0x55d8884dfe60 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1d0c62e740 10 client.16059 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1d0c62e740 10 client.16059 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1d0c62e740 20 client.16059 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55d8884f9260 0x55d8884fb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1d0c62e740 4 client.16059 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1d0c62e740 1 -- 172.21.15.32:0/4204082894 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d888452b00 con 0x55d8884f9260 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1d027fc640 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55d8884f9260 0x55d8884fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1d027fc640 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55d8884f9260 0x55d8884fb650 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65fa10f670f8bb9a server_cookie=d7bf883f60194ea6 in_seq=0 out_seq=0 2024-04-16T05:36:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.727+0000 7f1ce2ffd640 10 client.16059 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:36:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.728+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/4204082894 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d888452b00 con 0x55d8884f9260 2024-04-16T05:36:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.728+0000 7f1ce2ffd640 10 client.16059 handle_command_reply: tid=0 2024-04-16T05:36:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.728+0000 7f1d0c62e740 1 client.16059 shutdown 2024-04-16T05:36:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.728+0000 7f1d0c62e740 1 -- 172.21.15.32:0/4204082894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 msgr2=0x55d8884e6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.728+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884e6400 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1cf409cea0 tx=0x7f1cf40094a0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.728+0000 7f1d0c62e740 1 -- 172.21.15.32:0/4204082894 shutdown_connections 2024-04-16T05:36:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.728+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55d8884f9260 0x55d8884fb650 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.728+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.729+0000 7f1ce2ffd640 0 client.16059 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:36:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.729+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 0x55d8884e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.729+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/4204082894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884e6400 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.729+0000 7f1d0c62e740 1 -- 172.21.15.32:0/4204082894 >> 172.21.15.32:0/4204082894 conn(0x55d8884da590 msgr2=0x55d8884dc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.729+0000 7f1d0c62e740 1 -- 172.21.15.32:0/4204082894 shutdown_connections 2024-04-16T05:36:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.729+0000 7f1d0c62e740 1 -- 172.21.15.32:0/4204082894 wait complete. 2024-04-16T05:36:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.729+0000 7f1d0c62e740 20 client.16059 trim_cache size 0 max 16384 2024-04-16T05:36:37.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.742+0000 7f1d0c62e740 1 Processor -- start 2024-04-16T05:36:37.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d0c62e740 1 -- start start 2024-04-16T05:36:37.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 0x55d8884f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884fe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041b1c10 con 0x55d8884e0780 2024-04-16T05:36:37.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041af4e0 con 0x55d8884e9360 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041ad330 con 0x55d8884dfe60 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60734/0 (socket says 172.21.15.32:60734) 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 -- 172.21.15.32:0/3787864051 learned_addr learned my addr 172.21.15.32:0/3787864051 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d037fe640 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 -- 172.21.15.32:0/3787864051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 msgr2=0x55d8884f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 0x55d8884f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 0x55d8884f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 -- 172.21.15.32:0/3787864051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 msgr2=0x55d8884fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884fe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 -- 172.21.15.32:0/3787864051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8884ebac0 con 0x55d8884e0780 2024-04-16T05:36:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d037fe640 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1d027fc640 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e52b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1d04199750 tx=0x7f1cf800b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ad14c777ad80c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.743+0000 7f1ce97fa640 1 -- 172.21.15.32:0/3787864051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8013990 con 0x55d8884e0780 2024-04-16T05:36:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1ce97fa640 1 -- 172.21.15.32:0/3787864051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf801a330 con 0x55d8884e0780 2024-04-16T05:36:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1ce97fa640 1 -- 172.21.15.32:0/3787864051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8011860 con 0x55d8884e0780 2024-04-16T05:36:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 -- 172.21.15.32:0/3787864051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 msgr2=0x55d8884e52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e52b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1d04199750 tx=0x7f1cf800b370 comp rx=0 tx=0).stop 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 -- 172.21.15.32:0/3787864051 shutdown_connections 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e9360 0x55d8884fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884e0780 0x55d8884e52b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/3787864051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884dfe60 0x55d8884f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 -- 172.21.15.32:0/3787864051 >> 172.21.15.32:0/3787864051 conn(0x55d8884da590 msgr2=0x55d8884e4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 -- 172.21.15.32:0/3787864051 shutdown_connections 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.744+0000 7f1d0c62e740 1 -- 172.21.15.32:0/3787864051 wait complete. 2024-04-16T05:36:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 1 -- start start 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 1 Processor -- start 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884fc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 0x55d8884fc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041ad9a0 con 0x55d8884dfe60 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041aee10 con 0x55d8884e0780 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041ae070 con 0x55d8884e9360 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d0c62e740 10 client.0 _ll_register_callbacks cb 0x7f1cf8005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884fc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884fc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60738/0 (socket says 172.21.15.32:60738) 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d02ffd640 1 -- 172.21.15.32:0/1426825693 learned_addr learned my addr 172.21.15.32:0/1426825693 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d037fe640 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d02ffd640 1 -- 172.21.15.32:0/1426825693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 msgr2=0x55d8884fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884fcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d027fc640 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 0x55d8884fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d02ffd640 1 -- 172.21.15.32:0/1426825693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 msgr2=0x55d8884fc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 0x55d8884fc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.745+0000 7f1d02ffd640 1 -- 172.21.15.32:0/1426825693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88855e730 con 0x55d8884dfe60 2024-04-16T05:36:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d02ffd640 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884fc400 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40998e0 tx=0x7f1cf40d11e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366424b2f7baefd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/1426825693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf407f060 con 0x55d8884dfe60 2024-04-16T05:36:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/1426825693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1cf409d370 con 0x55d8884dfe60 2024-04-16T05:36:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 1 -- 172.21.15.32:0/1426825693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf801a330 con 0x55d8884dfe60 2024-04-16T05:36:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/1426825693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1cf4086050 con 0x55d8884dfe60 2024-04-16T05:36:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 10 client.16065 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 1 -- 172.21.15.32:0/1426825693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d88855f9a0 con 0x55d8884dfe60 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/1426825693 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf801a330 con 0x55d8884dfe60 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/1426825693 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1cf4098c60 con 0x55d8884dfe60 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 10 client.16065 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 10 client.16065 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 20 client.16065 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55d88855ff00 0x55d888562340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 4 client.16065 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d0c62e740 1 -- 172.21.15.32:0/1426825693 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d88844c970 con 0x55d88855ff00 2024-04-16T05:36:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.746+0000 7f1d027fc640 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55d88855ff00 0x55d888562340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d027fc640 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55d88855ff00 0x55d888562340 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c0fa6e1feed37f2 server_cookie=4ce5456e8ccef2ea in_seq=0 out_seq=0 2024-04-16T05:36:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1ce2ffd640 10 client.16065 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:36:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1ce2ffd640 1 -- 172.21.15.32:0/1426825693 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d88844c970 con 0x55d88855ff00 2024-04-16T05:36:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1ce2ffd640 10 client.16065 handle_command_reply: tid=0 2024-04-16T05:36:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 client.16065 shutdown 2024-04-16T05:36:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 -- 172.21.15.32:0/1426825693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 msgr2=0x55d8884fc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884fc400 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40998e0 tx=0x7f1cf40d11e0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 -- 172.21.15.32:0/1426825693 shutdown_connections 2024-04-16T05:36:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55d88855ff00 0x55d888562340 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8884e9360 0x55d8884fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8884e0780 0x55d8884fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1ce2ffd640 0 client.16065 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:36:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 --2- 172.21.15.32:0/1426825693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8884dfe60 0x55d8884fc400 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 -- 172.21.15.32:0/1426825693 >> 172.21.15.32:0/1426825693 conn(0x55d8884da590 msgr2=0x55d8884e5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 -- 172.21.15.32:0/1426825693 shutdown_connections 2024-04-16T05:36:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.747+0000 7f1d0c62e740 1 -- 172.21.15.32:0/1426825693 wait complete. 2024-04-16T05:36:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d0c62e740 20 client.16065 trim_cache size 0 max 16384 2024-04-16T05:36:37.758 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:36:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 -- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1cec0a3d20 msgr2=0x7f1cec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1cec0a3d20 0x7f1cec0a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc006010 tx=0x7f1cfc0056d0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 -- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d041032a0 msgr2=0x7f1d0419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d041032a0 0x7f1d0419e460 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400eab0 tx=0x7f1cf400ef70 comp rx=0 tx=0).stop 2024-04-16T05:36:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 -- 172.21.15.32:0/1825966640 shutdown_connections 2024-04-16T05:36:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d0410c5e0 0x7f1d04198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04103bc0 0x7f1d0419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1cec0a3d20 0x7f1cec0a61e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 --2- 172.21.15.32:0/1825966640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d041032a0 0x7f1d0419e460 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 -- 172.21.15.32:0/1825966640 >> 172.21.15.32:0/1825966640 conn(0x7f1d04077280 msgr2=0x7f1d040fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 -- 172.21.15.32:0/1825966640 shutdown_connections 2024-04-16T05:36:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.748+0000 7f1d09a47640 1 -- 172.21.15.32:0/1825966640 wait complete. 2024-04-16T05:36:37.761 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:37 smithi032 ceph-mon[32351]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:37.761 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:37 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/213657352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:37.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:36:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 -- 172.21.15.32:0/2774372856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc101af0 msgr2=0x7f55fc105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 --2- 172.21.15.32:0/2774372856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc101af0 0x7f55fc105b30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009900 tx=0x7f55ec02bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 -- 172.21.15.32:0/2774372856 shutdown_connections 2024-04-16T05:36:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 --2- 172.21.15.32:0/2774372856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc106210 0x7f55fc10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 --2- 172.21.15.32:0/2774372856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc101af0 0x7f55fc105b30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 --2- 172.21.15.32:0/2774372856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc101140 0x7f55fc101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 -- 172.21.15.32:0/2774372856 >> 172.21.15.32:0/2774372856 conn(0x7f55fc077280 msgr2=0x7f55fc077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 -- 172.21.15.32:0/2774372856 shutdown_connections 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.871+0000 7f5603d9d640 1 -- 172.21.15.32:0/2774372856 wait complete. 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5603d9d640 1 Processor -- start 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5603d9d640 1 -- start start 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5603d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc101140 0x7f55fc1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5603d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc106210 0x7f55fc199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5603d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc19a1b0 0x7f55fc1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5603d9d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc111280 con 0x7f55fc19a1b0 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5603d9d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc111100 con 0x7f55fc101140 2024-04-16T05:36:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5603d9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc111400 con 0x7f55fc106210 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f560233a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc19a1b0 0x7f55fc1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f560233a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc19a1b0 0x7f55fc1a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60748/0 (socket says 172.21.15.32:60748) 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f560233a640 1 -- 172.21.15.32:0/458278579 learned_addr learned my addr 172.21.15.32:0/458278579 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5601338640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc106210 0x7f55fc199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5601b39640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc101140 0x7f55fc1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f560233a640 1 -- 172.21.15.32:0/458278579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc106210 msgr2=0x7f55fc199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f560233a640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc106210 0x7f55fc199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f560233a640 1 -- 172.21.15.32:0/458278579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc101140 msgr2=0x7f55fc1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f560233a640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc101140 0x7f55fc1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f560233a640 1 -- 172.21.15.32:0/458278579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc1a4f60 con 0x7f55fc19a1b0 2024-04-16T05:36:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.872+0000 7f5601b39640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc101140 0x7f55fc1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.873+0000 7f560233a640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc19a1b0 0x7f55fc1a4760 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f55f400eaa0 tx=0x7f55f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.873+0000 7f55f2ffd640 1 -- 172.21.15.32:0/458278579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400cdb0 con 0x7f55fc19a1b0 2024-04-16T05:36:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.873+0000 7f55f2ffd640 1 -- 172.21.15.32:0/458278579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f4005020 con 0x7f55fc19a1b0 2024-04-16T05:36:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.873+0000 7f55f2ffd640 1 -- 172.21.15.32:0/458278579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f40116b0 con 0x7f55fc19a1b0 2024-04-16T05:36:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.873+0000 7f5603d9d640 1 -- 172.21.15.32:0/458278579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc1a5250 con 0x7f55fc19a1b0 2024-04-16T05:36:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.873+0000 7f5603d9d640 1 -- 172.21.15.32:0/458278579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc1a5720 con 0x7f55fc19a1b0 2024-04-16T05:36:37.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.875+0000 7f55f2ffd640 1 -- 172.21.15.32:0/458278579 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f55f4005560 con 0x7f55fc19a1b0 2024-04-16T05:36:37.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.875+0000 7f5606989740 1 Processor -- start 2024-04-16T05:36:37.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5606989740 1 -- start start 2024-04-16T05:36:37.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5606989740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f780240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5606989740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 0x559e6f788e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5606989740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5606989740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559e6f6e50e0 con 0x559e6f789360 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5606989740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559e6f682520 con 0x559e6f77fe60 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5606989740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e6f6e10a0 con 0x559e6f780780 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f55f2ffd640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f55e80a3d20 0x7f55e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60754/0 (socket says 172.21.15.32:60754) 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 -- 172.21.15.32:0/2671338864 learned_addr learned my addr 172.21.15.32:0/2671338864 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601b39640 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 0x559e6f788e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f560233a640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f55e80a3d20 0x7f55e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f780240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f55f2ffd640 1 -- 172.21.15.32:0/458278579 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f55f40d0d80 con 0x7f55fc19a1b0 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 -- 172.21.15.32:0/2671338864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 msgr2=0x559e6f788e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 0x559e6f788e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 -- 172.21.15.32:0/2671338864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 msgr2=0x559e6f780240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f780240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.876+0000 7f5601338640 1 -- 172.21.15.32:0/2671338864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6f6aa8d0 con 0x559e6f789360 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.877+0000 7f5601338640 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78b750 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f55fc06a470 tx=0x7f55ec005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac687f2b4f5b6e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.877+0000 7f560233a640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f55e80a3d20 0x7f55e80a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55f409c680 tx=0x7f55f40985e0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.877+0000 7f55e7fff640 1 -- 172.21.15.32:0/2671338864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004230 con 0x559e6f789360 2024-04-16T05:36:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.877+0000 7f55e7fff640 1 -- 172.21.15.32:0/2671338864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec02ccc0 con 0x559e6f789360 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.877+0000 7f55e7fff640 1 -- 172.21.15.32:0/2671338864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0054c0 con 0x559e6f789360 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.877+0000 7f5606989740 1 -- 172.21.15.32:0/2671338864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 msgr2=0x559e6f78b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.877+0000 7f5606989740 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78b750 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f55fc06a470 tx=0x7f55ec005dd0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 -- 172.21.15.32:0/2671338864 shutdown_connections 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78b750 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 0x559e6f788e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 --2- 172.21.15.32:0/2671338864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f780240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 -- 172.21.15.32:0/2671338864 >> 172.21.15.32:0/2671338864 conn(0x559e6f77a590 msgr2=0x559e6f77a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 -- 172.21.15.32:0/2671338864 shutdown_connections 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 -- 172.21.15.32:0/2671338864 wait complete. 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 -- start start 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 Processor -- start 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f786470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 0x559e6f786ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559e6f78bfa0 con 0x559e6f780780 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559e6f69e4d0 con 0x559e6f77fe60 2024-04-16T05:36:37.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e6f78c120 con 0x559e6f789360 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.878+0000 7f5606989740 10 client.0 _ll_register_callbacks cb 0x559e6f6832c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7869b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60768/0 (socket says 172.21.15.32:60768) 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 -- 172.21.15.32:0/3250481509 learned_addr learned my addr 172.21.15.32:0/3250481509 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601338640 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 0x559e6f786ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601338640 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f786470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 -- 172.21.15.32:0/3250481509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 msgr2=0x559e6f786ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 0x559e6f786ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 -- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 msgr2=0x559e6f786470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f786470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 -- 172.21.15.32:0/3250481509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6f3f73f0 con 0x559e6f780780 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601338640 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f786470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5601b39640 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7869b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f55f80113e0 tx=0x7f55f80118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef9a000f10b945f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f55e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f55e57fa640 1 -- 172.21.15.32:0/3250481509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8012980 con 0x559e6f780780 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f55e57fa640 1 -- 172.21.15.32:0/3250481509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f55f8012b20 con 0x559e6f780780 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5606989740 1 -- 172.21.15.32:0/3250481509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ec02ccc0 con 0x559e6f780780 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f55e57fa640 1 -- 172.21.15.32:0/3250481509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55f8025050 con 0x559e6f780780 2024-04-16T05:36:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5606989740 10 client.16077 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f5606989740 1 -- 172.21.15.32:0/3250481509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e6f4c17d0 con 0x559e6f780780 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.879+0000 7f55e57fa640 1 -- 172.21.15.32:0/3250481509 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec02ccc0 con 0x559e6f780780 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f55e57fa640 1 -- 172.21.15.32:0/3250481509 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f55f800f040 con 0x559e6f780780 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f5606989740 10 client.16077 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f5606989740 10 client.16077 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f5606989740 20 client.16077 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f5606989740 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x559e6f799420 0x559e6f79b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f5606989740 4 client.16077 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f5606989740 1 -- 172.21.15.32:0/3250481509 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6f6f2b00 con 0x559e6f799420 2024-04-16T05:36:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f5601338640 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x559e6f799420 0x559e6f79b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f55e57fa640 10 client.16077 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.880+0000 7f5601338640 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x559e6f799420 0x559e6f79b8c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5de937057c931f04 server_cookie=356e55e6e7e714 in_seq=0 out_seq=0 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f55e57fa640 1 -- 172.21.15.32:0/3250481509 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e6f6f2b00 con 0x559e6f799420 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f55e57fa640 10 client.16077 handle_command_reply: tid=0 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 client.16077 shutdown 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 -- 172.21.15.32:0/3250481509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 msgr2=0x559e6f7869b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7869b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f55f80113e0 tx=0x7f55f80118a0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 -- 172.21.15.32:0/3250481509 shutdown_connections 2024-04-16T05:36:37.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x559e6f799420 0x559e6f79b8c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 0x559e6f786ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7869b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f55e57fa640 0 client.16077 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:36:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 --2- 172.21.15.32:0/3250481509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f786470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.881+0000 7f5606989740 1 -- 172.21.15.32:0/3250481509 >> 172.21.15.32:0/3250481509 conn(0x559e6f77a590 msgr2=0x559e6f77c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.882+0000 7f5606989740 1 -- 172.21.15.32:0/3250481509 shutdown_connections 2024-04-16T05:36:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.882+0000 7f5606989740 1 -- 172.21.15.32:0/3250481509 wait complete. 2024-04-16T05:36:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.882+0000 7f5606989740 20 client.16077 trim_cache size 0 max 16384 2024-04-16T05:36:37.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5606989740 1 Processor -- start 2024-04-16T05:36:37.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5606989740 1 -- start start 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5606989740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f788ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5606989740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5606989740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 0x559e6f7a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5606989740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1b1c80 con 0x559e6f780780 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5606989740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1af550 con 0x559e6f77fe60 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5606989740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1ad3a0 con 0x559e6f789360 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.895+0000 7f5601b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f5601b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60770/0 (socket says 172.21.15.32:60770) 2024-04-16T05:36:37.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f5601b39640 1 -- 172.21.15.32:0/1353187066 learned_addr learned my addr 172.21.15.32:0/1353187066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f5601b39640 1 -- 172.21.15.32:0/1353187066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 msgr2=0x559e6f7a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f5601b39640 1 --2- 172.21.15.32:0/1353187066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 0x559e6f7a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f5601b39640 1 -- 172.21.15.32:0/1353187066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 msgr2=0x559e6f788ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f5601b39640 1 --2- 172.21.15.32:0/1353187066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f788ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f5601b39640 1 -- 172.21.15.32:0/1353187066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6f782030 con 0x559e6f780780 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f5601b39640 1 --2- 172.21.15.32:0/1353187066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7a1190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f55f8012480 tx=0x7f55f801af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b381b06d40d2511 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f55e77fe640 1 -- 172.21.15.32:0/1353187066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8005270 con 0x559e6f780780 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f55e77fe640 1 -- 172.21.15.32:0/1353187066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f8011da0 con 0x559e6f780780 2024-04-16T05:36:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.896+0000 7f55e77fe640 1 -- 172.21.15.32:0/1353187066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f80049b0 con 0x559e6f780780 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 -- 172.21.15.32:0/1353187066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 msgr2=0x559e6f7a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 --2- 172.21.15.32:0/1353187066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7a1190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f55f8012480 tx=0x7f55f801af90 comp rx=0 tx=0).stop 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 -- 172.21.15.32:0/1353187066 shutdown_connections 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 --2- 172.21.15.32:0/1353187066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f789360 0x559e6f7a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 --2- 172.21.15.32:0/1353187066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f780780 0x559e6f7a1190 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 --2- 172.21.15.32:0/1353187066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f788ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 -- 172.21.15.32:0/1353187066 >> 172.21.15.32:0/1353187066 conn(0x559e6f77a590 msgr2=0x559e6f7755e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 -- 172.21.15.32:0/1353187066 shutdown_connections 2024-04-16T05:36:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 -- 172.21.15.32:0/1353187066 wait complete. 2024-04-16T05:36:37.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 -- start start 2024-04-16T05:36:37.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 Processor -- start 2024-04-16T05:36:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f791760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 0x559e6f78cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.897+0000 7f5606989740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5606989740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1ada10 con 0x559e6f789360 2024-04-16T05:36:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5606989740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1aee80 con 0x559e6f77fe60 2024-04-16T05:36:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5606989740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1ae0e0 con 0x559e6f780780 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5606989740 10 client.0 _ll_register_callbacks cb 0x7f55f80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60778/0 (socket says 172.21.15.32:60778) 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 -- 172.21.15.32:0/1563938752 learned_addr learned my addr 172.21.15.32:0/1563938752 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601b39640 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 0x559e6f78cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 -- 172.21.15.32:0/1563938752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 msgr2=0x559e6f78cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 0x559e6f78cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 -- 172.21.15.32:0/1563938752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 msgr2=0x559e6f791760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f791760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 -- 172.21.15.32:0/1563938752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6f78d700 con 0x559e6f789360 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5601338640 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78d010 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x559e6f77b720 tx=0x7f55ec02c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cc6c531c88824d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f55e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f55e57fa640 1 -- 172.21.15.32:0/1563938752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004a40 con 0x559e6f789360 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f55e57fa640 1 -- 172.21.15.32:0/1563938752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f55ec004be0 con 0x559e6f789360 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5606989740 1 -- 172.21.15.32:0/1563938752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f8011da0 con 0x559e6f789360 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f55e57fa640 1 -- 172.21.15.32:0/1563938752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55ec009520 con 0x559e6f789360 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5606989740 10 client.16083 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.898+0000 7f5606989740 1 -- 172.21.15.32:0/1563938752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e6f7ff900 con 0x559e6f789360 2024-04-16T05:36:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f55e57fa640 1 -- 172.21.15.32:0/1563938752 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f8011da0 con 0x559e6f789360 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f55e57fa640 1 -- 172.21.15.32:0/1563938752 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f55ec055020 con 0x559e6f789360 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f5606989740 10 client.16083 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f5606989740 10 client.16083 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f5606989740 20 client.16083 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f5606989740 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x559e6f7ffbe0 0x559e6f801fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f5606989740 4 client.16083 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f5606989740 1 -- 172.21.15.32:0/1563938752 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e6f6ec970 con 0x559e6f7ffbe0 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f5601b39640 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x559e6f7ffbe0 0x559e6f801fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f5601b39640 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x559e6f7ffbe0 0x559e6f801fd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a015ffe38876495 server_cookie=f30eae4ac5728e22 in_seq=0 out_seq=0 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.899+0000 7f55e57fa640 10 client.16083 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f55e57fa640 1 -- 172.21.15.32:0/1563938752 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e6f6ec970 con 0x559e6f7ffbe0 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f55e57fa640 10 client.16083 handle_command_reply: tid=0 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 client.16083 shutdown 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 -- 172.21.15.32:0/1563938752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 msgr2=0x559e6f78d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78d010 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x559e6f77b720 tx=0x7f55ec02c7d0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 -- 172.21.15.32:0/1563938752 shutdown_connections 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x559e6f7ffbe0 0x559e6f801fd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e6f789360 0x559e6f78d010 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e6f780780 0x559e6f78cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f55e57fa640 0 client.16083 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:36:37.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 --2- 172.21.15.32:0/1563938752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e6f77fe60 0x559e6f791760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 -- 172.21.15.32:0/1563938752 >> 172.21.15.32:0/1563938752 conn(0x559e6f77a590 msgr2=0x559e6f778ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 -- 172.21.15.32:0/1563938752 shutdown_connections 2024-04-16T05:36:37.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 1 -- 172.21.15.32:0/1563938752 wait complete. 2024-04-16T05:36:37.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5606989740 20 client.16083 trim_cache size 0 max 16384 2024-04-16T05:36:37.907 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:36:37.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 -- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f55e80a3d20 msgr2=0x7f55e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f55e80a3d20 0x7f55e80a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55f409c680 tx=0x7f55f40985e0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 -- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc19a1b0 msgr2=0x7f55fc1a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc19a1b0 0x7f55fc1a4760 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f55f400eaa0 tx=0x7f55f400ef60 comp rx=0 tx=0).stop 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 -- 172.21.15.32:0/458278579 shutdown_connections 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f55e80a3d20 0x7f55e80a61e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55fc19a1b0 0x7f55fc1a4760 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc106210 0x7f55fc199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 --2- 172.21.15.32:0/458278579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc101140 0x7f55fc1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.900+0000 7f5603d9d640 1 -- 172.21.15.32:0/458278579 >> 172.21.15.32:0/458278579 conn(0x7f55fc077280 msgr2=0x7f55fc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.901+0000 7f5603d9d640 1 -- 172.21.15.32:0/458278579 shutdown_connections 2024-04-16T05:36:37.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:37.901+0000 7f5603d9d640 1 -- 172.21.15.32:0/458278579 wait complete. 2024-04-16T05:36:37.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:36:38.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 -- 172.21.15.32:0/3424448790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4190102250 msgr2=0x7f4190102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 --2- 172.21.15.32:0/3424448790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4190102250 0x7f4190102630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4180009900 tx=0x7f418002bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 -- 172.21.15.32:0/3424448790 shutdown_connections 2024-04-16T05:36:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 --2- 172.21.15.32:0/3424448790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4190107320 0x7f419010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 --2- 172.21.15.32:0/3424448790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4190102c00 0x7f4190106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 --2- 172.21.15.32:0/3424448790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4190102250 0x7f4190102630 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 -- 172.21.15.32:0/3424448790 >> 172.21.15.32:0/3424448790 conn(0x7f4190079490 msgr2=0x7f41900798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 -- 172.21.15.32:0/3424448790 shutdown_connections 2024-04-16T05:36:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.019+0000 7f4196904640 1 -- 172.21.15.32:0/3424448790 wait complete. 2024-04-16T05:36:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4196904640 1 Processor -- start 2024-04-16T05:36:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4196904640 1 -- start start 2024-04-16T05:36:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4196904640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4190102250 0x7f419019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4196904640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4190107320 0x7f419019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4196904640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41901986f0 0x7f4190198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4196904640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f41901123d0 con 0x7f41901986f0 2024-04-16T05:36:38.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4196904640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4190112250 con 0x7f4190102250 2024-04-16T05:36:38.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4196904640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4190112550 con 0x7f4190107320 2024-04-16T05:36:38.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4194ea1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41901986f0 0x7f4190198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4194ea1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41901986f0 0x7f4190198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60800/0 (socket says 172.21.15.32:60800) 2024-04-16T05:36:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.020+0000 7f4194ea1640 1 -- 172.21.15.32:0/89175794 learned_addr learned my addr 172.21.15.32:0/89175794 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f418ffff640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4190102250 0x7f419019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f418f7fe640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4190107320 0x7f419019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f4194ea1640 1 -- 172.21.15.32:0/89175794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4190107320 msgr2=0x7f419019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f4194ea1640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4190107320 0x7f419019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f4194ea1640 1 -- 172.21.15.32:0/89175794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4190102250 msgr2=0x7f419019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f4194ea1640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4190102250 0x7f419019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f4194ea1640 1 -- 172.21.15.32:0/89175794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41901992f0 con 0x7f41901986f0 2024-04-16T05:36:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f418f7fe640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4190107320 0x7f419019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f4194ea1640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41901986f0 0x7f4190198b70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f418800cad0 tx=0x7f418800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f418d7fa640 1 -- 172.21.15.32:0/89175794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418800ae20 con 0x7f41901986f0 2024-04-16T05:36:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f4196904640 1 -- 172.21.15.32:0/89175794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41901a53d0 con 0x7f41901986f0 2024-04-16T05:36:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f418d7fa640 1 -- 172.21.15.32:0/89175794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4188005020 con 0x7f41901986f0 2024-04-16T05:36:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f418d7fa640 1 -- 172.21.15.32:0/89175794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418800f6f0 con 0x7f41901986f0 2024-04-16T05:36:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.021+0000 7f4196904640 1 -- 172.21.15.32:0/89175794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41901a5780 con 0x7f41901986f0 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f418d7fa640 1 -- 172.21.15.32:0/89175794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41880051d0 con 0x7f41901986f0 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f41995c7740 1 Processor -- start 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f41995c7740 1 -- start start 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f41995c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b5e60 0x55605a1b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f41995c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f41995c7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1bf360 0x55605a1c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f41995c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55605a11b0e0 con 0x55605a1b5e60 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f41995c7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55605a0b8520 con 0x55605a1bf360 2024-04-16T05:36:38.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f41995c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55605a1170a0 con 0x55605a1b6780 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b5e60 0x55605a1b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b5e60 0x55605a1b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60810/0 (socket says 172.21.15.32:60810) 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f418f7fe640 1 -- 172.21.15.32:0/2982518845 learned_addr learned my addr 172.21.15.32:0/2982518845 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.024+0000 7f418ffff640 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418d7fa640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f41780a3d20 0x7f41780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418f7fe640 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1bf360 0x55605a1c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f4194ea1640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f41780a3d20 0x7f41780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418f7fe640 1 -- 172.21.15.32:0/2982518845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 msgr2=0x55605a1bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418f7fe640 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1bee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418f7fe640 1 -- 172.21.15.32:0/2982518845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1bf360 msgr2=0x55605a1c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418f7fe640 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1bf360 0x55605a1c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418f7fe640 1 -- 172.21.15.32:0/2982518845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605a0e08d0 con 0x55605a1b5e60 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418d7fa640 1 -- 172.21.15.32:0/89175794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f41880d0b30 con 0x7f41901986f0 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418ffff640 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f418f7fe640 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b5e60 0x55605a1b6240 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f41840137a0 tx=0x7f41840096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5df2fb747f1794a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f4194ea1640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f41780a3d20 0x7f41780a61e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41880ce540 tx=0x7f4188005960 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f41767fc640 1 -- 172.21.15.32:0/2982518845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41840110d0 con 0x55605a1b5e60 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f41767fc640 1 -- 172.21.15.32:0/2982518845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184013d20 con 0x55605a1b5e60 2024-04-16T05:36:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.025+0000 7f41767fc640 1 -- 172.21.15.32:0/2982518845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184004d90 con 0x55605a1b5e60 2024-04-16T05:36:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 -- 172.21.15.32:0/2982518845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b5e60 msgr2=0x55605a1b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b5e60 0x55605a1b6240 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f41840137a0 tx=0x7f41840096f0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 -- 172.21.15.32:0/2982518845 shutdown_connections 2024-04-16T05:36:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1bf360 0x55605a1c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 --2- 172.21.15.32:0/2982518845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b5e60 0x55605a1b6240 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 -- 172.21.15.32:0/2982518845 >> 172.21.15.32:0/2982518845 conn(0x55605a1b0590 msgr2=0x55605a1b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 -- 172.21.15.32:0/2982518845 shutdown_connections 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 -- 172.21.15.32:0/2982518845 wait complete. 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.026+0000 7f41995c7740 1 -- start start 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f41995c7740 1 Processor -- start 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f41995c7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 0x55605a1bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f41995c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b6780 0x55605a1bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f41995c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1bf360 0x55605a1bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f41995c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55605a1c1fa0 con 0x55605a1b6780 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f41995c7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55605a0d44d0 con 0x55605a1b5e60 2024-04-16T05:36:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f41995c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55605a1c2120 con 0x55605a1bf360 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f41995c7740 10 client.0 _ll_register_callbacks cb 0x55605a0b92c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b6780 0x55605a1bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b6780 0x55605a1bc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60818/0 (socket says 172.21.15.32:60818) 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 -- 172.21.15.32:0/2690177160 learned_addr learned my addr 172.21.15.32:0/2690177160 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418f7fe640 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1bf360 0x55605a1bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418f7fe640 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 0x55605a1bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 -- 172.21.15.32:0/2690177160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1bf360 msgr2=0x55605a1bce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1bf360 0x55605a1bce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 -- 172.21.15.32:0/2690177160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 msgr2=0x55605a1bc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 0x55605a1bc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 -- 172.21.15.32:0/2690177160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556059e2d3f0 con 0x55605a1b6780 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f418ffff640 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b6780 0x55605a1bc910 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f418002cfd0 tx=0x7f4180004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8189e45e01ba6d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.027+0000 7f416f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f416f7fe640 1 -- 172.21.15.32:0/2690177160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41800043f0 con 0x55605a1b6780 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f416f7fe640 1 -- 172.21.15.32:0/2690177160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f41800046c0 con 0x55605a1b6780 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 1 -- 172.21.15.32:0/2690177160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4184013d20 con 0x55605a1b6780 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f416f7fe640 1 -- 172.21.15.32:0/2690177160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41800048c0 con 0x55605a1b6780 2024-04-16T05:36:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 10 client.16095 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 1 -- 172.21.15.32:0/2690177160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556059ef77d0 con 0x55605a1b6780 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f416f7fe640 1 -- 172.21.15.32:0/2690177160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184013d20 con 0x55605a1b6780 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f416f7fe640 1 -- 172.21.15.32:0/2690177160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f41800036f0 con 0x55605a1b6780 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 10 client.16095 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 20 client.16095 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55605a1cf2a0 0x55605a1d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 4 client.16095 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f41995c7740 1 -- 172.21.15.32:0/2690177160 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55605a128b00 con 0x55605a1cf2a0 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.028+0000 7f418f7fe640 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55605a1cf2a0 0x55605a1d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.029+0000 7f416f7fe640 10 client.16095 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.029+0000 7f418f7fe640 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55605a1cf2a0 0x55605a1d16d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443836c4fc47f085 server_cookie=d7237ad3581f28ce in_seq=0 out_seq=0 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.029+0000 7f416f7fe640 1 -- 172.21.15.32:0/2690177160 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55605a128b00 con 0x55605a1cf2a0 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.029+0000 7f416f7fe640 10 client.16095 handle_command_reply: tid=0 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 client.16095 shutdown 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 -- 172.21.15.32:0/2690177160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b6780 msgr2=0x55605a1bc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b6780 0x55605a1bc910 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f418002cfd0 tx=0x7f4180004040 comp rx=0 tx=0).stop 2024-04-16T05:36:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 -- 172.21.15.32:0/2690177160 shutdown_connections 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55605a1cf2a0 0x55605a1d16d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1bf360 0x55605a1bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1b6780 0x55605a1bc910 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 --2- 172.21.15.32:0/2690177160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 0x55605a1bc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f416f7fe640 0 client.16095 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 -- 172.21.15.32:0/2690177160 >> 172.21.15.32:0/2690177160 conn(0x55605a1b0590 msgr2=0x55605a1b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 -- 172.21.15.32:0/2690177160 shutdown_connections 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 1 -- 172.21.15.32:0/2690177160 wait complete. 2024-04-16T05:36:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.030+0000 7f41995c7740 20 client.16095 trim_cache size 0 max 16384 2024-04-16T05:36:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.043+0000 7f41995c7740 1 Processor -- start 2024-04-16T05:36:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.043+0000 7f41995c7740 1 -- start start 2024-04-16T05:36:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.043+0000 7f41995c7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 0x55605a1bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.043+0000 7f41995c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.043+0000 7f41995c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.043+0000 7f41995c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f41901b1b90 con 0x55605a1bf360 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.043+0000 7f41995c7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f41901af460 con 0x55605a1b5e60 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.043+0000 7f41995c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41901ad2b0 con 0x55605a1b6780 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1ab5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60830/0 (socket says 172.21.15.32:60830) 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 -- 172.21.15.32:0/180846979 learned_addr learned my addr 172.21.15.32:0/180846979 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418ffff640 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 0x55605a1bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 -- 172.21.15.32:0/180846979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 msgr2=0x55605a1caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 -- 172.21.15.32:0/180846979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 msgr2=0x55605a1bea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 0x55605a1bea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 -- 172.21.15.32:0/180846979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605a1bb380 con 0x55605a1bf360 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f418f7fe640 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1ab5c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f41840049e0 tx=0x7f4184004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5406edd9d61a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f4175ffb640 1 -- 172.21.15.32:0/180846979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184009a20 con 0x55605a1bf360 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f4175ffb640 1 -- 172.21.15.32:0/180846979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184013d20 con 0x55605a1bf360 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.044+0000 7f4175ffb640 1 -- 172.21.15.32:0/180846979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184004d00 con 0x55605a1bf360 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 -- 172.21.15.32:0/180846979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 msgr2=0x55605a1ab5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1ab5c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f41840049e0 tx=0x7f4184004a10 comp rx=0 tx=0).stop 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 -- 172.21.15.32:0/180846979 shutdown_connections 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1ab5c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b6780 0x55605a1caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 --2- 172.21.15.32:0/180846979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b5e60 0x55605a1bea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 -- 172.21.15.32:0/180846979 >> 172.21.15.32:0/180846979 conn(0x55605a1b0590 msgr2=0x55605a1af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 -- 172.21.15.32:0/180846979 shutdown_connections 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 -- 172.21.15.32:0/180846979 wait complete. 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 -- start start 2024-04-16T05:36:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.045+0000 7f41995c7740 1 Processor -- start 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f41995c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b5e60 0x55605a1c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f41995c7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b6780 0x55605a1c3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f41995c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f41995c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f41901ad920 con 0x55605a1bf360 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f41995c7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f41901aed90 con 0x55605a1b6780 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f41995c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41901adff0 con 0x55605a1b5e60 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f41995c7740 10 client.0 _ll_register_callbacks cb 0x7f4180009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1c39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60834/0 (socket says 172.21.15.32:60834) 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 -- 172.21.15.32:0/26680818 learned_addr learned my addr 172.21.15.32:0/26680818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b5e60 0x55605a1c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418ffff640 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b6780 0x55605a1c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 -- 172.21.15.32:0/26680818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b5e60 msgr2=0x55605a1c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b5e60 0x55605a1c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 -- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b6780 msgr2=0x55605a1c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b6780 0x55605a1c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 -- 172.21.15.32:0/26680818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605a234890 con 0x55605a1bf360 2024-04-16T05:36:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418ffff640 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b6780 0x55605a1c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f418f7fe640 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1c39c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4184011330 tx=0x7f4184009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b4e103bf6e8083 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f416f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.046+0000 7f416f7fe640 1 -- 172.21.15.32:0/26680818 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41840132f0 con 0x55605a1bf360 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f416f7fe640 1 -- 172.21.15.32:0/26680818 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4184031410 con 0x55605a1bf360 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 1 -- 172.21.15.32:0/26680818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4184013d20 con 0x55605a1bf360 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f416f7fe640 1 -- 172.21.15.32:0/26680818 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4184031610 con 0x55605a1bf360 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 10 client.16101 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 1 -- 172.21.15.32:0/26680818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55605a235b70 con 0x55605a1bf360 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f416f7fe640 1 -- 172.21.15.32:0/26680818 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184013d20 con 0x55605a1bf360 2024-04-16T05:36:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f416f7fe640 1 -- 172.21.15.32:0/26680818 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f418400f600 con 0x55605a1bf360 2024-04-16T05:36:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 10 client.16101 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 20 client.16101 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55605a236020 0x55605a238440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 4 client.16101 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:36:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f41995c7740 1 -- 172.21.15.32:0/26680818 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55605a122970 con 0x55605a236020 2024-04-16T05:36:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.047+0000 7f418ffff640 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55605a236020 0x55605a238440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f418ffff640 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55605a236020 0x55605a238440 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e7e5ff69bae590 server_cookie=d6ef6adf24e8b011 in_seq=0 out_seq=0 2024-04-16T05:36:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f416f7fe640 10 client.16101 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f416f7fe640 1 -- 172.21.15.32:0/26680818 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55605a122970 con 0x55605a236020 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f416f7fe640 10 client.16101 handle_command_reply: tid=0 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 client.16101 shutdown 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 -- 172.21.15.32:0/26680818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 msgr2=0x55605a1c39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1c39c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4184011330 tx=0x7f4184009de0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 -- 172.21.15.32:0/26680818 shutdown_connections 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55605a1bf360 0x55605a1c39c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55605a1b6780 0x55605a1c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55605a236020 0x55605a238440 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 --2- 172.21.15.32:0/26680818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55605a1b5e60 0x55605a1c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f416f7fe640 0 client.16101 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.048+0000 7f41995c7740 1 -- 172.21.15.32:0/26680818 >> 172.21.15.32:0/26680818 conn(0x55605a1b0590 msgr2=0x55605a1b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f41995c7740 1 -- 172.21.15.32:0/26680818 shutdown_connections 2024-04-16T05:36:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f41995c7740 1 -- 172.21.15.32:0/26680818 wait complete. 2024-04-16T05:36:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f41995c7740 20 client.16101 trim_cache size 0 max 16384 2024-04-16T05:36:38.056 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:36:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 -- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f41780a3d20 msgr2=0x7f41780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f41780a3d20 0x7f41780a61e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41880ce540 tx=0x7f4188005960 comp rx=0 tx=0).stop 2024-04-16T05:36:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 -- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41901986f0 msgr2=0x7f4190198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41901986f0 0x7f4190198b70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f418800cad0 tx=0x7f418800cf90 comp rx=0 tx=0).stop 2024-04-16T05:36:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 -- 172.21.15.32:0/89175794 shutdown_connections 2024-04-16T05:36:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f41780a3d20 0x7f41780a61e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41901986f0 0x7f4190198b70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4190107320 0x7f419019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 --2- 172.21.15.32:0/89175794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4190102250 0x7f419019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 -- 172.21.15.32:0/89175794 >> 172.21.15.32:0/89175794 conn(0x7f4190079490 msgr2=0x7f41900fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 -- 172.21.15.32:0/89175794 shutdown_connections 2024-04-16T05:36:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:38.049+0000 7f4196904640 1 -- 172.21.15.32:0/89175794 wait complete. 2024-04-16T05:36:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:37 smithi134 ceph-mon[36153]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:37 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/213657352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:38.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:37 smithi082 ceph-mon[36195]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-16T05:36:38.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:37 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/213657352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:36:39.060 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:36:39.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:36:39.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:36:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 -- 172.21.15.32:0/2658645191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102bf0 msgr2=0x7f2358106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 --2- 172.21.15.32:0/2658645191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102bf0 0x7f2358106c30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2348009900 tx=0x7f234802bc30 comp rx=0 tx=0).stop 2024-04-16T05:36:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 -- 172.21.15.32:0/2658645191 shutdown_connections 2024-04-16T05:36:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 --2- 172.21.15.32:0/2658645191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2358107310 0x7f235810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 --2- 172.21.15.32:0/2658645191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102bf0 0x7f2358106c30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 --2- 172.21.15.32:0/2658645191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2358102240 0x7f2358102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 -- 172.21.15.32:0/2658645191 >> 172.21.15.32:0/2658645191 conn(0x7f2358079450 msgr2=0x7f2358079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 -- 172.21.15.32:0/2658645191 shutdown_connections 2024-04-16T05:36:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.185+0000 7f235fe0d640 1 -- 172.21.15.32:0/2658645191 wait complete. 2024-04-16T05:36:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235fe0d640 1 Processor -- start 2024-04-16T05:36:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235fe0d640 1 -- start start 2024-04-16T05:36:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235fe0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102240 0x7f235819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235fe0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2358107310 0x7f235819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235fe0d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2358198740 0x7f2358198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235fe0d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23581123a0 con 0x7f2358102240 2024-04-16T05:36:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235fe0d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2358112220 con 0x7f2358107310 2024-04-16T05:36:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235fe0d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2358112520 con 0x7f2358198740 2024-04-16T05:36:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235dba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102240 0x7f235819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235dba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102240 0x7f235819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34396/0 (socket says 172.21.15.32:34396) 2024-04-16T05:36:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235dba9640 1 -- 172.21.15.32:0/1843815289 learned_addr learned my addr 172.21.15.32:0/1843815289 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235e3aa640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2358198740 0x7f2358198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235d3a8640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2358107310 0x7f235819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235dba9640 1 -- 172.21.15.32:0/1843815289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2358198740 msgr2=0x7f2358198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235dba9640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2358198740 0x7f2358198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235dba9640 1 -- 172.21.15.32:0/1843815289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2358107310 msgr2=0x7f235819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235dba9640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2358107310 0x7f235819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.186+0000 7f235dba9640 1 -- 172.21.15.32:0/1843815289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2358199340 con 0x7f2358102240 2024-04-16T05:36:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.187+0000 7f235d3a8640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2358107310 0x7f235819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.187+0000 7f235dba9640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102240 0x7f235819e680 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f235400ea80 tx=0x7f235400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.187+0000 7f234effd640 1 -- 172.21.15.32:0/1843815289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235400ce40 con 0x7f2358102240 2024-04-16T05:36:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.187+0000 7f234effd640 1 -- 172.21.15.32:0/1843815289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2354005020 con 0x7f2358102240 2024-04-16T05:36:39.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.187+0000 7f234effd640 1 -- 172.21.15.32:0/1843815289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354011760 con 0x7f2358102240 2024-04-16T05:36:39.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.187+0000 7f235fe0d640 1 -- 172.21.15.32:0/1843815289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23581a5420 con 0x7f2358102240 2024-04-16T05:36:39.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.187+0000 7f235fe0d640 1 -- 172.21.15.32:0/1843815289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23581a5820 con 0x7f2358102240 2024-04-16T05:36:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.189+0000 7f234effd640 1 -- 172.21.15.32:0/1843815289 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23540051d0 con 0x7f2358102240 2024-04-16T05:36:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f23629f3740 1 Processor -- start 2024-04-16T05:36:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f23629f3740 1 -- start start 2024-04-16T05:36:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f23629f3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f23629f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 0x5579a13bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f23629f3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f23629f3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5579a131be90 con 0x5579a13b6ab0 2024-04-16T05:36:39.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f23629f3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5579a12b94c0 con 0x5579a13bffb0 2024-04-16T05:36:39.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f23629f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5579a1317e50 con 0x5579a13b73d0 2024-04-16T05:36:39.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f235e3aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f235e3aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13b6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34408/0 (socket says 172.21.15.32:34408) 2024-04-16T05:36:39.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f235e3aa640 1 -- 172.21.15.32:0/276544730 learned_addr learned my addr 172.21.15.32:0/276544730 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.190+0000 7f234effd640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f23440a3da0 0x7f23440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235d3a8640 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235dba9640 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 0x5579a13bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235e3aa640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f23440a3da0 0x7f23440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f234effd640 1 -- 172.21.15.32:0/1843815289 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f23540d11d0 con 0x7f2358102240 2024-04-16T05:36:39.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235e3aa640 1 -- 172.21.15.32:0/276544730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 msgr2=0x5579a13bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235e3aa640 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 0x5579a13bfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235e3aa640 1 -- 172.21.15.32:0/276544730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 msgr2=0x5579a13c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235e3aa640 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235e3aa640 1 -- 172.21.15.32:0/276544730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a10f67e0 con 0x5579a13b6ab0 2024-04-16T05:36:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f235e3aa640 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13b6e90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2350015620 tx=0x7f2350015ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b6472db91fe2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f2343fff640 1 -- 172.21.15.32:0/276544730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350022070 con 0x5579a13b6ab0 2024-04-16T05:36:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f2343fff640 1 -- 172.21.15.32:0/276544730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2350009bc0 con 0x5579a13b6ab0 2024-04-16T05:36:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.191+0000 7f2343fff640 1 -- 172.21.15.32:0/276544730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235000f9c0 con 0x5579a13b6ab0 2024-04-16T05:36:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 -- 172.21.15.32:0/276544730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 msgr2=0x5579a13b6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f235e3aa640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f23440a3da0 0x7f23440a6260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f235001f000 tx=0x7f235001b000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13b6e90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2350015620 tx=0x7f2350015ae0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 -- 172.21.15.32:0/276544730 shutdown_connections 2024-04-16T05:36:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 0x5579a13bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 --2- 172.21.15.32:0/276544730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13b6e90 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 -- 172.21.15.32:0/276544730 >> 172.21.15.32:0/276544730 conn(0x5579a13b11e0 msgr2=0x5579a13b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 -- 172.21.15.32:0/276544730 shutdown_connections 2024-04-16T05:36:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 -- 172.21.15.32:0/276544730 wait complete. 2024-04-16T05:36:39.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.192+0000 7f23629f3740 1 -- start start 2024-04-16T05:36:39.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f23629f3740 1 Processor -- start 2024-04-16T05:36:39.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f23629f3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f23629f3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13b73d0 0x5579a13bd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f23629f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13bffb0 0x5579a13bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f23629f3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5579a13c2da0 con 0x5579a13b6ab0 2024-04-16T05:36:39.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f23629f3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5579a12a4410 con 0x5579a13b73d0 2024-04-16T05:36:39.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f23629f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5579a13c2f20 con 0x5579a13bffb0 2024-04-16T05:36:39.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f23629f3740 10 client.0 _ll_register_callbacks cb 0x5579a129fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:39.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235dba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235dba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34424/0 (socket says 172.21.15.32:34424) 2024-04-16T05:36:39.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235dba9640 1 -- 172.21.15.32:0/648062958 learned_addr learned my addr 172.21.15.32:0/648062958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235e3aa640 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13bffb0 0x5579a13bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235d3a8640 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13b73d0 0x5579a13bd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235dba9640 1 -- 172.21.15.32:0/648062958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13bffb0 msgr2=0x5579a13bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235dba9640 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13bffb0 0x5579a13bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235dba9640 1 -- 172.21.15.32:0/648062958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13b73d0 msgr2=0x5579a13bd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235dba9640 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13b73d0 0x5579a13bd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235dba9640 1 -- 172.21.15.32:0/648062958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a1156f70 con 0x5579a13b6ab0 2024-04-16T05:36:39.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.193+0000 7f235e3aa640 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13bffb0 0x5579a13bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:39.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f235dba9640 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bcff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f23540cebe0 tx=0x7f235406b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63813de1e607f5fe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23417fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:39.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23417fa640 1 -- 172.21.15.32:0/648062958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235401f040 con 0x5579a13b6ab0 2024-04-16T05:36:39.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23417fa640 1 -- 172.21.15.32:0/648062958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f23540a5610 con 0x5579a13b6ab0 2024-04-16T05:36:39.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 1 -- 172.21.15.32:0/648062958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2350009bc0 con 0x5579a13b6ab0 2024-04-16T05:36:39.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23417fa640 1 -- 172.21.15.32:0/648062958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23540a11e0 con 0x5579a13b6ab0 2024-04-16T05:36:39.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 10 client.16113 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:39.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 1 -- 172.21.15.32:0/648062958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579a0fee170 con 0x5579a13b6ab0 2024-04-16T05:36:39.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23417fa640 1 -- 172.21.15.32:0/648062958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2350009bc0 con 0x5579a13b6ab0 2024-04-16T05:36:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23417fa640 1 -- 172.21.15.32:0/648062958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f235400f040 con 0x5579a13b6ab0 2024-04-16T05:36:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 10 client.16113 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 20 client.16113 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5579a13d0350 0x5579a13d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 4 client.16113 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.194+0000 7f23629f3740 1 -- 172.21.15.32:0/648062958 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579a13298b0 con 0x5579a13d0350 2024-04-16T05:36:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.195+0000 7f235d3a8640 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5579a13d0350 0x5579a13d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.195+0000 7f235d3a8640 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5579a13d0350 0x5579a13d2780 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e2434c93d7ccc81 server_cookie=48829e63ccd0043c in_seq=0 out_seq=0 2024-04-16T05:36:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.195+0000 7f23417fa640 10 client.16113 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23417fa640 1 -- 172.21.15.32:0/648062958 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579a13298b0 con 0x5579a13d0350 2024-04-16T05:36:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23417fa640 10 client.16113 handle_command_reply: tid=0 2024-04-16T05:36:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 client.16113 shutdown 2024-04-16T05:36:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 -- 172.21.15.32:0/648062958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 msgr2=0x5579a13bcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bcff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f23540cebe0 tx=0x7f235406b7e0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 -- 172.21.15.32:0/648062958 shutdown_connections 2024-04-16T05:36:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5579a13d0350 0x5579a13d2780 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13bffb0 0x5579a13bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13b73d0 0x5579a13bd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 --2- 172.21.15.32:0/648062958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bcff0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23417fa640 0 client.16113 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:39.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 -- 172.21.15.32:0/648062958 >> 172.21.15.32:0/648062958 conn(0x5579a13b11e0 msgr2=0x5579a13b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 -- 172.21.15.32:0/648062958 shutdown_connections 2024-04-16T05:36:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 1 -- 172.21.15.32:0/648062958 wait complete. 2024-04-16T05:36:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.196+0000 7f23629f3740 20 client.16113 trim_cache size 0 max 16384 2024-04-16T05:36:39.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f23629f3740 1 Processor -- start 2024-04-16T05:36:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f23629f3740 1 -- start start 2024-04-16T05:36:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f23629f3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f23629f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 0x5579a13b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f23629f3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f23629f3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b1cc0 con 0x5579a13b6ab0 2024-04-16T05:36:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f23629f3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f23581af590 con 0x5579a13bffb0 2024-04-16T05:36:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f23629f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23581ad3e0 con 0x5579a13b73d0 2024-04-16T05:36:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f235dba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.214+0000 7f235dba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34430/0 (socket says 172.21.15.32:34430) 2024-04-16T05:36:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235dba9640 1 -- 172.21.15.32:0/1344760854 learned_addr learned my addr 172.21.15.32:0/1344760854 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235e3aa640 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235d3a8640 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 0x5579a13b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235dba9640 1 -- 172.21.15.32:0/1344760854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 msgr2=0x5579a13b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235dba9640 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 0x5579a13b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235dba9640 1 -- 172.21.15.32:0/1344760854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 msgr2=0x5579a13bd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235dba9640 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13bd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235dba9640 1 -- 172.21.15.32:0/1344760854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a13af660 con 0x5579a13b6ab0 2024-04-16T05:36:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f235dba9640 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bf720 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f235400efd0 tx=0x7f2354005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e8d60892d6fba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f23437fe640 1 -- 172.21.15.32:0/1344760854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235401f050 con 0x5579a13b6ab0 2024-04-16T05:36:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f23437fe640 1 -- 172.21.15.32:0/1344760854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23540a5aa0 con 0x5579a13b6ab0 2024-04-16T05:36:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.215+0000 7f23437fe640 1 -- 172.21.15.32:0/1344760854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23540a5870 con 0x5579a13b6ab0 2024-04-16T05:36:39.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 -- 172.21.15.32:0/1344760854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 msgr2=0x5579a13bf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bf720 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f235400efd0 tx=0x7f2354005750 comp rx=0 tx=0).stop 2024-04-16T05:36:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 -- 172.21.15.32:0/1344760854 shutdown_connections 2024-04-16T05:36:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13bd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b73d0 0x5579a13b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 --2- 172.21.15.32:0/1344760854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b6ab0 0x5579a13bf720 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 -- 172.21.15.32:0/1344760854 >> 172.21.15.32:0/1344760854 conn(0x5579a13b11e0 msgr2=0x5579a13b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 -- 172.21.15.32:0/1344760854 shutdown_connections 2024-04-16T05:36:39.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 -- 172.21.15.32:0/1344760854 wait complete. 2024-04-16T05:36:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 -- start start 2024-04-16T05:36:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.216+0000 7f23629f3740 1 Processor -- start 2024-04-16T05:36:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23629f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b6ab0 0x5579a13c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23629f3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b73d0 0x5579a13c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23629f3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23629f3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23581ada50 con 0x5579a13b73d0 2024-04-16T05:36:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23629f3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f23581aeec0 con 0x5579a13bffb0 2024-04-16T05:36:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23629f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23581ae120 con 0x5579a13b6ab0 2024-04-16T05:36:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23629f3740 10 client.0 _ll_register_callbacks cb 0x7f2354067bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b73d0 0x5579a13c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b73d0 0x5579a13c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34440/0 (socket says 172.21.15.32:34440) 2024-04-16T05:36:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 -- 172.21.15.32:0/3665471352 learned_addr learned my addr 172.21.15.32:0/3665471352 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235e3aa640 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 -- 172.21.15.32:0/3665471352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b6ab0 msgr2=0x5579a13c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235dba9640 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b6ab0 0x5579a13c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b6ab0 0x5579a13c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 -- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 msgr2=0x5579a13c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 -- 172.21.15.32:0/3665471352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a13c5910 con 0x5579a13b73d0 2024-04-16T05:36:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235e3aa640 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f235d3a8640 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b73d0 0x5579a13c86b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f234802cfd0 tx=0x7f234802cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cae35b10fd6cfe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23417fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23417fa640 1 -- 172.21.15.32:0/3665471352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234802cd60 con 0x5579a13b73d0 2024-04-16T05:36:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23417fa640 1 -- 172.21.15.32:0/3665471352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2348004250 con 0x5579a13b73d0 2024-04-16T05:36:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.217+0000 7f23629f3740 1 -- 172.21.15.32:0/3665471352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23540a5aa0 con 0x5579a13b73d0 2024-04-16T05:36:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23417fa640 1 -- 172.21.15.32:0/3665471352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2348004450 con 0x5579a13b73d0 2024-04-16T05:36:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23629f3740 10 client.16119 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23629f3740 1 -- 172.21.15.32:0/3665471352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579a1437320 con 0x5579a13b73d0 2024-04-16T05:36:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23417fa640 1 -- 172.21.15.32:0/3665471352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23540a5aa0 con 0x5579a13b73d0 2024-04-16T05:36:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23417fa640 1 -- 172.21.15.32:0/3665471352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2348004c60 con 0x5579a13b73d0 2024-04-16T05:36:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23629f3740 10 client.16119 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23629f3740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23629f3740 20 client.16119 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23629f3740 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5579a1437ad0 0x5579a1439ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23629f3740 4 client.16119 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:36:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f23629f3740 1 -- 172.21.15.32:0/3665471352 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5579a1323720 con 0x5579a1437ad0 2024-04-16T05:36:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.218+0000 7f235dba9640 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5579a1437ad0 0x5579a1439ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.219+0000 7f23417fa640 10 client.16119 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.219+0000 7f235dba9640 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5579a1437ad0 0x5579a1439ec0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=378b4241d129a95a server_cookie=ba7cf9795bcc161a in_seq=0 out_seq=0 2024-04-16T05:36:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23417fa640 1 -- 172.21.15.32:0/3665471352 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5579a1323720 con 0x5579a1437ad0 2024-04-16T05:36:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23417fa640 10 client.16119 handle_command_reply: tid=0 2024-04-16T05:36:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 client.16119 shutdown 2024-04-16T05:36:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 -- 172.21.15.32:0/3665471352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b73d0 msgr2=0x5579a13c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b73d0 0x5579a13c86b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f234802cfd0 tx=0x7f234802cc30 comp rx=0 tx=0).stop 2024-04-16T05:36:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 -- 172.21.15.32:0/3665471352 shutdown_connections 2024-04-16T05:36:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579a13bffb0 0x5579a13c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5579a13b73d0 0x5579a13c86b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5579a1437ad0 0x5579a1439ec0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 --2- 172.21.15.32:0/3665471352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579a13b6ab0 0x5579a13c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 -- 172.21.15.32:0/3665471352 >> 172.21.15.32:0/3665471352 conn(0x5579a13b11e0 msgr2=0x5579a13b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23417fa640 0 client.16119 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 -- 172.21.15.32:0/3665471352 shutdown_connections 2024-04-16T05:36:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 1 -- 172.21.15.32:0/3665471352 wait complete. 2024-04-16T05:36:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.220+0000 7f23629f3740 20 client.16119 trim_cache size 0 max 16384 2024-04-16T05:36:39.230 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:36:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:36:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "c3ecdf43-d82d-4068-a8f0-ec5206f861df", 2024-04-16T05:36:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:36:39.231 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:36:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 -- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f23440a3da0 msgr2=0x7f23440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f23440a3da0 0x7f23440a6260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f235001f000 tx=0x7f235001b000 comp rx=0 tx=0).stop 2024-04-16T05:36:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 -- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102240 msgr2=0x7f235819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102240 0x7f235819e680 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f235400ea80 tx=0x7f235400ef40 comp rx=0 tx=0).stop 2024-04-16T05:36:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 -- 172.21.15.32:0/1843815289 shutdown_connections 2024-04-16T05:36:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2358198740 0x7f2358198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2358107310 0x7f235819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f23440a3da0 0x7f23440a6260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 --2- 172.21.15.32:0/1843815289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2358102240 0x7f235819e680 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 -- 172.21.15.32:0/1843815289 >> 172.21.15.32:0/1843815289 conn(0x7f2358079450 msgr2=0x7f2358100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 -- 172.21.15.32:0/1843815289 shutdown_connections 2024-04-16T05:36:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.221+0000 7f235fe0d640 1 -- 172.21.15.32:0/1843815289 wait complete. 2024-04-16T05:36:39.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:36:39.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:36:39.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.343+0000 7f2969131640 1 -- 172.21.15.32:0/2553352880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 msgr2=0x7f2964103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.343+0000 7f2969131640 1 --2- 172.21.15.32:0/2553352880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 0x7f2964103660 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f295c009ca0 tx=0x7f295c02c230 comp rx=0 tx=0).stop 2024-04-16T05:36:39.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.343+0000 7f2969131640 1 -- 172.21.15.32:0/2553352880 shutdown_connections 2024-04-16T05:36:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.343+0000 7f2969131640 1 --2- 172.21.15.32:0/2553352880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f296410c5c0 0x7f296410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.343+0000 7f2969131640 1 --2- 172.21.15.32:0/2553352880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964103ba0 0x7f296410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.343+0000 7f2969131640 1 --2- 172.21.15.32:0/2553352880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 0x7f2964103660 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.343+0000 7f2969131640 1 -- 172.21.15.32:0/2553352880 >> 172.21.15.32:0/2553352880 conn(0x7f2964078000 msgr2=0x7f2964078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.344+0000 7f2969131640 1 -- 172.21.15.32:0/2553352880 shutdown_connections 2024-04-16T05:36:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.344+0000 7f2969131640 1 -- 172.21.15.32:0/2553352880 wait complete. 2024-04-16T05:36:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.344+0000 7f2969131640 1 Processor -- start 2024-04-16T05:36:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.344+0000 7f2969131640 1 -- start start 2024-04-16T05:36:39.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2969131640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 0x7f296419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2969131640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964103ba0 0x7f296419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2969131640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f296410c5c0 0x7f2964198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2969131640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2964112380 con 0x7f2964103280 2024-04-16T05:36:39.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2969131640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2964112200 con 0x7f296410c5c0 2024-04-16T05:36:39.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2969131640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2964112500 con 0x7f2964103ba0 2024-04-16T05:36:39.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 0x7f296419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 0x7f296419e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34456/0 (socket says 172.21.15.32:34456) 2024-04-16T05:36:39.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 -- 172.21.15.32:0/334076341 learned_addr learned my addr 172.21.15.32:0/334076341 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f296259c640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964103ba0 0x7f296419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 -- 172.21.15.32:0/334076341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964103ba0 msgr2=0x7f296419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f296359e640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f296410c5c0 0x7f2964198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964103ba0 0x7f296419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 -- 172.21.15.32:0/334076341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f296410c5c0 msgr2=0x7f2964198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f296410c5c0 0x7f2964198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 -- 172.21.15.32:0/334076341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2964198f20 con 0x7f2964103280 2024-04-16T05:36:39.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2962d9d640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 0x7f296419e5a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f295c005e30 tx=0x7f295c035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f295bfff640 1 -- 172.21.15.32:0/334076341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c039040 con 0x7f2964103280 2024-04-16T05:36:39.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2969131640 1 -- 172.21.15.32:0/334076341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29641991b0 con 0x7f2964103280 2024-04-16T05:36:39.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f295bfff640 1 -- 172.21.15.32:0/334076341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f295c044070 con 0x7f2964103280 2024-04-16T05:36:39.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f295bfff640 1 -- 172.21.15.32:0/334076341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c004260 con 0x7f2964103280 2024-04-16T05:36:39.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.345+0000 7f2969131640 1 -- 172.21.15.32:0/334076341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29641a5340 con 0x7f2964103280 2024-04-16T05:36:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.348+0000 7f296bdf1740 1 Processor -- start 2024-04-16T05:36:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.348+0000 7f295bfff640 1 -- 172.21.15.32:0/334076341 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f295c004440 con 0x7f2964103280 2024-04-16T05:36:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.348+0000 7f296bdf1740 1 -- start start 2024-04-16T05:36:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdee50 0x55b4c6bdf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdf770 0x55b4c6be7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6bea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296bdf1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b4c6b44180 con 0x55b4c6be8350 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296bdf1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b4c6ae1360 con 0x55b4c6bdee50 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296bdf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b4c6b40140 con 0x55b4c6bdf770 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6bea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6bea740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34460/0 (socket says 172.21.15.32:34460) 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296259c640 1 -- 172.21.15.32:0/1903438257 learned_addr learned my addr 172.21.15.32:0/1903438257 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f2962d9d640 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdf770 0x55b4c6be7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296359e640 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdee50 0x55b4c6bdf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296259c640 1 -- 172.21.15.32:0/1903438257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdf770 msgr2=0x55b4c6be7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296259c640 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdf770 0x55b4c6be7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296259c640 1 -- 172.21.15.32:0/1903438257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdee50 msgr2=0x55b4c6bdf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f295bfff640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29500a3d20 0x7f29500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296259c640 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdee50 0x55b4c6bdf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296259c640 1 -- 172.21.15.32:0/1903438257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c69207d0 con 0x55b4c6be8350 2024-04-16T05:36:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.349+0000 7f296359e640 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdee50 0x55b4c6bdf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.350+0000 7f296359e640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29500a3d20 0x7f29500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.350+0000 7f295bfff640 1 -- 172.21.15.32:0/334076341 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f295c0f3170 con 0x7f2964103280 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.350+0000 7f296259c640 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6bea740 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f294c00cc40 tx=0x7f294c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f306281f66538306 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.350+0000 7f29597fa640 1 -- 172.21.15.32:0/1903438257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c014070 con 0x55b4c6be8350 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.350+0000 7f29597fa640 1 -- 172.21.15.32:0/1903438257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c0047a0 con 0x55b4c6be8350 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.350+0000 7f29597fa640 1 -- 172.21.15.32:0/1903438257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c010740 con 0x55b4c6be8350 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.350+0000 7f296bdf1740 1 -- 172.21.15.32:0/1903438257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 msgr2=0x55b4c6bea740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6bea740 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f294c00cc40 tx=0x7f294c00e720 comp rx=0 tx=0).stop 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.350+0000 7f296359e640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29500a3d20 0x7f29500a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2954009b80 tx=0x7f29540046c0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 -- 172.21.15.32:0/1903438257 shutdown_connections 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6bea740 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdf770 0x55b4c6be7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1903438257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdee50 0x55b4c6bdf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 -- 172.21.15.32:0/1903438257 >> 172.21.15.32:0/1903438257 conn(0x55b4c6bd9580 msgr2=0x55b4c6bd9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 -- 172.21.15.32:0/1903438257 shutdown_connections 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 -- 172.21.15.32:0/1903438257 wait complete. 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 -- start start 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 Processor -- start 2024-04-16T05:36:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6be5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6be58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6be5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b4c6beb140 con 0x55b4c6be8350 2024-04-16T05:36:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b4c6afd2c0 con 0x55b4c6bdf770 2024-04-16T05:36:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b4c6beb2c0 con 0x55b4c6bdee50 2024-04-16T05:36:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296bdf1740 10 client.0 _ll_register_callbacks cb 0x55b4c6ae4090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6be5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6be5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34466/0 (socket says 172.21.15.32:34466) 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.351+0000 7f296359e640 1 -- 172.21.15.32:0/2808443146 learned_addr learned my addr 172.21.15.32:0/2808443146 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296259c640 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6be58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f2962d9d640 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6be5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296359e640 1 -- 172.21.15.32:0/2808443146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 msgr2=0x55b4c6be5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296359e640 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6be5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296359e640 1 -- 172.21.15.32:0/2808443146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 msgr2=0x55b4c6be58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296359e640 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6be58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296359e640 1 -- 172.21.15.32:0/2808443146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c68563f0 con 0x55b4c6be8350 2024-04-16T05:36:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f2962d9d640 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6be5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296359e640 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6be5df0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2954013730 tx=0x7f2954004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b0f94f40be2285 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f294a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f294a7fc640 1 -- 172.21.15.32:0/2808443146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295401ea60 con 0x55b4c6be8350 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f294a7fc640 1 -- 172.21.15.32:0/2808443146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f295401ec00 con 0x55b4c6be8350 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296bdf1740 1 -- 172.21.15.32:0/2808443146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294c0047a0 con 0x55b4c6be8350 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f294a7fc640 1 -- 172.21.15.32:0/2808443146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2954031050 con 0x55b4c6be8350 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296bdf1740 10 client.16131 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f296bdf1740 1 -- 172.21.15.32:0/2808443146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b4c68160f0 con 0x55b4c6be8350 2024-04-16T05:36:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.352+0000 7f294a7fc640 1 -- 172.21.15.32:0/2808443146 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c0047a0 con 0x55b4c6be8350 2024-04-16T05:36:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f294a7fc640 1 -- 172.21.15.32:0/2808443146 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f295400f040 con 0x55b4c6be8350 2024-04-16T05:36:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f296bdf1740 10 client.16131 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f296bdf1740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f296bdf1740 20 client.16131 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f296bdf1740 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b4c6bf8770 0x55b4c6bfaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f296bdf1740 4 client.16131 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:36:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f296bdf1740 1 -- 172.21.15.32:0/2808443146 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4c6b51ba0 con 0x55b4c6bf8770 2024-04-16T05:36:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f2962d9d640 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b4c6bf8770 0x55b4c6bfaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f294a7fc640 10 client.16131 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.353+0000 7f2962d9d640 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b4c6bf8770 0x55b4c6bfaba0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44242f3636c68eb server_cookie=812fa2fd382ee430 in_seq=0 out_seq=0 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f294a7fc640 1 -- 172.21.15.32:0/2808443146 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4c6b51ba0 con 0x55b4c6bf8770 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f294a7fc640 10 client.16131 handle_command_reply: tid=0 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 client.16131 shutdown 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 -- 172.21.15.32:0/2808443146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 msgr2=0x55b4c6be5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6be5df0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2954013730 tx=0x7f2954004730 comp rx=0 tx=0).stop 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 -- 172.21.15.32:0/2808443146 shutdown_connections 2024-04-16T05:36:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6be8350 0x55b4c6be5df0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6be58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b4c6bf8770 0x55b4c6bfaba0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 --2- 172.21.15.32:0/2808443146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6be5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 -- 172.21.15.32:0/2808443146 >> 172.21.15.32:0/2808443146 conn(0x55b4c6bd9580 msgr2=0x55b4c6bdb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f294a7fc640 0 client.16131 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 -- 172.21.15.32:0/2808443146 shutdown_connections 2024-04-16T05:36:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 1 -- 172.21.15.32:0/2808443146 wait complete. 2024-04-16T05:36:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.354+0000 7f296bdf1740 20 client.16131 trim_cache size 0 max 16384 2024-04-16T05:36:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296bdf1740 1 Processor -- start 2024-04-16T05:36:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296bdf1740 1 -- start start 2024-04-16T05:36:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bd9580 0x55b4c6be7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6bf4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6bdb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296bdf1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29641b1b00 con 0x55b4c6bd9580 2024-04-16T05:36:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296bdf1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29641af3d0 con 0x55b4c6bdf770 2024-04-16T05:36:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296bdf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29641ad220 con 0x55b4c6bdee50 2024-04-16T05:36:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f2962d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bd9580 0x55b4c6be7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f2962d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bd9580 0x55b4c6be7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34472/0 (socket says 172.21.15.32:34472) 2024-04-16T05:36:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f2962d9d640 1 -- 172.21.15.32:0/1955338599 learned_addr learned my addr 172.21.15.32:0/1955338599 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296359e640 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6bdb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.372+0000 7f296259c640 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6bf4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2962d9d640 1 -- 172.21.15.32:0/1955338599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 msgr2=0x55b4c6bf4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2962d9d640 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6bf4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2962d9d640 1 -- 172.21.15.32:0/1955338599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 msgr2=0x55b4c6bdb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2962d9d640 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6bdb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2962d9d640 1 -- 172.21.15.32:0/1955338599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c6bda150 con 0x55b4c6bd9580 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f296259c640 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6bf4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2962d9d640 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bd9580 0x55b4c6be7aa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f295c0bee10 tx=0x7f295c0f6c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8bf1798905b8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2958ff9640 1 -- 172.21.15.32:0/1955338599 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c0be7e0 con 0x55b4c6bd9580 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2958ff9640 1 -- 172.21.15.32:0/1955338599 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f295c003dc0 con 0x55b4c6bd9580 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f2958ff9640 1 -- 172.21.15.32:0/1955338599 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c0f6d90 con 0x55b4c6bd9580 2024-04-16T05:36:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f296bdf1740 1 -- 172.21.15.32:0/1955338599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bd9580 msgr2=0x55b4c6be7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.373+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bd9580 0x55b4c6be7aa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f295c0bee10 tx=0x7f295c0f6c20 comp rx=0 tx=0).stop 2024-04-16T05:36:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 -- 172.21.15.32:0/1955338599 shutdown_connections 2024-04-16T05:36:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6bdb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bdee50 0x55b4c6bf4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1955338599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bd9580 0x55b4c6be7aa0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 -- 172.21.15.32:0/1955338599 >> 172.21.15.32:0/1955338599 conn(0x55b4c6be3fb0 msgr2=0x55b4c6bda9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 -- 172.21.15.32:0/1955338599 shutdown_connections 2024-04-16T05:36:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 -- 172.21.15.32:0/1955338599 wait complete. 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 -- start start 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 Processor -- start 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bd9580 0x55b4c6bee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bdee50 0x55b4c6bee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6beeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29641ad890 con 0x55b4c6bdee50 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29641aed00 con 0x55b4c6bdf770 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29641adf60 con 0x55b4c6bd9580 2024-04-16T05:36:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296bdf1740 10 client.0 _ll_register_callbacks cb 0x7f2954009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:36:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bdee50 0x55b4c6bee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bdee50 0x55b4c6bee5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34478/0 (socket says 172.21.15.32:34478) 2024-04-16T05:36:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296259c640 1 -- 172.21.15.32:0/1211498495 learned_addr learned my addr 172.21.15.32:0/1211498495 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:36:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f2962d9d640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bd9580 0x55b4c6bee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.374+0000 7f296359e640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6beeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296259c640 1 -- 172.21.15.32:0/1211498495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bd9580 msgr2=0x55b4c6bee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296259c640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bd9580 0x55b4c6bee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296259c640 1 -- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 msgr2=0x55b4c6beeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296259c640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6beeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296259c640 1 -- 172.21.15.32:0/1211498495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c6c5dda0 con 0x55b4c6bdee50 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f2962d9d640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bd9580 0x55b4c6bee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296359e640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6beeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296259c640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bdee50 0x55b4c6bee5a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55b4c6bda710 tx=0x7f294c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a302711d5eaf4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f294a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f294a7fc640 1 -- 172.21.15.32:0/1211498495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c028060 con 0x55b4c6bdee50 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f294a7fc640 1 -- 172.21.15.32:0/1211498495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f294c004890 con 0x55b4c6bdee50 2024-04-16T05:36:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296bdf1740 1 -- 172.21.15.32:0/1211498495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b4c6b4ba10 con 0x55b4c6bdee50 2024-04-16T05:36:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f294a7fc640 1 -- 172.21.15.32:0/1211498495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f294c02f050 con 0x55b4c6bdee50 2024-04-16T05:36:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296bdf1740 10 client.16137 fetch_fsmap learned FSMap version 25 2024-04-16T05:36:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296bdf1740 1 -- 172.21.15.32:0/1211498495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b4c6c5f030 con 0x55b4c6bdee50 2024-04-16T05:36:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f294a7fc640 1 -- 172.21.15.32:0/1211498495 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b4c6b4ba10 con 0x55b4c6bdee50 2024-04-16T05:36:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f294a7fc640 1 -- 172.21.15.32:0/1211498495 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f294c022070 con 0x55b4c6bdee50 2024-04-16T05:36:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296bdf1740 10 client.16137 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:36:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296bdf1740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:36:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.375+0000 7f296bdf1740 20 client.16137 populate_metadata read hostname 'smithi032' 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b4c6be8350 0x55b4c6c61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f296bdf1740 4 client.16137 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f296bdf1740 1 -- 172.21.15.32:0/1211498495 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4c6ac7560 con 0x55b4c6be8350 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f2962d9d640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b4c6be8350 0x55b4c6c61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f2962d9d640 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b4c6be8350 0x55b4c6c61570 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ac770ffcca4da7 server_cookie=19de1c3aa2a6b63e in_seq=0 out_seq=0 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f294a7fc640 10 client.16137 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f294a7fc640 1 -- 172.21.15.32:0/1211498495 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b4c6ac7560 con 0x55b4c6be8350 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f294a7fc640 10 client.16137 handle_command_reply: tid=0 2024-04-16T05:36:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.376+0000 7f296bdf1740 1 client.16137 shutdown 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 -- 172.21.15.32:0/1211498495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bdee50 msgr2=0x55b4c6bee5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bdee50 0x55b4c6bee5a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55b4c6bda710 tx=0x7f294c00c980 comp rx=0 tx=0).stop 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 -- 172.21.15.32:0/1211498495 shutdown_connections 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4c6bdf770 0x55b4c6beeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4c6bdee50 0x55b4c6bee5a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b4c6be8350 0x55b4c6c61570 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 --2- 172.21.15.32:0/1211498495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4c6bd9580 0x55b4c6bee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f294a7fc640 0 client.16137 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:36:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 -- 172.21.15.32:0/1211498495 >> 172.21.15.32:0/1211498495 conn(0x55b4c6be3fb0 msgr2=0x55b4c6bd8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 -- 172.21.15.32:0/1211498495 shutdown_connections 2024-04-16T05:36:39.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 1 -- 172.21.15.32:0/1211498495 wait complete. 2024-04-16T05:36:39.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f296bdf1740 20 client.16137 trim_cache size 0 max 16384 2024-04-16T05:36:39.386 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:36:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:36:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:36:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "c3ecdf43-d82d-4068-a8f0-ec5206f861df": { 2024-04-16T05:36:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:36:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "c3ecdf43-d82d-4068-a8f0-ec5206f861df", 2024-04-16T05:36:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:36:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:36:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:36:39.387 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:36:39.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 -- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29500a3d20 msgr2=0x7f29500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29500a3d20 0x7f29500a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2954009b80 tx=0x7f29540046c0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 -- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 msgr2=0x7f296419e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:36:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 0x7f296419e5a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f295c005e30 tx=0x7f295c035d90 comp rx=0 tx=0).stop 2024-04-16T05:36:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 -- 172.21.15.32:0/334076341 shutdown_connections 2024-04-16T05:36:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f296410c5c0 0x7f2964198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964103ba0 0x7f296419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29500a3d20 0x7f29500a61e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 --2- 172.21.15.32:0/334076341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2964103280 0x7f296419e5a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:36:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 -- 172.21.15.32:0/334076341 >> 172.21.15.32:0/334076341 conn(0x7f2964078000 msgr2=0x7f29640fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:36:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 -- 172.21.15.32:0/334076341 shutdown_connections 2024-04-16T05:36:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:36:39.377+0000 7f2969131640 1 -- 172.21.15.32:0/334076341 wait complete. 2024-04-16T05:36:39.390 INFO:tasks.cephfs.filesystem:scrub status for tag:c3ecdf43-d82d-4068-a8f0-ec5206f861df - {'path': '/', 'tag': 'c3ecdf43-d82d-4068-a8f0-ec5206f861df', 'options': 'recursive,force'} 2024-04-16T05:36:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:39 smithi134 ceph-mon[36153]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-16T05:36:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:39 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:36:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:39 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:39 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:36:40.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:39 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:40.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:39 smithi082 ceph-mon[36195]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-16T05:36:40.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:39 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:36:40.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:39 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:40.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:39 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:36:40.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:39 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:39 smithi032 ceph-mon[32351]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-16T05:36:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:39 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:36:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:39 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:39 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:36:40.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:39 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:36:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:40 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:36:41.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:40 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:36:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:40 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:36:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:41 smithi134 ceph-mon[36153]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-16T05:36:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:41 smithi082 ceph-mon[36195]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-16T05:36:42.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:41 smithi032 ceph-mon[32351]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-16T05:36:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:43 smithi134 ceph-mon[36153]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:36:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:43 smithi082 ceph-mon[36195]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:36:44.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:43 smithi032 ceph-mon[32351]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:36:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:45 smithi134 ceph-mon[36153]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:36:46.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:45 smithi082 ceph-mon[36195]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:36:46.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:45 smithi032 ceph-mon[32351]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:36:48.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:47 smithi134 ceph-mon[36153]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:36:48.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:47 smithi082 ceph-mon[36195]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:36:48.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:47 smithi032 ceph-mon[32351]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-16T05:36:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:49 smithi134 ceph-mon[36153]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:36:50.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:49 smithi082 ceph-mon[36195]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:36:50.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:49 smithi032 ceph-mon[32351]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-16T05:36:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:51 smithi134 ceph-mon[36153]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-04-16T05:36:52.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:51 smithi082 ceph-mon[36195]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-04-16T05:36:52.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:51 smithi032 ceph-mon[32351]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-04-16T05:36:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:53 smithi134 ceph-mon[36153]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:36:54.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:53 smithi082 ceph-mon[36195]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:36:54.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:53 smithi032 ceph-mon[32351]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-16T05:36:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:55 smithi134 ceph-mon[36153]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-16T05:36:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:55 smithi082 ceph-mon[36195]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-16T05:36:56.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:55 smithi032 ceph-mon[32351]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-16T05:36:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:57 smithi134 ceph-mon[36153]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-16T05:36:58.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:57 smithi082 ceph-mon[36195]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-16T05:36:58.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:57 smithi032 ceph-mon[32351]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-16T05:37:00.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:36:59 smithi134 ceph-mon[36153]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-16T05:37:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:36:59 smithi082 ceph-mon[36195]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-16T05:37:00.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:36:59 smithi032 ceph-mon[32351]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-16T05:37:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:01 smithi134 ceph-mon[36153]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-16T05:37:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:01 smithi082 ceph-mon[36195]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-16T05:37:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:01 smithi032 ceph-mon[32351]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-16T05:37:03.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:02 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:37:03.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:02 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:37:03.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:02 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:37:04.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:03 smithi134 ceph-mon[36153]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:37:04.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:03 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:04.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:03 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:04.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:03 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:37:04.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:03 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:37:04.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:03 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:03 smithi082 ceph-mon[36195]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:37:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:03 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:03 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:03 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:37:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:03 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:37:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:03 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:04.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:03 smithi032 ceph-mon[32351]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:37:04.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:03 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:04.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:03 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:04.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:03 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:37:04.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:03 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:37:04.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:03 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:37:06.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:05 smithi134 ceph-mon[36153]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:37:06.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:05 smithi082 ceph-mon[36195]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:37:06.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:05 smithi032 ceph-mon[32351]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:37:08.035 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:07 smithi032 ceph-mon[32351]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:37:08.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:07 smithi134 ceph-mon[36153]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:37:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:07 smithi082 ceph-mon[36195]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:37:09.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:37:09.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:37:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 -- 172.21.15.32:0/731339328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c106210 msgr2=0x7f8b5c10dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 --2- 172.21.15.32:0/731339328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c106210 0x7f8b5c10dde0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009ca0 tx=0x7f8b5402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 -- 172.21.15.32:0/731339328 shutdown_connections 2024-04-16T05:37:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 --2- 172.21.15.32:0/731339328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c106210 0x7f8b5c10dde0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 --2- 172.21.15.32:0/731339328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b5c101af0 0x7f8b5c105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 --2- 172.21.15.32:0/731339328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b5c101140 0x7f8b5c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 -- 172.21.15.32:0/731339328 >> 172.21.15.32:0/731339328 conn(0x7f8b5c078000 msgr2=0x7f8b5c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 -- 172.21.15.32:0/731339328 shutdown_connections 2024-04-16T05:37:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 -- 172.21.15.32:0/731339328 wait complete. 2024-04-16T05:37:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.514+0000 7f8b625f4640 1 Processor -- start 2024-04-16T05:37:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b625f4640 1 -- start start 2024-04-16T05:37:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b625f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b5c101140 0x7f8b5c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b625f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b5c106210 0x7f8b5c199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b625f4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c19a1e0 0x7f8b5c1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b625f4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1112d0 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b625f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c111150 con 0x7f8b5c106210 2024-04-16T05:37:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b625f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c111450 con 0x7f8b5c101140 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b60b91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c19a1e0 0x7f8b5c1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b60b91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c19a1e0 0x7f8b5c1a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36528/0 (socket says 172.21.15.32:36528) 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b60b91640 1 -- 172.21.15.32:0/120534923 learned_addr learned my addr 172.21.15.32:0/120534923 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b5c106210 0x7f8b5c199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b5c101140 0x7f8b5c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b60b91640 1 -- 172.21.15.32:0/120534923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b5c101140 msgr2=0x7f8b5c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b60b91640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b5c101140 0x7f8b5c199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b60b91640 1 -- 172.21.15.32:0/120534923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b5c106210 msgr2=0x7f8b5c199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b60b91640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b5c106210 0x7f8b5c199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.515+0000 7f8b60b91640 1 -- 172.21.15.32:0/120534923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c1a4f60 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.516+0000 7f8b60b91640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c19a1e0 0x7f8b5c1a4790 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8b5402c700 tx=0x7f8b54032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.516+0000 7f8b597fa640 1 -- 172.21.15.32:0/120534923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54041020 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.516+0000 7f8b597fa640 1 -- 172.21.15.32:0/120534923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b54034cd0 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.516+0000 7f8b597fa640 1 -- 172.21.15.32:0/120534923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54039040 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.516+0000 7f8b625f4640 1 -- 172.21.15.32:0/120534923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5c1a51f0 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.516+0000 7f8b625f4640 1 -- 172.21.15.32:0/120534923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5c1a56e0 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.518+0000 7f8b597fa640 1 -- 172.21.15.32:0/120534923 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b54044020 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.518+0000 7f8b651e6740 1 Processor -- start 2024-04-16T05:37:09.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.518+0000 7f8b651e6740 1 -- start start 2024-04-16T05:37:09.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b9770 0x5568309c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 0x5568309c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b651e6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55683091e180 con 0x5568309b9770 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b651e6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5568308bb360 con 0x5568309c2350 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b651e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55683091a140 con 0x5568309b8e50 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b597fa640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b440a3d20 0x7f8b440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b9770 0x5568309c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b9770 0x5568309c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36534/0 (socket says 172.21.15.32:36534) 2024-04-16T05:37:09.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 -- 172.21.15.32:0/257507484 learned_addr learned my addr 172.21.15.32:0/257507484 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:09.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b60b91640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b440a3d20 0x7f8b440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 0x5568309c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b597fa640 1 -- 172.21.15.32:0/120534923 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b540f28f0 con 0x7f8b5c19a1e0 2024-04-16T05:37:09.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 -- 172.21.15.32:0/257507484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 msgr2=0x5568309b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:09.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 -- 172.21.15.32:0/257507484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 msgr2=0x5568309c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:09.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 0x5568309c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 -- 172.21.15.32:0/257507484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568306fa7d0 con 0x5568309b9770 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b9770 0x5568309c1e10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00bcf0 tx=0x7f8b4c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3b970813831cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.519+0000 7f8b60b91640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b440a3d20 0x7f8b440a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b540be300 tx=0x7f8b540a9040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b427fc640 1 -- 172.21.15.32:0/257507484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c015070 con 0x5568309b9770 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b427fc640 1 -- 172.21.15.32:0/257507484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c01a940 con 0x5568309b9770 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b427fc640 1 -- 172.21.15.32:0/257507484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c011a30 con 0x5568309b9770 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 -- 172.21.15.32:0/257507484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b9770 msgr2=0x5568309c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b9770 0x5568309c1e10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00bcf0 tx=0x7f8b4c00f730 comp rx=0 tx=0).stop 2024-04-16T05:37:09.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 -- 172.21.15.32:0/257507484 shutdown_connections 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 0x5568309c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b9770 0x5568309c1e10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 --2- 172.21.15.32:0/257507484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 -- 172.21.15.32:0/257507484 >> 172.21.15.32:0/257507484 conn(0x5568309b3580 msgr2=0x5568309b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 -- 172.21.15.32:0/257507484 shutdown_connections 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.520+0000 7f8b651e6740 1 -- 172.21.15.32:0/257507484 wait complete. 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 1 -- start start 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 1 Processor -- start 2024-04-16T05:37:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b8e50 0x5568309bf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309bf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5568309c5140 con 0x5568309c2350 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5568308d72c0 con 0x5568309b8e50 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5568309c52c0 con 0x5568309b9770 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b651e6740 10 client.0 _ll_register_callbacks cb 0x5568308be090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309bfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36538/0 (socket says 172.21.15.32:36538) 2024-04-16T05:37:09.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.521+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/1713915072 learned_addr learned my addr 172.21.15.32:0/1713915072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b8e50 0x5568309bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/1713915072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 msgr2=0x5568309bf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309bf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/1713915072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b8e50 msgr2=0x5568309bf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b8e50 0x5568309bf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/1713915072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568306303f0 con 0x5568309c2350 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309bfea0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8b500118c0 tx=0x7f8b50013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c060074cbf97639 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:09.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/1713915072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000fb50 con 0x5568309c2350 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/1713915072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b5001a040 con 0x5568309c2350 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b651e6740 1 -- 172.21.15.32:0/1713915072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4c01a940 con 0x5568309c2350 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/1713915072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b5000fcf0 con 0x5568309c2350 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b651e6740 10 client.16149 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b651e6740 1 -- 172.21.15.32:0/1713915072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5568305f00f0 con 0x5568309c2350 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.522+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/1713915072 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c01a940 con 0x5568309c2350 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/1713915072 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8b50004cc0 con 0x5568309c2350 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b651e6740 10 client.16149 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b651e6740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b651e6740 20 client.16149 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b651e6740 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5568309d2810 0x5568309d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b651e6740 4 client.16149 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b651e6740 1 -- 172.21.15.32:0/1713915072 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55683092bba0 con 0x5568309d2810 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5568309d2810 0x5568309d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5568309d2810 0x5568309d4c70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d41e7466e8dec4 server_cookie=3a521c15601bddd6 in_seq=0 out_seq=0 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.523+0000 7f8b3b7fe640 10 client.16149 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.524+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/1713915072 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55683092bba0 con 0x5568309d2810 2024-04-16T05:37:09.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.524+0000 7f8b3b7fe640 10 client.16149 handle_command_reply: tid=0 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.524+0000 7f8b651e6740 1 client.16149 shutdown 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.524+0000 7f8b651e6740 1 -- 172.21.15.32:0/1713915072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 msgr2=0x5568309bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.524+0000 7f8b651e6740 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309bfea0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8b500118c0 tx=0x7f8b50013040 comp rx=0 tx=0).stop 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 1 -- 172.21.15.32:0/1713915072 shutdown_connections 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309bfea0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5568309d2810 0x5568309d4c70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 1 --2- 172.21.15.32:0/1713915072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b8e50 0x5568309bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b3b7fe640 0 client.16149 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:09.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 1 -- 172.21.15.32:0/1713915072 >> 172.21.15.32:0/1713915072 conn(0x5568309b3580 msgr2=0x5568309b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:09.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 1 -- 172.21.15.32:0/1713915072 shutdown_connections 2024-04-16T05:37:09.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 1 -- 172.21.15.32:0/1713915072 wait complete. 2024-04-16T05:37:09.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.525+0000 7f8b651e6740 20 client.16149 trim_cache size 0 max 16384 2024-04-16T05:37:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b651e6740 1 Processor -- start 2024-04-16T05:37:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b651e6740 1 -- start start 2024-04-16T05:37:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b9770 0x5568309be5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b651e6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1b1c40 con 0x5568309c2350 2024-04-16T05:37:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b651e6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1af510 con 0x5568309b9770 2024-04-16T05:37:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b651e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1ad360 con 0x5568309b8e50 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.542+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36548/0 (socket says 172.21.15.32:36548) 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/3993039129 learned_addr learned my addr 172.21.15.32:0/3993039129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/3993039129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/3993039129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 msgr2=0x5568309c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/3993039129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/3993039129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b9770 msgr2=0x5568309be5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/3993039129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b9770 0x5568309be5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/3993039129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568309ce540 con 0x5568309c2350 2024-04-16T05:37:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/3993039129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309ce000 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000ff40 tx=0x7f8b50005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f86cbfa17598e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b41ffb640 1 -- 172.21.15.32:0/3993039129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50035070 con 0x5568309c2350 2024-04-16T05:37:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b41ffb640 1 -- 172.21.15.32:0/3993039129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5000f790 con 0x5568309c2350 2024-04-16T05:37:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.543+0000 7f8b41ffb640 1 -- 172.21.15.32:0/3993039129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50003970 con 0x5568309c2350 2024-04-16T05:37:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 -- 172.21.15.32:0/3993039129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 msgr2=0x5568309ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 --2- 172.21.15.32:0/3993039129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309ce000 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000ff40 tx=0x7f8b50005740 comp rx=0 tx=0).stop 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 -- 172.21.15.32:0/3993039129 shutdown_connections 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 --2- 172.21.15.32:0/3993039129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309c2350 0x5568309ce000 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 --2- 172.21.15.32:0/3993039129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309b9770 0x5568309be5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 --2- 172.21.15.32:0/3993039129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b8e50 0x5568309c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 -- 172.21.15.32:0/3993039129 >> 172.21.15.32:0/3993039129 conn(0x5568309b3580 msgr2=0x5568309b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 -- 172.21.15.32:0/3993039129 shutdown_connections 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 -- 172.21.15.32:0/3993039129 wait complete. 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 -- start start 2024-04-16T05:37:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 Processor -- start 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.544+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b8e50 0x5568309c7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 0x556830a36db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1ad9d0 con 0x5568309b8e50 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1aee40 con 0x5568309c2350 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1ae0a0 con 0x5568309b9770 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 10 client.0 _ll_register_callbacks cb 0x7f8b500098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b8e50 0x5568309c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b8e50 0x5568309c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36564/0 (socket says 172.21.15.32:36564) 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/2198848554 learned_addr learned my addr 172.21.15.32:0/2198848554 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/2198848554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 msgr2=0x5568309c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/2198848554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 msgr2=0x556830a36db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 0x556830a36db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 -- 172.21.15.32:0/2198848554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556830a373c0 con 0x5568309b8e50 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b5b7fe640 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b8e50 0x5568309c7770 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8b50014040 tx=0x7f8b50011820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d9fc747d365515 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/2198848554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50035040 con 0x5568309b8e50 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/2198848554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b50005040 con 0x5568309b8e50 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 1 -- 172.21.15.32:0/2198848554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b5000f790 con 0x5568309b8e50 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/2198848554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b50005240 con 0x5568309b8e50 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 10 client.16155 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.545+0000 7f8b651e6740 1 -- 172.21.15.32:0/2198848554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556830a386d0 con 0x5568309b8e50 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/2198848554 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5000f790 con 0x5568309b8e50 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/2198848554 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8b5002e610 con 0x5568309b8e50 2024-04-16T05:37:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b651e6740 10 client.16155 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b651e6740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b651e6740 20 client.16155 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b651e6740 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556830a38ce0 0x556830a3b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b651e6740 4 client.16155 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b651e6740 1 -- 172.21.15.32:0/2198848554 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556830925a10 con 0x556830a38ce0 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556830a38ce0 0x556830a3b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b5bfff640 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556830a38ce0 0x556830a3b100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d8a10e150a69c9 server_cookie=ca9fb242ca64ef0d in_seq=0 out_seq=0 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.546+0000 7f8b3b7fe640 10 client.16155 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b3b7fe640 1 -- 172.21.15.32:0/2198848554 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556830925a10 con 0x556830a38ce0 2024-04-16T05:37:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b3b7fe640 10 client.16155 handle_command_reply: tid=0 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 client.16155 shutdown 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 -- 172.21.15.32:0/2198848554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b8e50 msgr2=0x5568309c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b8e50 0x5568309c7770 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8b50014040 tx=0x7f8b50011820 comp rx=0 tx=0).stop 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 -- 172.21.15.32:0/2198848554 shutdown_connections 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568309c2350 0x556830a36db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556830a38ce0 0x556830a3b100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568309b9770 0x5568309c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 --2- 172.21.15.32:0/2198848554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568309b8e50 0x5568309c7770 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b3b7fe640 0 client.16155 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 -- 172.21.15.32:0/2198848554 >> 172.21.15.32:0/2198848554 conn(0x5568309b3580 msgr2=0x5568309b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 -- 172.21.15.32:0/2198848554 shutdown_connections 2024-04-16T05:37:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 1 -- 172.21.15.32:0/2198848554 wait complete. 2024-04-16T05:37:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b651e6740 20 client.16155 trim_cache size 0 max 16384 2024-04-16T05:37:09.555 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:37:09.555 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:37:09.555 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:37:09.556 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:37:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b625f4640 1 -- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b440a3d20 msgr2=0x7f8b440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.547+0000 7f8b625f4640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b440a3d20 0x7f8b440a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b540be300 tx=0x7f8b540a9040 comp rx=0 tx=0).stop 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 -- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c19a1e0 msgr2=0x7f8b5c1a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c19a1e0 0x7f8b5c1a4790 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8b5402c700 tx=0x7f8b54032f20 comp rx=0 tx=0).stop 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 -- 172.21.15.32:0/120534923 shutdown_connections 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b440a3d20 0x7f8b440a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b5c19a1e0 0x7f8b5c1a4790 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b5c106210 0x7f8b5c199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 --2- 172.21.15.32:0/120534923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b5c101140 0x7f8b5c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 -- 172.21.15.32:0/120534923 >> 172.21.15.32:0/120534923 conn(0x7f8b5c078000 msgr2=0x7f8b5c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 -- 172.21.15.32:0/120534923 shutdown_connections 2024-04-16T05:37:09.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.548+0000 7f8b625f4640 1 -- 172.21.15.32:0/120534923 wait complete. 2024-04-16T05:37:09.558 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:37:09.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:37:09.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 -- 172.21.15.32:0/526434068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 msgr2=0x7f178810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 --2- 172.21.15.32:0/526434068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 0x7f178810dde0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1780009ca0 tx=0x7f178002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 -- 172.21.15.32:0/526434068 shutdown_connections 2024-04-16T05:37:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 --2- 172.21.15.32:0/526434068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 0x7f178810dde0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 --2- 172.21.15.32:0/526434068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1788101af0 0x7f1788105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 --2- 172.21.15.32:0/526434068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788101140 0x7f1788101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 -- 172.21.15.32:0/526434068 >> 172.21.15.32:0/526434068 conn(0x7f1788077280 msgr2=0x7f1788077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 -- 172.21.15.32:0/526434068 shutdown_connections 2024-04-16T05:37:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 -- 172.21.15.32:0/526434068 wait complete. 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 Processor -- start 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 -- start start 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788101140 0x7f1788199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 0x7f1788199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.666+0000 7f1790100640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178819a190 0x7f17881a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f1790100640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17881112b0 con 0x7f1788106210 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f1790100640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1788111130 con 0x7f178819a190 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f1790100640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1788111430 con 0x7f1788101140 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 0x7f1788199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 0x7f1788199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36570/0 (socket says 172.21.15.32:36570) 2024-04-16T05:37:09.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 -- 172.21.15.32:0/1337990301 learned_addr learned my addr 172.21.15.32:0/1337990301 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:09.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178de9c640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788101140 0x7f1788199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178e69d640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178819a190 0x7f17881a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 -- 172.21.15.32:0/1337990301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788101140 msgr2=0x7f1788199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788101140 0x7f1788199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 -- 172.21.15.32:0/1337990301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178819a190 msgr2=0x7f17881a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178819a190 0x7f17881a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 -- 172.21.15.32:0/1337990301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17881a4f10 con 0x7f1788106210 2024-04-16T05:37:09.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f178d69b640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 0x7f1788199c50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f177c00d770 tx=0x7f177c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:09.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f177affd640 1 -- 172.21.15.32:0/1337990301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c017070 con 0x7f1788106210 2024-04-16T05:37:09.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f177affd640 1 -- 172.21.15.32:0/1337990301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177c009c30 con 0x7f1788106210 2024-04-16T05:37:09.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f177affd640 1 -- 172.21.15.32:0/1337990301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c00f620 con 0x7f1788106210 2024-04-16T05:37:09.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17881a5200 con 0x7f1788106210 2024-04-16T05:37:09.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.667+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881a5750 con 0x7f1788106210 2024-04-16T05:37:09.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.668+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1788102610 con 0x7f1788106210 2024-04-16T05:37:09.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.669+0000 7f177affd640 1 -- 172.21.15.32:0/1337990301 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f177c01a070 con 0x7f1788106210 2024-04-16T05:37:09.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.670+0000 7f177affd640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17740a3d20 0x7f17740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:09.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.670+0000 7f178de9c640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17740a3d20 0x7f17740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:09.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.670+0000 7f177affd640 1 -- 172.21.15.32:0/1337990301 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f177c0d0630 con 0x7f1788106210 2024-04-16T05:37:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.671+0000 7f178de9c640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17740a3d20 0x7f17740a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f178400bbe0 tx=0x7f178400b390 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.675+0000 7f177affd640 1 -- 172.21.15.32:0/1337990301 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f177c009de0 con 0x7f1788106210 2024-04-16T05:37:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.902+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1788109440 con 0x7f1788106210 2024-04-16T05:37:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.903+0000 7f177affd640 1 -- 172.21.15.32:0/1337990301 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f177c09c930 con 0x7f1788106210 2024-04-16T05:37:09.902 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:37:09.902 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:37:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:37:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17740a3d20 msgr2=0x7f17740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17740a3d20 0x7f17740a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f178400bbe0 tx=0x7f178400b390 comp rx=0 tx=0).stop 2024-04-16T05:37:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 msgr2=0x7f1788199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 0x7f1788199c50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f177c00d770 tx=0x7f177c00dc30 comp rx=0 tx=0).stop 2024-04-16T05:37:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 shutdown_connections 2024-04-16T05:37:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178819a190 0x7f17881a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17740a3d20 0x7f17740a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1788106210 0x7f1788199c50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 --2- 172.21.15.32:0/1337990301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788101140 0x7f1788199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 >> 172.21.15.32:0/1337990301 conn(0x7f1788077280 msgr2=0x7f17880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 shutdown_connections 2024-04-16T05:37:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:09.905+0000 7f1790100640 1 -- 172.21.15.32:0/1337990301 wait complete. 2024-04-16T05:37:09.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:37:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 -- 172.21.15.32:0/3822226264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 msgr2=0x7f54a410eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 --2- 172.21.15.32:0/3822226264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 0x7f54a410eef0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f549c009ca0 tx=0x7f549c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 -- 172.21.15.32:0/3822226264 shutdown_connections 2024-04-16T05:37:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 --2- 172.21.15.32:0/3822226264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 0x7f54a410eef0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 --2- 172.21.15.32:0/3822226264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a4102c00 0x7f54a4106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 --2- 172.21.15.32:0/3822226264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a4102250 0x7f54a4102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 -- 172.21.15.32:0/3822226264 >> 172.21.15.32:0/3822226264 conn(0x7f54a4079490 msgr2=0x7f54a40798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 -- 172.21.15.32:0/3822226264 shutdown_connections 2024-04-16T05:37:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.024+0000 7f54abb40640 1 -- 172.21.15.32:0/3822226264 wait complete. 2024-04-16T05:37:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54abb40640 1 Processor -- start 2024-04-16T05:37:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54abb40640 1 -- start start 2024-04-16T05:37:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54abb40640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a4102250 0x7f54a4074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54abb40640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 0x7f54a4074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54abb40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a4074d80 0x7f54a41a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54abb40640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54a41123e0 con 0x7f54a4107320 2024-04-16T05:37:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54abb40640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4112260 con 0x7f54a4074d80 2024-04-16T05:37:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54abb40640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4112560 con 0x7f54a4102250 2024-04-16T05:37:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a90db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 0x7f54a4074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a90db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 0x7f54a4074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36592/0 (socket says 172.21.15.32:36592) 2024-04-16T05:37:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a90db640 1 -- 172.21.15.32:0/1432791551 learned_addr learned my addr 172.21.15.32:0/1432791551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a4102250 0x7f54a4074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54aa0dd640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a4074d80 0x7f54a41a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a90db640 1 -- 172.21.15.32:0/1432791551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a4102250 msgr2=0x7f54a4074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a90db640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a4102250 0x7f54a4074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a90db640 1 -- 172.21.15.32:0/1432791551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a4074d80 msgr2=0x7f54a41a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a90db640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a4074d80 0x7f54a41a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.025+0000 7f54a90db640 1 -- 172.21.15.32:0/1432791551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a41a7110 con 0x7f54a4107320 2024-04-16T05:37:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.026+0000 7f54a90db640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 0x7f54a4074840 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f549400d740 tx=0x7f549400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.026+0000 7f549affd640 1 -- 172.21.15.32:0/1432791551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494017070 con 0x7f54a4107320 2024-04-16T05:37:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.026+0000 7f54abb40640 1 -- 172.21.15.32:0/1432791551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a41a7400 con 0x7f54a4107320 2024-04-16T05:37:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.026+0000 7f549affd640 1 -- 172.21.15.32:0/1432791551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549400de50 con 0x7f54a4107320 2024-04-16T05:37:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.026+0000 7f549affd640 1 -- 172.21.15.32:0/1432791551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004e00 con 0x7f54a4107320 2024-04-16T05:37:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.026+0000 7f54abb40640 1 -- 172.21.15.32:0/1432791551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a41a7950 con 0x7f54a4107320 2024-04-16T05:37:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.028+0000 7f549affd640 1 -- 172.21.15.32:0/1432791551 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5494004fa0 con 0x7f54a4107320 2024-04-16T05:37:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.028+0000 7f54ae807740 1 Processor -- start 2024-04-16T05:37:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54ae807740 1 -- start start 2024-04-16T05:37:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54ae807740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a39240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54ae807740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a39780 0x55b071a41e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54ae807740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a42360 0x55b071a44750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54ae807740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b07199e0e0 con 0x55b071a38e60 2024-04-16T05:37:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f549affd640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54900a3d50 0x7f54900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54ae807740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b07193b520 con 0x55b071a42360 2024-04-16T05:37:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54ae807740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b07199a0a0 con 0x55b071a39780 2024-04-16T05:37:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54aa0dd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a39240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54aa0dd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a39240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36608/0 (socket says 172.21.15.32:36608) 2024-04-16T05:37:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54aa0dd640 1 -- 172.21.15.32:0/1183690853 learned_addr learned my addr 172.21.15.32:0/1183690853 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a39780 0x55b071a41e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54aa0dd640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54900a3d50 0x7f54900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f549affd640 1 -- 172.21.15.32:0/1432791551 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f54940d0430 con 0x7f54a4107320 2024-04-16T05:37:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54a90db640 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a42360 0x55b071a44750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54a98dc640 1 -- 172.21.15.32:0/1183690853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a42360 msgr2=0x55b071a44750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.029+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a42360 0x55b071a44750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.030+0000 7f54a98dc640 1 -- 172.21.15.32:0/1183690853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 msgr2=0x55b071a39240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.030+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a39240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.030+0000 7f54a98dc640 1 -- 172.21.15.32:0/1183690853 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0719638d0 con 0x55b071a39780 2024-04-16T05:37:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.030+0000 7f54aa0dd640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54900a3d50 0x7f54900a6210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f549c004af0 tx=0x7f549c009360 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.030+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a39780 0x55b071a41e20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54a000dbe0 tx=0x7f54a000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c0884c35668c632 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.030+0000 7f548ffff640 1 -- 172.21.15.32:0/1183690853 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0005020 con 0x55b071a39780 2024-04-16T05:37:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.030+0000 7f548ffff640 1 -- 172.21.15.32:0/1183690853 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a00047a0 con 0x55b071a39780 2024-04-16T05:37:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.030+0000 7f548ffff640 1 -- 172.21.15.32:0/1183690853 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0011680 con 0x55b071a39780 2024-04-16T05:37:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 -- 172.21.15.32:0/1183690853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a39780 msgr2=0x55b071a41e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a39780 0x55b071a41e20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54a000dbe0 tx=0x7f54a000f730 comp rx=0 tx=0).stop 2024-04-16T05:37:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 -- 172.21.15.32:0/1183690853 shutdown_connections 2024-04-16T05:37:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a42360 0x55b071a44750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a39780 0x55b071a41e20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 --2- 172.21.15.32:0/1183690853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a39240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 -- 172.21.15.32:0/1183690853 >> 172.21.15.32:0/1183690853 conn(0x55b071a33590 msgr2=0x55b071a33970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 -- 172.21.15.32:0/1183690853 shutdown_connections 2024-04-16T05:37:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 -- 172.21.15.32:0/1183690853 wait complete. 2024-04-16T05:37:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 -- start start 2024-04-16T05:37:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.031+0000 7f54ae807740 1 Processor -- start 2024-04-16T05:37:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54ae807740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a3f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54ae807740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a39780 0x55b071a3f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54ae807740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a3fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54ae807740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b071a45000 con 0x55b071a38e60 2024-04-16T05:37:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54ae807740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b0719574d0 con 0x55b071a39780 2024-04-16T05:37:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54ae807740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b071a45180 con 0x55b071a42360 2024-04-16T05:37:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54ae807740 10 client.0 _ll_register_callbacks cb 0x55b07193c2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a3f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a3f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36616/0 (socket says 172.21.15.32:36616) 2024-04-16T05:37:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 -- 172.21.15.32:0/1949827022 learned_addr learned my addr 172.21.15.32:0/1949827022 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54aa0dd640 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a3fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a90db640 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a39780 0x55b071a3f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 -- 172.21.15.32:0/1949827022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 msgr2=0x55b071a3fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a3fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 -- 172.21.15.32:0/1949827022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a39780 msgr2=0x55b071a3f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a39780 0x55b071a3f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 -- 172.21.15.32:0/1949827022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0716b03f0 con 0x55b071a38e60 2024-04-16T05:37:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a90db640 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a39780 0x55b071a3f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a3f310 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f54a000df30 tx=0x7f54a000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270c8e42366d906e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f548d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f548d7fa640 1 -- 172.21.15.32:0/1949827022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a002b060 con 0x55b071a38e60 2024-04-16T05:37:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f548d7fa640 1 -- 172.21.15.32:0/1949827022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f54a0004b60 con 0x55b071a38e60 2024-04-16T05:37:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.032+0000 7f54ae807740 1 -- 172.21.15.32:0/1949827022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a00047a0 con 0x55b071a38e60 2024-04-16T05:37:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f548d7fa640 1 -- 172.21.15.32:0/1949827022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54a0032050 con 0x55b071a38e60 2024-04-16T05:37:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54ae807740 10 client.16173 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54ae807740 1 -- 172.21.15.32:0/1949827022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b07177a7d0 con 0x55b071a38e60 2024-04-16T05:37:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f548d7fa640 1 -- 172.21.15.32:0/1949827022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a00047a0 con 0x55b071a38e60 2024-04-16T05:37:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f548d7fa640 1 -- 172.21.15.32:0/1949827022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54a001b990 con 0x55b071a38e60 2024-04-16T05:37:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54ae807740 10 client.16173 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54ae807740 10 client.16173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:37:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54ae807740 20 client.16173 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54ae807740 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55b071a522d0 0x55b071a546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54ae807740 4 client.16173 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54ae807740 1 -- 172.21.15.32:0/1949827022 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0719abb00 con 0x55b071a522d0 2024-04-16T05:37:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.033+0000 7f54a90db640 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55b071a522d0 0x55b071a546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.034+0000 7f548d7fa640 10 client.16173 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:37:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.034+0000 7f54a90db640 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55b071a522d0 0x55b071a546c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96b36d6e60df9391 server_cookie=4690d985052f3e6d in_seq=0 out_seq=0 2024-04-16T05:37:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.034+0000 7f548d7fa640 1 -- 172.21.15.32:0/1949827022 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0719abb00 con 0x55b071a522d0 2024-04-16T05:37:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.034+0000 7f548d7fa640 10 client.16173 handle_command_reply: tid=0 2024-04-16T05:37:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.034+0000 7f54ae807740 1 client.16173 shutdown 2024-04-16T05:37:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.034+0000 7f54ae807740 1 -- 172.21.15.32:0/1949827022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 msgr2=0x55b071a3f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.034+0000 7f54ae807740 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a3f310 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f54a000df30 tx=0x7f54a000df60 comp rx=0 tx=0).stop 2024-04-16T05:37:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 1 -- 172.21.15.32:0/1949827022 shutdown_connections 2024-04-16T05:37:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55b071a522d0 0x55b071a546c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a3fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a39780 0x55b071a3f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 1 --2- 172.21.15.32:0/1949827022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a38e60 0x55b071a3f310 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f548d7fa640 0 client.16173 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:37:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 1 -- 172.21.15.32:0/1949827022 >> 172.21.15.32:0/1949827022 conn(0x55b071a33590 msgr2=0x55b071a352e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 1 -- 172.21.15.32:0/1949827022 shutdown_connections 2024-04-16T05:37:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 1 -- 172.21.15.32:0/1949827022 wait complete. 2024-04-16T05:37:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.035+0000 7f54ae807740 20 client.16173 trim_cache size 0 max 16384 2024-04-16T05:37:10.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54ae807740 1 Processor -- start 2024-04-16T05:37:10.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54ae807740 1 -- start start 2024-04-16T05:37:10.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54ae807740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a41a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54ae807740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54ae807740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a4e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54ae807740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54a41b3dc0 con 0x55b071a39780 2024-04-16T05:37:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54ae807740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f54a41b1690 con 0x55b071a38e60 2024-04-16T05:37:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54ae807740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54a41af4e0 con 0x55b071a42360 2024-04-16T05:37:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54a90db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54a90db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36628/0 (socket says 172.21.15.32:36628) 2024-04-16T05:37:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54a90db640 1 -- 172.21.15.32:0/1442425035 learned_addr learned my addr 172.21.15.32:0/1442425035 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54aa0dd640 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a4e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54a90db640 1 -- 172.21.15.32:0/1442425035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 msgr2=0x55b071a4e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54a90db640 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a4e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.048+0000 7f54a98dc640 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a41a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54a90db640 1 -- 172.21.15.32:0/1442425035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 msgr2=0x55b071a41a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54a90db640 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a41a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54a90db640 1 -- 172.21.15.32:0/1442425035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b071a333c0 con 0x55b071a39780 2024-04-16T05:37:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54a90db640 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3e580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f549409be30 tx=0x7f549401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e27cf69b6a5a3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f548f7fe640 1 -- 172.21.15.32:0/1442425035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940ba070 con 0x55b071a39780 2024-04-16T05:37:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f548f7fe640 1 -- 172.21.15.32:0/1442425035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549409c130 con 0x55b071a39780 2024-04-16T05:37:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f548f7fe640 1 -- 172.21.15.32:0/1442425035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494014030 con 0x55b071a39780 2024-04-16T05:37:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54ae807740 1 -- 172.21.15.32:0/1442425035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 msgr2=0x55b071a3e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54ae807740 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3e580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f549409be30 tx=0x7f549401e000 comp rx=0 tx=0).stop 2024-04-16T05:37:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54ae807740 1 -- 172.21.15.32:0/1442425035 shutdown_connections 2024-04-16T05:37:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54ae807740 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a4e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54ae807740 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3e580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54ae807740 1 --2- 172.21.15.32:0/1442425035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a41a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.049+0000 7f54ae807740 1 -- 172.21.15.32:0/1442425035 >> 172.21.15.32:0/1442425035 conn(0x55b071a33590 msgr2=0x55b071a32730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 -- 172.21.15.32:0/1442425035 shutdown_connections 2024-04-16T05:37:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 -- 172.21.15.32:0/1442425035 wait complete. 2024-04-16T05:37:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 -- start start 2024-04-16T05:37:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 Processor -- start 2024-04-16T05:37:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a4aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a3c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54a41afb50 con 0x55b071a39780 2024-04-16T05:37:10.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f54a41b0fc0 con 0x55b071a38e60 2024-04-16T05:37:10.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54a41b0220 con 0x55b071a42360 2024-04-16T05:37:10.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54ae807740 10 client.0 _ll_register_callbacks cb 0x7f54a0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:10.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54a90db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54a90db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36644/0 (socket says 172.21.15.32:36644) 2024-04-16T05:37:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.050+0000 7f54a90db640 1 -- 172.21.15.32:0/3821930655 learned_addr learned my addr 172.21.15.32:0/3821930655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54aa0dd640 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a3c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54a98dc640 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a4aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54a90db640 1 -- 172.21.15.32:0/3821930655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 msgr2=0x55b071a3c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54a90db640 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a3c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54a90db640 1 -- 172.21.15.32:0/3821930655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 msgr2=0x55b071a4aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54a90db640 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a4aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54a90db640 1 -- 172.21.15.32:0/3821930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b071ab75c0 con 0x55b071a39780 2024-04-16T05:37:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54a98dc640 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a4aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54a90db640 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3bde0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5494005b60 tx=0x7f54940cdf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626460574e520b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f548d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f548d7fa640 1 -- 172.21.15.32:0/3821930655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494014070 con 0x55b071a39780 2024-04-16T05:37:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f548d7fa640 1 -- 172.21.15.32:0/3821930655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f54940d0120 con 0x55b071a39780 2024-04-16T05:37:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54ae807740 1 -- 172.21.15.32:0/3821930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f549409c130 con 0x55b071a39780 2024-04-16T05:37:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f548d7fa640 1 -- 172.21.15.32:0/3821930655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54940ba070 con 0x55b071a39780 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54ae807740 10 client.16179 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f54ae807740 1 -- 172.21.15.32:0/3821930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b071ab86a0 con 0x55b071a39780 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f548d7fa640 1 -- 172.21.15.32:0/3821930655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549409c130 con 0x55b071a39780 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.051+0000 7f548d7fa640 1 -- 172.21.15.32:0/3821930655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54940bc040 con 0x55b071a39780 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 10 client.16179 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 10 client.16179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 20 client.16179 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55b071ab8d90 0x55b071abb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 4 client.16179 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 1 -- 172.21.15.32:0/3821930655 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0719a5970 con 0x55b071ab8d90 2024-04-16T05:37:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54a98dc640 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55b071ab8d90 0x55b071abb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54a98dc640 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55b071ab8d90 0x55b071abb1b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69b2f960a35a2c0f server_cookie=77c1fcf26284f8af in_seq=0 out_seq=0 2024-04-16T05:37:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f548d7fa640 10 client.16179 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:37:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f548d7fa640 1 -- 172.21.15.32:0/3821930655 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0719a5970 con 0x55b071ab8d90 2024-04-16T05:37:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f548d7fa640 10 client.16179 handle_command_reply: tid=0 2024-04-16T05:37:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 1 client.16179 shutdown 2024-04-16T05:37:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 1 -- 172.21.15.32:0/3821930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 msgr2=0x55b071a3bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.052+0000 7f54ae807740 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3bde0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5494005b60 tx=0x7f54940cdf90 comp rx=0 tx=0).stop 2024-04-16T05:37:10.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 1 -- 172.21.15.32:0/3821930655 shutdown_connections 2024-04-16T05:37:10.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55b071ab8d90 0x55b071abb1b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b071a42360 0x55b071a3c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b071a39780 0x55b071a3bde0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f548d7fa640 0 client.16179 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:37:10.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 1 --2- 172.21.15.32:0/3821930655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b071a38e60 0x55b071a4aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 1 -- 172.21.15.32:0/3821930655 >> 172.21.15.32:0/3821930655 conn(0x55b071a33590 msgr2=0x55b071a3f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 1 -- 172.21.15.32:0/3821930655 shutdown_connections 2024-04-16T05:37:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 1 -- 172.21.15.32:0/3821930655 wait complete. 2024-04-16T05:37:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54ae807740 20 client.16179 trim_cache size 0 max 16384 2024-04-16T05:37:10.067 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:37:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 -- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54900a3d50 msgr2=0x7f54900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54900a3d50 0x7f54900a6210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f549c004af0 tx=0x7f549c009360 comp rx=0 tx=0).stop 2024-04-16T05:37:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 -- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 msgr2=0x7f54a4074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 0x7f54a4074840 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f549400d740 tx=0x7f549400dc00 comp rx=0 tx=0).stop 2024-04-16T05:37:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 -- 172.21.15.32:0/1432791551 shutdown_connections 2024-04-16T05:37:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a4074d80 0x7f54a41a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54900a3d50 0x7f54900a6210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a4107320 0x7f54a4074840 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 --2- 172.21.15.32:0/1432791551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a4102250 0x7f54a4074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 -- 172.21.15.32:0/1432791551 >> 172.21.15.32:0/1432791551 conn(0x7f54a4079490 msgr2=0x7f54a40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 -- 172.21.15.32:0/1432791551 shutdown_connections 2024-04-16T05:37:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.053+0000 7f54abb40640 1 -- 172.21.15.32:0/1432791551 wait complete. 2024-04-16T05:37:10.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:09 smithi032 ceph-mon[32351]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:37:10.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:37:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:09 smithi134 ceph-mon[36153]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:37:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:09 smithi082 ceph-mon[36195]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 -- 172.21.15.32:0/2696270332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc4102c00 msgr2=0x7f4bc4106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 --2- 172.21.15.32:0/2696270332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc4102c00 0x7f4bc4106c40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009900 tx=0x7f4bb802bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 -- 172.21.15.32:0/2696270332 shutdown_connections 2024-04-16T05:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 --2- 172.21.15.32:0/2696270332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bc4107320 0x7f4bc410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 --2- 172.21.15.32:0/2696270332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc4102c00 0x7f4bc4106c40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 --2- 172.21.15.32:0/2696270332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bc4102250 0x7f4bc4102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 -- 172.21.15.32:0/2696270332 >> 172.21.15.32:0/2696270332 conn(0x7f4bc40fdaf0 msgr2=0x7f4bc40fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 -- 172.21.15.32:0/2696270332 shutdown_connections 2024-04-16T05:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.179+0000 7f4bcc088640 1 -- 172.21.15.32:0/2696270332 wait complete. 2024-04-16T05:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bcc088640 1 Processor -- start 2024-04-16T05:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bcc088640 1 -- start start 2024-04-16T05:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bcc088640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bc4102250 0x7f4bc419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bcc088640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bc4107320 0x7f4bc419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bcc088640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc41986f0 0x7f4bc4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bcc088640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41123b0 con 0x7f4bc41986f0 2024-04-16T05:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bcc088640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc4112230 con 0x7f4bc4107320 2024-04-16T05:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bcc088640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc4112530 con 0x7f4bc4102250 2024-04-16T05:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc41986f0 0x7f4bc4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc41986f0 0x7f4bc4198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36662/0 (socket says 172.21.15.32:36662) 2024-04-16T05:37:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 -- 172.21.15.32:0/1388416799 learned_addr learned my addr 172.21.15.32:0/1388416799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bc9623640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bc4107320 0x7f4bc419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bc4102250 0x7f4bc419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 -- 172.21.15.32:0/1388416799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bc4102250 msgr2=0x7f4bc419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bc4102250 0x7f4bc419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 -- 172.21.15.32:0/1388416799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bc4107320 msgr2=0x7f4bc419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bc4107320 0x7f4bc419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 -- 172.21.15.32:0/1388416799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc41992f0 con 0x7f4bc41986f0 2024-04-16T05:37:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.180+0000 7f4bca625640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc41986f0 0x7f4bc4198b70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400eaa0 tx=0x7f4bb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.181+0000 7f4bc2ffd640 1 -- 172.21.15.32:0/1388416799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400cdb0 con 0x7f4bc41986f0 2024-04-16T05:37:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.181+0000 7f4bcc088640 1 -- 172.21.15.32:0/1388416799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc41a53d0 con 0x7f4bc41986f0 2024-04-16T05:37:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.181+0000 7f4bc2ffd640 1 -- 172.21.15.32:0/1388416799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4005020 con 0x7f4bc41986f0 2024-04-16T05:37:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.181+0000 7f4bc2ffd640 1 -- 172.21.15.32:0/1388416799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40116d0 con 0x7f4bc41986f0 2024-04-16T05:37:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.181+0000 7f4bcc088640 1 -- 172.21.15.32:0/1388416799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc41a5780 con 0x7f4bc41986f0 2024-04-16T05:37:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.183+0000 7f4bc2ffd640 1 -- 172.21.15.32:0/1388416799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4bb40054d0 con 0x7f4bc41986f0 2024-04-16T05:37:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.183+0000 7f4bcec70740 1 Processor -- start 2024-04-16T05:37:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.183+0000 7f4bcec70740 1 -- start start 2024-04-16T05:37:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e53240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e5e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bcec70740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560386db80e0 con 0x560386e53780 2024-04-16T05:37:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bcec70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560386d55520 con 0x560386e52e60 2024-04-16T05:37:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bcec70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560386db40a0 con 0x560386e5c360 2024-04-16T05:37:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36678/0 (socket says 172.21.15.32:36678) 2024-04-16T05:37:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3226088693 learned_addr learned my addr 172.21.15.32:0/3226088693 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e53240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc2ffd640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4bac0a3d20 0x7f4bac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3226088693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 msgr2=0x560386e5e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e5e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3226088693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 msgr2=0x560386e53240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e53240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3226088693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560386d7d8d0 con 0x560386e53780 2024-04-16T05:37:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e53240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bca625640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4bac0a3d20 0x7f4bac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5be20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00cc60 tx=0x7f4bbc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f21340f3d3dbf09 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4bc2ffd640 1 -- 172.21.15.32:0/1388416799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4bb40d0dc0 con 0x7f4bc41986f0 2024-04-16T05:37:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.184+0000 7f4babfff640 1 -- 172.21.15.32:0/3226088693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc014070 con 0x560386e53780 2024-04-16T05:37:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4babfff640 1 -- 172.21.15.32:0/3226088693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc0047a0 con 0x560386e53780 2024-04-16T05:37:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4babfff640 1 -- 172.21.15.32:0/3226088693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc010760 con 0x560386e53780 2024-04-16T05:37:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 -- 172.21.15.32:0/3226088693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 msgr2=0x560386e5be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bca625640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4bac0a3d20 0x7f4bac0a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40ce7d0 tx=0x7f4bb40bd040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5be20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00cc60 tx=0x7f4bbc00e720 comp rx=0 tx=0).stop 2024-04-16T05:37:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 -- 172.21.15.32:0/3226088693 shutdown_connections 2024-04-16T05:37:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e5e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5be20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3226088693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e53240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 -- 172.21.15.32:0/3226088693 >> 172.21.15.32:0/3226088693 conn(0x560386e4d590 msgr2=0x560386e4d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 -- 172.21.15.32:0/3226088693 shutdown_connections 2024-04-16T05:37:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.185+0000 7f4bcec70740 1 -- 172.21.15.32:0/3226088693 wait complete. 2024-04-16T05:37:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 1 -- start start 2024-04-16T05:37:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 1 Processor -- start 2024-04-16T05:37:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e52e60 0x560386e59300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e53780 0x560386e59840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e5c360 0x560386e59d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560386e5f060 con 0x560386e5c360 2024-04-16T05:37:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560386d714d0 con 0x560386e53780 2024-04-16T05:37:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560386e5f1e0 con 0x560386e52e60 2024-04-16T05:37:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bcec70740 10 client.0 _ll_register_callbacks cb 0x560386d562c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bc9623640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e5c360 0x560386e59d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bc9623640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e5c360 0x560386e59d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36686/0 (socket says 172.21.15.32:36686) 2024-04-16T05:37:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.186+0000 7f4bc9623640 1 -- 172.21.15.32:0/3770564881 learned_addr learned my addr 172.21.15.32:0/3770564881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e52e60 0x560386e59300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e53780 0x560386e59840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bc9623640 1 -- 172.21.15.32:0/3770564881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e52e60 msgr2=0x560386e59300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e52e60 0x560386e59300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bc9623640 1 -- 172.21.15.32:0/3770564881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e53780 msgr2=0x560386e59840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e53780 0x560386e59840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bc9623640 1 -- 172.21.15.32:0/3770564881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560386aca3f0 con 0x560386e5c360 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e5c360 0x560386e59d80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80096c0 tx=0x7f4bb802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caebd60356b7e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4ba97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3770564881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8003800 con 0x560386e5c360 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3770564881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4bb80039a0 con 0x560386e5c360 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bcec70740 1 -- 172.21.15.32:0/3770564881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bbc0047a0 con 0x560386e5c360 2024-04-16T05:37:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3770564881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4bb8050050 con 0x560386e5c360 2024-04-16T05:37:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bcec70740 10 client.16191 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4bcec70740 1 -- 172.21.15.32:0/3770564881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560386b947d0 con 0x560386e5c360 2024-04-16T05:37:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.187+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3770564881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc0047a0 con 0x560386e5c360 2024-04-16T05:37:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3770564881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4bb8034bf0 con 0x560386e5c360 2024-04-16T05:37:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4bcec70740 10 client.16191 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4bcec70740 10 client.16191 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:37:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4bcec70740 20 client.16191 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x560386e6c2c0 0x560386e6e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4bcec70740 4 client.16191 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4bcec70740 1 -- 172.21.15.32:0/3770564881 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560386dc5b00 con 0x560386e6c2c0 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x560386e6c2c0 0x560386e6e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4ba97fa640 10 client.16191 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.188+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x560386e6c2c0 0x560386e6e6b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=299f35c00087777e server_cookie=27c3222e58e61761 in_seq=0 out_seq=0 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3770564881 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560386dc5b00 con 0x560386e6c2c0 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4ba97fa640 10 client.16191 handle_command_reply: tid=0 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 client.16191 shutdown 2024-04-16T05:37:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 -- 172.21.15.32:0/3770564881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e5c360 msgr2=0x560386e59d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e5c360 0x560386e59d80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80096c0 tx=0x7f4bb802bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 -- 172.21.15.32:0/3770564881 shutdown_connections 2024-04-16T05:37:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x560386e6c2c0 0x560386e6e6b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e5c360 0x560386e59d80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e53780 0x560386e59840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3770564881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e52e60 0x560386e59300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4ba97fa640 0 client.16191 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:37:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 -- 172.21.15.32:0/3770564881 >> 172.21.15.32:0/3770564881 conn(0x560386e4d590 msgr2=0x560386e4f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 -- 172.21.15.32:0/3770564881 shutdown_connections 2024-04-16T05:37:10.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 1 -- 172.21.15.32:0/3770564881 wait complete. 2024-04-16T05:37:10.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.189+0000 7f4bcec70740 20 client.16191 trim_cache size 0 max 16384 2024-04-16T05:37:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bcec70740 1 Processor -- start 2024-04-16T05:37:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bcec70740 1 -- start start 2024-04-16T05:37:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e68060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e73f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bcec70740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41b1bc0 con 0x560386e53780 2024-04-16T05:37:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bcec70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41af490 con 0x560386e52e60 2024-04-16T05:37:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bcec70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41ad2e0 con 0x560386e5c360 2024-04-16T05:37:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e73f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e73f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36696/0 (socket says 172.21.15.32:36696) 2024-04-16T05:37:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3646466432 learned_addr learned my addr 172.21.15.32:0/3646466432 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3646466432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e68060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3646466432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 msgr2=0x560386e744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3646466432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3646466432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 msgr2=0x560386e68060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3646466432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e68060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3646466432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560386e5e750 con 0x560386e53780 2024-04-16T05:37:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.203+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3646466432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e73f80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c9f0 tx=0x7f4bbc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcb67210693bf84 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bab7fe640 1 -- 172.21.15.32:0/3646466432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc014040 con 0x560386e53780 2024-04-16T05:37:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bab7fe640 1 -- 172.21.15.32:0/3646466432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc005390 con 0x560386e53780 2024-04-16T05:37:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bab7fe640 1 -- 172.21.15.32:0/3646466432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0250d0 con 0x560386e53780 2024-04-16T05:37:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 -- 172.21.15.32:0/3646466432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 msgr2=0x560386e73f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3646466432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e73f80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c9f0 tx=0x7f4bbc00cf60 comp rx=0 tx=0).stop 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 -- 172.21.15.32:0/3646466432 shutdown_connections 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3646466432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3646466432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e73f80 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3646466432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e68060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 -- 172.21.15.32:0/3646466432 >> 172.21.15.32:0/3646466432 conn(0x560386e4d590 msgr2=0x560386e5b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 -- 172.21.15.32:0/3646466432 shutdown_connections 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 -- 172.21.15.32:0/3646466432 wait complete. 2024-04-16T05:37:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.204+0000 7f4bcec70740 1 -- start start 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bcec70740 1 Processor -- start 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e64a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bcec70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bcec70740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41ad950 con 0x560386e53780 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bcec70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41aedc0 con 0x560386e52e60 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bcec70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41ae020 con 0x560386e5c360 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bcec70740 10 client.0 _ll_register_callbacks cb 0x7f4bb8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36698/0 (socket says 172.21.15.32:36698) 2024-04-16T05:37:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3679761717 learned_addr learned my addr 172.21.15.32:0/3679761717 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e64a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3679761717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 msgr2=0x560386e603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3679761717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 msgr2=0x560386e64a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e64a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 -- 172.21.15.32:0/3679761717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560386e60c10 con 0x560386e53780 2024-04-16T05:37:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4bc9e24640 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5fdd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c3f0 tx=0x7f4bbc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839092a282d5dcf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.205+0000 7f4ba97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3679761717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc033060 con 0x560386e53780 2024-04-16T05:37:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3679761717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4bbc0108c0 con 0x560386e53780 2024-04-16T05:37:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 1 -- 172.21.15.32:0/3679761717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bbc005390 con 0x560386e53780 2024-04-16T05:37:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3679761717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4bbc014070 con 0x560386e53780 2024-04-16T05:37:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 10 client.16197 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 1 -- 172.21.15.32:0/3679761717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560386ed2920 con 0x560386e53780 2024-04-16T05:37:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3679761717 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc005390 con 0x560386e53780 2024-04-16T05:37:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3679761717 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4bbc010ce0 con 0x560386e53780 2024-04-16T05:37:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 10 client.16197 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 10 client.16197 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:37:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 20 client.16197 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x560386ed2e70 0x560386ed5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 4 client.16197 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bcec70740 1 -- 172.21.15.32:0/3679761717 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560386dbf970 con 0x560386ed2e70 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.206+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x560386ed2e70 0x560386ed5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4ba97fa640 10 client.16197 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bc9623640 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x560386ed2e70 0x560386ed5290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cf57ade5cd6b55b server_cookie=fd03f61f2a1b1946 in_seq=0 out_seq=0 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4ba97fa640 1 -- 172.21.15.32:0/3679761717 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560386dbf970 con 0x560386ed2e70 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4ba97fa640 10 client.16197 handle_command_reply: tid=0 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 client.16197 shutdown 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 -- 172.21.15.32:0/3679761717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 msgr2=0x560386e5fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5fdd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c3f0 tx=0x7f4bbc00eca0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 -- 172.21.15.32:0/3679761717 shutdown_connections 2024-04-16T05:37:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x560386ed2e70 0x560386ed5290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560386e5c360 0x560386e603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560386e53780 0x560386e5fdd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4ba97fa640 0 client.16197 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:37:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 --2- 172.21.15.32:0/3679761717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560386e52e60 0x560386e64a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 -- 172.21.15.32:0/3679761717 >> 172.21.15.32:0/3679761717 conn(0x560386e4d590 msgr2=0x560386e4e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 -- 172.21.15.32:0/3679761717 shutdown_connections 2024-04-16T05:37:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 1 -- 172.21.15.32:0/3679761717 wait complete. 2024-04-16T05:37:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.207+0000 7f4bcec70740 20 client.16197 trim_cache size 0 max 16384 2024-04-16T05:37:10.219 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:37:10.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 -- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4bac0a3d20 msgr2=0x7f4bac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4bac0a3d20 0x7f4bac0a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40ce7d0 tx=0x7f4bb40bd040 comp rx=0 tx=0).stop 2024-04-16T05:37:10.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 -- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc41986f0 msgr2=0x7f4bc4198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc41986f0 0x7f4bc4198b70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400eaa0 tx=0x7f4bb400ef60 comp rx=0 tx=0).stop 2024-04-16T05:37:10.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 -- 172.21.15.32:0/1388416799 shutdown_connections 2024-04-16T05:37:10.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4bac0a3d20 0x7f4bac0a61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc41986f0 0x7f4bc4198b70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bc4107320 0x7f4bc419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 --2- 172.21.15.32:0/1388416799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bc4102250 0x7f4bc419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 -- 172.21.15.32:0/1388416799 >> 172.21.15.32:0/1388416799 conn(0x7f4bc40fdaf0 msgr2=0x7f4bc40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 -- 172.21.15.32:0/1388416799 shutdown_connections 2024-04-16T05:37:10.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.208+0000 7f4bcc088640 1 -- 172.21.15.32:0/1388416799 wait complete. 2024-04-16T05:37:10.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 -- 172.21.15.32:0/211207529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568101150 msgr2=0x7f75681015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 --2- 172.21.15.32:0/211207529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568101150 0x7f75681015b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7558009900 tx=0x7f755802bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 -- 172.21.15.32:0/211207529 shutdown_connections 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 --2- 172.21.15.32:0/211207529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7568101af0 0x7f756810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 --2- 172.21.15.32:0/211207529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568101150 0x7f75681015b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 --2- 172.21.15.32:0/211207529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7568105e80 0x7f7568106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 -- 172.21.15.32:0/211207529 >> 172.21.15.32:0/211207529 conn(0x7f7568077cf0 msgr2=0x7f7568078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 -- 172.21.15.32:0/211207529 shutdown_connections 2024-04-16T05:37:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.331+0000 7f757031c640 1 -- 172.21.15.32:0/211207529 wait complete. 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f757031c640 1 Processor -- start 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f757031c640 1 -- start start 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f757031c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7568101150 0x7f75681997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f757031c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7568101af0 0x7f7568199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f757031c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568105e80 0x7f75681a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f757031c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7568111380 con 0x7f7568105e80 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f757031c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7568111200 con 0x7f7568101af0 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f757031c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7568111500 con 0x7f7568101150 2024-04-16T05:37:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568105e80 0x7f75681a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568105e80 0x7f75681a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36714/0 (socket says 172.21.15.32:36714) 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 -- 172.21.15.32:0/476816991 learned_addr learned my addr 172.21.15.32:0/476816991 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e0b8640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7568101150 0x7f75681997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756d8b7640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7568101af0 0x7f7568199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 -- 172.21.15.32:0/476816991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7568101150 msgr2=0x7f75681997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7568101150 0x7f75681997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 -- 172.21.15.32:0/476816991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7568101af0 msgr2=0x7f7568199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7568101af0 0x7f7568199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 -- 172.21.15.32:0/476816991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75681a4d60 con 0x7f7568105e80 2024-04-16T05:37:10.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756d8b7640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7568101af0 0x7f7568199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:10.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.332+0000 7f756e8b9640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568105e80 0x7f75681a4820 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f756000eaa0 tx=0x7f756000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.333+0000 7f755f7fe640 1 -- 172.21.15.32:0/476816991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756000cdb0 con 0x7f7568105e80 2024-04-16T05:37:10.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.333+0000 7f755f7fe640 1 -- 172.21.15.32:0/476816991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7560005020 con 0x7f7568105e80 2024-04-16T05:37:10.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.333+0000 7f757031c640 1 -- 172.21.15.32:0/476816991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75681a4f30 con 0x7f7568105e80 2024-04-16T05:37:10.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.333+0000 7f755f7fe640 1 -- 172.21.15.32:0/476816991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75600116b0 con 0x7f7568105e80 2024-04-16T05:37:10.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.333+0000 7f757031c640 1 -- 172.21.15.32:0/476816991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75681a53f0 con 0x7f7568105e80 2024-04-16T05:37:10.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.335+0000 7f755f7fe640 1 -- 172.21.15.32:0/476816991 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7560005560 con 0x7f7568105e80 2024-04-16T05:37:10.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.335+0000 7f7572fe1740 1 Processor -- start 2024-04-16T05:37:10.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f7572fe1740 1 -- start start 2024-04-16T05:37:10.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93b9ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93bafb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f7572fe1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556b93b050e0 con 0x556b93ba04b0 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f7572fe1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556b93aa2520 con 0x556b93b9fb90 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f7572fe1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556b93b010a0 con 0x556b93bad770 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f755f7fe640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f75540a3d20 0x7f75540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e0b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e0b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36720/0 (socket says 172.21.15.32:36720) 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e0b8640 1 -- 172.21.15.32:0/847314307 learned_addr learned my addr 172.21.15.32:0/847314307 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e8b9640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f75540a3d20 0x7f75540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756d8b7640 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93b9ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756d8b7640 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93bafb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f755f7fe640 1 -- 172.21.15.32:0/476816991 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f75600d0d80 con 0x7f7568105e80 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e0b8640 1 -- 172.21.15.32:0/847314307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 msgr2=0x556b93bafb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e0b8640 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93bafb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e0b8640 1 -- 172.21.15.32:0/847314307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 msgr2=0x556b93b9ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e0b8640 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93b9ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.336+0000 7f756e0b8640 1 -- 172.21.15.32:0/847314307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b93aca8d0 con 0x556b93ba04b0 2024-04-16T05:37:10.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.337+0000 7f756e0b8640 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bad230 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f756400bc50 tx=0x7f75640107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c91d8ffc435d416 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.337+0000 7f756e8b9640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f75540a3d20 0x7f75540a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7560018000 tx=0x7f7560000f40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.337+0000 7f755cbfe640 1 -- 172.21.15.32:0/847314307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756401c480 con 0x556b93ba04b0 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.337+0000 7f755cbfe640 1 -- 172.21.15.32:0/847314307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756401cb00 con 0x556b93ba04b0 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.337+0000 7f755cbfe640 1 -- 172.21.15.32:0/847314307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564011ce0 con 0x556b93ba04b0 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- 172.21.15.32:0/847314307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 msgr2=0x556b93bad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bad230 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f756400bc50 tx=0x7f75640107a0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- 172.21.15.32:0/847314307 shutdown_connections 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93bafb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bad230 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 --2- 172.21.15.32:0/847314307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93b9ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- 172.21.15.32:0/847314307 >> 172.21.15.32:0/847314307 conn(0x556b93b9a590 msgr2=0x556b93b9a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- 172.21.15.32:0/847314307 shutdown_connections 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- 172.21.15.32:0/847314307 wait complete. 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- start start 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 Processor -- start 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93baa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93ba04b0 0x556b93baac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93bab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556b93bb0530 con 0x556b93b9fb90 2024-04-16T05:37:10.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556b93abe4d0 con 0x556b93ba04b0 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556b93bb06b0 con 0x556b93bad770 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f7572fe1740 10 client.0 _ll_register_callbacks cb 0x556b93aa32c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f756d8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93baa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f756d8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93baa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36732/0 (socket says 172.21.15.32:36732) 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.338+0000 7f756d8b7640 1 -- 172.21.15.32:0/1400071383 learned_addr learned my addr 172.21.15.32:0/1400071383 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f756e0b8640 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93ba04b0 0x556b93baac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f756d8b7640 1 -- 172.21.15.32:0/1400071383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 msgr2=0x556b93bab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f756d8b7640 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93bab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f756d8b7640 1 -- 172.21.15.32:0/1400071383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93ba04b0 msgr2=0x556b93baac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f756d8b7640 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93ba04b0 0x556b93baac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f756d8b7640 1 -- 172.21.15.32:0/1400071383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b938173f0 con 0x556b93b9fb90 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f756d8b7640 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93baa750 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f755802cfd0 tx=0x7f7558004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db84358e1ba62dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f7551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f7551ffb640 1 -- 172.21.15.32:0/1400071383 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558045070 con 0x556b93b9fb90 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f7572fe1740 1 -- 172.21.15.32:0/1400071383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f756401cb00 con 0x556b93b9fb90 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f7551ffb640 1 -- 172.21.15.32:0/1400071383 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f75580035f0 con 0x556b93b9fb90 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f7551ffb640 1 -- 172.21.15.32:0/1400071383 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7558051050 con 0x556b93b9fb90 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f7572fe1740 10 client.16209 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f7572fe1740 1 -- 172.21.15.32:0/1400071383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b938e17d0 con 0x556b93b9fb90 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.339+0000 7f7551ffb640 1 -- 172.21.15.32:0/1400071383 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756401cb00 con 0x556b93b9fb90 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f7551ffb640 1 -- 172.21.15.32:0/1400071383 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7558041a00 con 0x556b93b9fb90 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f7572fe1740 10 client.16209 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f7572fe1740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f7572fe1740 20 client.16209 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556b93bbd690 0x556b93bbfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f7572fe1740 4 client.16209 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f7572fe1740 1 -- 172.21.15.32:0/1400071383 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b93b12b00 con 0x556b93bbd690 2024-04-16T05:37:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f756e0b8640 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556b93bbd690 0x556b93bbfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f756e0b8640 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556b93bbd690 0x556b93bbfae0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10cb10a3c4370e49 server_cookie=275d304010a15e2d in_seq=0 out_seq=0 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.340+0000 7f7551ffb640 10 client.16209 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7551ffb640 1 -- 172.21.15.32:0/1400071383 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b93b12b00 con 0x556b93bbd690 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7551ffb640 10 client.16209 handle_command_reply: tid=0 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 client.16209 shutdown 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 -- 172.21.15.32:0/1400071383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 msgr2=0x556b93baa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93baa750 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f755802cfd0 tx=0x7f7558004040 comp rx=0 tx=0).stop 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 -- 172.21.15.32:0/1400071383 shutdown_connections 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556b93bbd690 0x556b93bbfae0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93bab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7551ffb640 0 client.16209 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:10.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93ba04b0 0x556b93baac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1400071383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93baa750 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 -- 172.21.15.32:0/1400071383 >> 172.21.15.32:0/1400071383 conn(0x556b93b9a590 msgr2=0x556b93b9c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 -- 172.21.15.32:0/1400071383 shutdown_connections 2024-04-16T05:37:10.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 1 -- 172.21.15.32:0/1400071383 wait complete. 2024-04-16T05:37:10.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.341+0000 7f7572fe1740 20 client.16209 trim_cache size 0 max 16384 2024-04-16T05:37:10.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f7572fe1740 1 Processor -- start 2024-04-16T05:37:10.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f7572fe1740 1 -- start start 2024-04-16T05:37:10.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93bb9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93ba04b0 0x556b93b99970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93bad770 0x556b93baa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f7572fe1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f75681afb00 con 0x556b93b9fb90 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f7572fe1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f75681ad3d0 con 0x556b93bad770 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f7572fe1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75681ab220 con 0x556b93ba04b0 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93bb9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93bb9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36734/0 (socket says 172.21.15.32:36734) 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 -- 172.21.15.32:0/3633876148 learned_addr learned my addr 172.21.15.32:0/3633876148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 -- 172.21.15.32:0/3633876148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93ba04b0 msgr2=0x556b93b99970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 --2- 172.21.15.32:0/3633876148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93ba04b0 0x556b93b99970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 -- 172.21.15.32:0/3633876148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93bad770 msgr2=0x556b93baa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 --2- 172.21.15.32:0/3633876148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93bad770 0x556b93baa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 -- 172.21.15.32:0/3633876148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b93ba02e0 con 0x556b93b9fb90 2024-04-16T05:37:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.355+0000 7f756d8b7640 1 --2- 172.21.15.32:0/3633876148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93bb9460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f755802c170 tx=0x7f75580040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9bb16d04cdfb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7553fff640 1 -- 172.21.15.32:0/3633876148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558045040 con 0x556b93b9fb90 2024-04-16T05:37:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7553fff640 1 -- 172.21.15.32:0/3633876148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7558034440 con 0x556b93b9fb90 2024-04-16T05:37:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7553fff640 1 -- 172.21.15.32:0/3633876148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558041820 con 0x556b93b9fb90 2024-04-16T05:37:10.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 -- 172.21.15.32:0/3633876148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 msgr2=0x556b93bb9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 --2- 172.21.15.32:0/3633876148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93bb9460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f755802c170 tx=0x7f75580040b0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 -- 172.21.15.32:0/3633876148 shutdown_connections 2024-04-16T05:37:10.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 --2- 172.21.15.32:0/3633876148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93bad770 0x556b93baa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 --2- 172.21.15.32:0/3633876148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93ba04b0 0x556b93b99970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 --2- 172.21.15.32:0/3633876148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93b9fb90 0x556b93bb9460 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 -- 172.21.15.32:0/3633876148 >> 172.21.15.32:0/3633876148 conn(0x556b93b9a590 msgr2=0x556b93b9c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 -- 172.21.15.32:0/3633876148 shutdown_connections 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 -- 172.21.15.32:0/3633876148 wait complete. 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.356+0000 7f7572fe1740 1 -- start start 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 1 Processor -- start 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93bb5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93ba6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f75681ab890 con 0x556b93ba04b0 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f75681acd00 con 0x556b93b9fb90 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75681abf60 con 0x556b93bad770 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 10 client.0 _ll_register_callbacks cb 0x7f7558009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bb5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bb5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36746/0 (socket says 172.21.15.32:36746) 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 -- 172.21.15.32:0/1297004876 learned_addr learned my addr 172.21.15.32:0/1297004876 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756d8b7640 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93ba6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756d8b7640 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93bb5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 -- 172.21.15.32:0/1297004876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 msgr2=0x556b93ba6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93ba6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 -- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 msgr2=0x556b93bb5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93bb5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 -- 172.21.15.32:0/1297004876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b93ba7600 con 0x556b93ba04b0 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756d8b7640 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93bb5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f756e0b8640 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bb5eb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f756400bc50 tx=0x7f756400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b059675bb9be7507 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7551ffb640 1 -- 172.21.15.32:0/1297004876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564030060 con 0x556b93ba04b0 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7551ffb640 1 -- 172.21.15.32:0/1297004876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7564011640 con 0x556b93ba04b0 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.357+0000 7f7572fe1740 1 -- 172.21.15.32:0/1297004876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7558034440 con 0x556b93ba04b0 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7551ffb640 1 -- 172.21.15.32:0/1297004876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7564037050 con 0x556b93ba04b0 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7572fe1740 10 client.16215 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7572fe1740 1 -- 172.21.15.32:0/1297004876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b93c237d0 con 0x556b93ba04b0 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7551ffb640 1 -- 172.21.15.32:0/1297004876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7558034440 con 0x556b93ba04b0 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7551ffb640 1 -- 172.21.15.32:0/1297004876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f756402a170 con 0x556b93ba04b0 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7572fe1740 10 client.16215 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7572fe1740 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7572fe1740 20 client.16215 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556b93c23dc0 0x556b93c261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7572fe1740 4 client.16215 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f7572fe1740 1 -- 172.21.15.32:0/1297004876 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b93b0c970 con 0x556b93c23dc0 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.358+0000 7f756d8b7640 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556b93c23dc0 0x556b93c261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f756d8b7640 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556b93c23dc0 0x556b93c261e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebbd2f09f3ecb71 server_cookie=769ef3a9a0a3ed6a in_seq=0 out_seq=0 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7551ffb640 10 client.16215 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7551ffb640 1 -- 172.21.15.32:0/1297004876 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b93b0c970 con 0x556b93c23dc0 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7551ffb640 10 client.16215 handle_command_reply: tid=0 2024-04-16T05:37:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 client.16215 shutdown 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 -- 172.21.15.32:0/1297004876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 msgr2=0x556b93bb5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bb5eb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f756400bc50 tx=0x7f756400bea0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 -- 172.21.15.32:0/1297004876 shutdown_connections 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556b93c23dc0 0x556b93c261e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b93bad770 0x556b93ba6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7551ffb640 0 client.16215 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b93ba04b0 0x556b93bb5eb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 --2- 172.21.15.32:0/1297004876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b93b9fb90 0x556b93bb5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 -- 172.21.15.32:0/1297004876 >> 172.21.15.32:0/1297004876 conn(0x556b93b9a590 msgr2=0x556b93b9b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 -- 172.21.15.32:0/1297004876 shutdown_connections 2024-04-16T05:37:10.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 1 -- 172.21.15.32:0/1297004876 wait complete. 2024-04-16T05:37:10.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.359+0000 7f7572fe1740 20 client.16215 trim_cache size 0 max 16384 2024-04-16T05:37:10.367 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:37:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 -- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f75540a3d20 msgr2=0x7f75540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f75540a3d20 0x7f75540a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7560018000 tx=0x7f7560000f40 comp rx=0 tx=0).stop 2024-04-16T05:37:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 -- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568105e80 msgr2=0x7f75681a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568105e80 0x7f75681a4820 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f756000eaa0 tx=0x7f756000ef60 comp rx=0 tx=0).stop 2024-04-16T05:37:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 -- 172.21.15.32:0/476816991 shutdown_connections 2024-04-16T05:37:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f75540a3d20 0x7f75540a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7568105e80 0x7f75681a4820 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7568101af0 0x7f7568199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 --2- 172.21.15.32:0/476816991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7568101150 0x7f75681997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 -- 172.21.15.32:0/476816991 >> 172.21.15.32:0/476816991 conn(0x7f7568077cf0 msgr2=0x7f75680feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 -- 172.21.15.32:0/476816991 shutdown_connections 2024-04-16T05:37:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:10.360+0000 7f757031c640 1 -- 172.21.15.32:0/476816991 wait complete. 2024-04-16T05:37:11.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:10 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1337990301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:11.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:10 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1337990301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:11.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:10 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1337990301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:11.370 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:37:11.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:37:11.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:37:11.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/2001578679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 msgr2=0x7f6db81153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/2001578679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 0x7f6db81153d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009ca0 tx=0x7f6db002c180 comp rx=0 tx=0).stop 2024-04-16T05:37:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/2001578679 shutdown_connections 2024-04-16T05:37:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/2001578679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 0x7f6db81153d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/2001578679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6db8074cc0 0x7f6db8112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/2001578679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6db80743a0 0x7f6db8074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/2001578679 >> 172.21.15.32:0/2001578679 conn(0x7f6db80fda40 msgr2=0x7f6db80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/2001578679 shutdown_connections 2024-04-16T05:37:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.494+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/2001578679 wait complete. 2024-04-16T05:37:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbfb1b640 1 Processor -- start 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbfb1b640 1 -- start start 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbfb1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6db80743a0 0x7f6db819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbfb1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6db8074cc0 0x7f6db819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbfb1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 0x7f6db81a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbfb1b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81189e0 con 0x7f6db8112fe0 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbfb1b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8118860 con 0x7f6db80743a0 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbfb1b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8118b60 con 0x7f6db8074cc0 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbe0b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 0x7f6db81a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbe0b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 0x7f6db81a37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36770/0 (socket says 172.21.15.32:36770) 2024-04-16T05:37:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.495+0000 7f6dbe0b8640 1 -- 172.21.15.32:0/238095928 learned_addr learned my addr 172.21.15.32:0/238095928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6db8074cc0 0x7f6db819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbe0b8640 1 -- 172.21.15.32:0/238095928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6db8074cc0 msgr2=0x7f6db819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbe0b8640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6db8074cc0 0x7f6db819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbe0b8640 1 -- 172.21.15.32:0/238095928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6db80743a0 msgr2=0x7f6db819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbe0b8640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6db80743a0 0x7f6db819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbe0b8640 1 -- 172.21.15.32:0/238095928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db81a3f00 con 0x7f6db8112fe0 2024-04-16T05:37:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6db8074cc0 0x7f6db819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbe0b8640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 0x7f6db81a37d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6db00348f0 tx=0x7f6db0035e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6daeffd640 1 -- 172.21.15.32:0/238095928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0039040 con 0x7f6db8112fe0 2024-04-16T05:37:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6daeffd640 1 -- 172.21.15.32:0/238095928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0044070 con 0x7f6db8112fe0 2024-04-16T05:37:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/238095928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db81a4190 con 0x7f6db8112fe0 2024-04-16T05:37:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6daeffd640 1 -- 172.21.15.32:0/238095928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00329c0 con 0x7f6db8112fe0 2024-04-16T05:37:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.496+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/238095928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db81aba80 con 0x7f6db8112fe0 2024-04-16T05:37:11.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.498+0000 7f6daeffd640 1 -- 172.21.15.32:0/238095928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6db0003bf0 con 0x7f6db8112fe0 2024-04-16T05:37:11.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.499+0000 7f6dc27d6740 1 Processor -- start 2024-04-16T05:37:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.499+0000 7f6dc27d6740 1 -- start start 2024-04-16T05:37:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.499+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c86519be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.499+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c8651a4fb0 0x55c8651a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c865100e90 con 0x55c8651a4fb0 2024-04-16T05:37:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c86509e4c0 con 0x55c86519bab0 2024-04-16T05:37:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c8650fce50 con 0x55c86519c3d0 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6daeffd640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6da40a3d20 0x7f6da40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c8651a4fb0 0x55c8651a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbe0b8640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6da40a3d20 0x7f6da40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c8651a4fb0 0x55c8651a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36786/0 (socket says 172.21.15.32:36786) 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 -- 172.21.15.32:0/1171974874 learned_addr learned my addr 172.21.15.32:0/1171974874 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c86519be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6daeffd640 1 -- 172.21.15.32:0/238095928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6db00f3080 con 0x7f6db8112fe0 2024-04-16T05:37:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 -- 172.21.15.32:0/1171974874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 msgr2=0x55c8651a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 -- 172.21.15.32:0/1171974874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 msgr2=0x55c86519be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c86519be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 -- 172.21.15.32:0/1171974874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c864edb7e0 con 0x55c8651a4fb0 2024-04-16T05:37:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbe0b8640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6da40a3d20 0x7f6da40a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6db00bea90 tx=0x7f6db002c180 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c8651a4fb0 0x55c8651a73a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6da80127a0 tx=0x7f6da800d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4012a7ef9155620d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.500+0000 7f6da3fff640 1 -- 172.21.15.32:0/1171974874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da800a850 con 0x55c8651a4fb0 2024-04-16T05:37:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6da3fff640 1 -- 172.21.15.32:0/1171974874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da80103d0 con 0x55c8651a4fb0 2024-04-16T05:37:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6da3fff640 1 -- 172.21.15.32:0/1171974874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da800f620 con 0x55c8651a4fb0 2024-04-16T05:37:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 -- 172.21.15.32:0/1171974874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c8651a4fb0 msgr2=0x55c8651a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c8651a4fb0 0x55c8651a73a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6da80127a0 tx=0x7f6da800d360 comp rx=0 tx=0).stop 2024-04-16T05:37:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 -- 172.21.15.32:0/1171974874 shutdown_connections 2024-04-16T05:37:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c8651a4fb0 0x55c8651a73a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/1171974874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c86519be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 -- 172.21.15.32:0/1171974874 >> 172.21.15.32:0/1171974874 conn(0x55c8651961e0 msgr2=0x55c8651965c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 -- 172.21.15.32:0/1171974874 shutdown_connections 2024-04-16T05:37:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.501+0000 7f6dc27d6740 1 -- 172.21.15.32:0/1171974874 wait complete. 2024-04-16T05:37:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 1 -- start start 2024-04-16T05:37:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 1 Processor -- start 2024-04-16T05:37:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c8651a2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519c3d0 0x55c8651a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651a4fb0 0x55c8651a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c8651a7da0 con 0x55c86519c3d0 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c865089410 con 0x55c86519bab0 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c8651a7f20 con 0x55c8651a4fb0 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dc27d6740 10 client.0 _ll_register_callbacks cb 0x55c865084b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dbd8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519c3d0 0x55c8651a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dbd8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519c3d0 0x55c8651a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36796/0 (socket says 172.21.15.32:36796) 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.502+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/2594341603 learned_addr learned my addr 172.21.15.32:0/2594341603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651a4fb0 0x55c8651a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c8651a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/2594341603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651a4fb0 msgr2=0x55c8651a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651a4fb0 0x55c8651a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 msgr2=0x55c8651a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c8651a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/2594341603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c864f3bf70 con 0x55c86519c3d0 2024-04-16T05:37:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c8651a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519c3d0 0x55c8651a25c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6db40038e0 tx=0x7f6db4003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490292c7ebdc174d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6da17fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6da17fa640 1 -- 172.21.15.32:0/2594341603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db401a670 con 0x55c86519c3d0 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6da17fa640 1 -- 172.21.15.32:0/2594341603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6db401a810 con 0x55c86519c3d0 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dc27d6740 1 -- 172.21.15.32:0/2594341603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da80103d0 con 0x55c86519c3d0 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6da17fa640 1 -- 172.21.15.32:0/2594341603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6db4023050 con 0x55c86519c3d0 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dc27d6740 10 client.16227 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6dc27d6740 1 -- 172.21.15.32:0/2594341603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c864dd3170 con 0x55c86519c3d0 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.503+0000 7f6da17fa640 1 -- 172.21.15.32:0/2594341603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da80103d0 con 0x55c86519c3d0 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6da17fa640 1 -- 172.21.15.32:0/2594341603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6db4011b20 con 0x55c86519c3d0 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6dc27d6740 10 client.16227 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6dc27d6740 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6dc27d6740 20 client.16227 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c8651b5450 0x55c8651b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6dc27d6740 4 client.16227 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6dc27d6740 1 -- 172.21.15.32:0/2594341603 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c86510e8b0 con 0x55c8651b5450 2024-04-16T05:37:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c8651b5450 0x55c8651b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6da17fa640 10 client.16227 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.504+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c8651b5450 0x55c8651b78c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b30cd9cf56b4f5 server_cookie=e2098af5b8e18575 in_seq=0 out_seq=0 2024-04-16T05:37:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6da17fa640 1 -- 172.21.15.32:0/2594341603 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c86510e8b0 con 0x55c8651b5450 2024-04-16T05:37:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6da17fa640 10 client.16227 handle_command_reply: tid=0 2024-04-16T05:37:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 client.16227 shutdown 2024-04-16T05:37:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 -- 172.21.15.32:0/2594341603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519c3d0 msgr2=0x55c8651a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519c3d0 0x55c8651a25c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6db40038e0 tx=0x7f6db4003da0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 -- 172.21.15.32:0/2594341603 shutdown_connections 2024-04-16T05:37:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c8651b5450 0x55c8651b78c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651a4fb0 0x55c8651a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519c3d0 0x55c8651a25c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/2594341603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519bab0 0x55c8651a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6da17fa640 0 client.16227 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 -- 172.21.15.32:0/2594341603 >> 172.21.15.32:0/2594341603 conn(0x55c8651961e0 msgr2=0x55c865197f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 -- 172.21.15.32:0/2594341603 shutdown_connections 2024-04-16T05:37:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 1 -- 172.21.15.32:0/2594341603 wait complete. 2024-04-16T05:37:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.505+0000 7f6dc27d6740 20 client.16227 trim_cache size 0 max 16384 2024-04-16T05:37:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.523+0000 7f6dc27d6740 1 Processor -- start 2024-04-16T05:37:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.523+0000 7f6dc27d6740 1 -- start start 2024-04-16T05:37:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.523+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651961e0 0x55c8651b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.523+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.523+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519c3d0 0x55c8651bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.523+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81b8310 con 0x55c86519bab0 2024-04-16T05:37:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.523+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81b5be0 con 0x55c86519c3d0 2024-04-16T05:37:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.523+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81b3a30 con 0x55c8651961e0 2024-04-16T05:37:11.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651bd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36812/0 (socket says 172.21.15.32:36812) 2024-04-16T05:37:11.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/872085155 learned_addr learned my addr 172.21.15.32:0/872085155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651961e0 0x55c8651b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/872085155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651961e0 msgr2=0x55c8651b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519c3d0 0x55c8651bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651961e0 0x55c8651b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/872085155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519c3d0 msgr2=0x55c8651bd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519c3d0 0x55c8651bd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/872085155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c865195d10 con 0x55c86519bab0 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651bd190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b470 tx=0x7f6db4004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bb2ad2fb05b67a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6da37fe640 1 -- 172.21.15.32:0/872085155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400b960 con 0x55c86519bab0 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6da37fe640 1 -- 172.21.15.32:0/872085155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db4015070 con 0x55c86519bab0 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.524+0000 7f6da37fe640 1 -- 172.21.15.32:0/872085155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db40117c0 con 0x55c86519bab0 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 -- 172.21.15.32:0/872085155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 msgr2=0x55c8651bd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651bd190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b470 tx=0x7f6db4004000 comp rx=0 tx=0).stop 2024-04-16T05:37:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 -- 172.21.15.32:0/872085155 shutdown_connections 2024-04-16T05:37:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c86519c3d0 0x55c8651bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651bd190 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/872085155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8651961e0 0x55c8651b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 -- 172.21.15.32:0/872085155 >> 172.21.15.32:0/872085155 conn(0x55c8651a0cc0 msgr2=0x55c8651949c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 -- 172.21.15.32:0/872085155 shutdown_connections 2024-04-16T05:37:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 -- 172.21.15.32:0/872085155 wait complete. 2024-04-16T05:37:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 -- start start 2024-04-16T05:37:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.525+0000 7f6dc27d6740 1 Processor -- start 2024-04-16T05:37:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8651961e0 0x55c8651ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dc27d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651aa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81b40a0 con 0x55c86519bab0 2024-04-16T05:37:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81b5510 con 0x55c8651961e0 2024-04-16T05:37:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dc27d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81b4770 con 0x55c86519c3d0 2024-04-16T05:37:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dc27d6740 10 client.0 _ll_register_callbacks cb 0x7f6db40041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651a9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36818/0 (socket says 172.21.15.32:36818) 2024-04-16T05:37:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/4277032715 learned_addr learned my addr 172.21.15.32:0/4277032715 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8651961e0 0x55c8651ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/4277032715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 msgr2=0x55c8651aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/4277032715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8651961e0 msgr2=0x55c8651ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8651961e0 0x55c8651ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 -- 172.21.15.32:0/4277032715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86521ba10 con 0x55c86519bab0 2024-04-16T05:37:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dbd8b7640 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651a9a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6db4000c00 tx=0x7f6db40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0645a8ef764e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6da17fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6da17fa640 1 -- 172.21.15.32:0/4277032715 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4010bf0 con 0x55c86519bab0 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6da17fa640 1 -- 172.21.15.32:0/4277032715 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6db4010d90 con 0x55c86519bab0 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.526+0000 7f6dc27d6740 1 -- 172.21.15.32:0/4277032715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db4015070 con 0x55c86519bab0 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6da17fa640 1 -- 172.21.15.32:0/4277032715 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6db4011440 con 0x55c86519bab0 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dc27d6740 10 client.16233 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dc27d6740 1 -- 172.21.15.32:0/4277032715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c86521cbd0 con 0x55c86519bab0 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6da17fa640 1 -- 172.21.15.32:0/4277032715 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db4015070 con 0x55c86519bab0 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6da17fa640 1 -- 172.21.15.32:0/4277032715 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6db401a880 con 0x55c86519bab0 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dc27d6740 10 client.16233 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dc27d6740 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dc27d6740 20 client.16233 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c8651a4fb0 0x55c86521f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dc27d6740 4 client.16233 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dc27d6740 1 -- 172.21.15.32:0/4277032715 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c865108720 con 0x55c8651a4fb0 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.527+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c8651a4fb0 0x55c86521f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.528+0000 7f6da17fa640 10 client.16233 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.528+0000 7f6dbd0b6640 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c8651a4fb0 0x55c86521f240 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287a0e9d28c33d33 server_cookie=e7aff5da03af5ece in_seq=0 out_seq=0 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6da17fa640 1 -- 172.21.15.32:0/4277032715 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c865108720 con 0x55c8651a4fb0 2024-04-16T05:37:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6da17fa640 10 client.16233 handle_command_reply: tid=0 2024-04-16T05:37:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 client.16233 shutdown 2024-04-16T05:37:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 -- 172.21.15.32:0/4277032715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 msgr2=0x55c8651a9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651a9a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6db4000c00 tx=0x7f6db40046c0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 -- 172.21.15.32:0/4277032715 shutdown_connections 2024-04-16T05:37:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c8651a4fb0 0x55c86521f240 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c86519c3d0 0x55c8651aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c86519bab0 0x55c8651a9a50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 --2- 172.21.15.32:0/4277032715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8651961e0 0x55c8651ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6da17fa640 0 client.16233 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:11.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 -- 172.21.15.32:0/4277032715 >> 172.21.15.32:0/4277032715 conn(0x55c8651a0cc0 msgr2=0x55c865195250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 -- 172.21.15.32:0/4277032715 shutdown_connections 2024-04-16T05:37:11.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 1 -- 172.21.15.32:0/4277032715 wait complete. 2024-04-16T05:37:11.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.529+0000 7f6dc27d6740 20 client.16233 trim_cache size 0 max 16384 2024-04-16T05:37:11.537 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:37:11.538 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:37:11.538 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "4869a531-4bdd-43eb-a1ad-06753c739078", 2024-04-16T05:37:11.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:37:11.538 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:37:11.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6da40a3d20 msgr2=0x7f6da40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6da40a3d20 0x7f6da40a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6db00bea90 tx=0x7f6db002c180 comp rx=0 tx=0).stop 2024-04-16T05:37:11.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 msgr2=0x7f6db81a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 0x7f6db81a37d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6db00348f0 tx=0x7f6db0035e70 comp rx=0 tx=0).stop 2024-04-16T05:37:11.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/238095928 shutdown_connections 2024-04-16T05:37:11.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6da40a3d20 0x7f6da40a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db8112fe0 0x7f6db81a37d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6db8074cc0 0x7f6db819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 --2- 172.21.15.32:0/238095928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6db80743a0 0x7f6db819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/238095928 >> 172.21.15.32:0/238095928 conn(0x7f6db80fda40 msgr2=0x7f6db80fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/238095928 shutdown_connections 2024-04-16T05:37:11.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.530+0000 7f6dbfb1b640 1 -- 172.21.15.32:0/238095928 wait complete. 2024-04-16T05:37:11.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:37:11.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:37:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 -- 172.21.15.32:0/1846707754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008101550 msgr2=0x7f4008106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 --2- 172.21.15.32:0/1846707754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008101550 0x7f4008106c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009900 tx=0x7f3ff802bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 -- 172.21.15.32:0/1846707754 shutdown_connections 2024-04-16T05:37:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 --2- 172.21.15.32:0/1846707754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4008107360 0x7f400810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 --2- 172.21.15.32:0/1846707754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008101550 0x7f4008106c80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 --2- 172.21.15.32:0/1846707754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4008100ba0 0x7f4008100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 -- 172.21.15.32:0/1846707754 >> 172.21.15.32:0/1846707754 conn(0x7f4008076190 msgr2=0x7f40080765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 -- 172.21.15.32:0/1846707754 shutdown_connections 2024-04-16T05:37:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.650+0000 7f400f938640 1 -- 172.21.15.32:0/1846707754 wait complete. 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400f938640 1 Processor -- start 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400f938640 1 -- start start 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400f938640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4008100ba0 0x7f400819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400f938640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4008107360 0x7f400819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400f938640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008198730 0x7f4008198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400f938640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4008112430 con 0x7f4008198730 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400f938640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f40081122b0 con 0x7f4008107360 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400f938640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f40081125b0 con 0x7f4008100ba0 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008198730 0x7f4008198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008198730 0x7f4008198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36826/0 (socket says 172.21.15.32:36826) 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 -- 172.21.15.32:0/273066140 learned_addr learned my addr 172.21.15.32:0/273066140 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400d6d4640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4008100ba0 0x7f400819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ced3640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4008107360 0x7f400819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 -- 172.21.15.32:0/273066140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4008100ba0 msgr2=0x7f400819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4008100ba0 0x7f400819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 -- 172.21.15.32:0/273066140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4008107360 msgr2=0x7f400819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4008107360 0x7f400819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 -- 172.21.15.32:0/273066140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4008199330 con 0x7f4008198730 2024-04-16T05:37:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ced3640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4008107360 0x7f400819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400d6d4640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4008100ba0 0x7f400819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.651+0000 7f400ded5640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008198730 0x7f4008198bb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f400000eaa0 tx=0x7f400000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.652+0000 7f3ffe7fc640 1 -- 172.21.15.32:0/273066140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400000cdb0 con 0x7f4008198730 2024-04-16T05:37:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.652+0000 7f400f938640 1 -- 172.21.15.32:0/273066140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40081a5410 con 0x7f4008198730 2024-04-16T05:37:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.652+0000 7f3ffe7fc640 1 -- 172.21.15.32:0/273066140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4000005020 con 0x7f4008198730 2024-04-16T05:37:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.652+0000 7f3ffe7fc640 1 -- 172.21.15.32:0/273066140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40000116d0 con 0x7f4008198730 2024-04-16T05:37:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.652+0000 7f400f938640 1 -- 172.21.15.32:0/273066140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40081a5870 con 0x7f4008198730 2024-04-16T05:37:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f3ffe7fc640 1 -- 172.21.15.32:0/273066140 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f40000054d0 con 0x7f4008198730 2024-04-16T05:37:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f40125f4740 1 Processor -- start 2024-04-16T05:37:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f40125f4740 1 -- start start 2024-04-16T05:37:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f40125f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa433e50 0x564caa434230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f40125f4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa434770 0x564caa43ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f40125f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f40125f4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564caa399180 con 0x564caa434770 2024-04-16T05:37:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f40125f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564caa336360 con 0x564caa43d350 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.654+0000 7f40125f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564caa395140 con 0x564caa433e50 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f3ffe7fc640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3ff40a3d20 0x7f3ff40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa434770 0x564caa43ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa434770 0x564caa43ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36842/0 (socket says 172.21.15.32:36842) 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 -- 172.21.15.32:0/531694609 learned_addr learned my addr 172.21.15.32:0/531694609 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f3ffe7fc640 1 -- 172.21.15.32:0/273066140 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f40000d0df0 con 0x7f4008198730 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400ded5640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3ff40a3d20 0x7f3ff40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400ced3640 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa433e50 0x564caa434230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 -- 172.21.15.32:0/531694609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa433e50 msgr2=0x564caa434230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa433e50 0x564caa434230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400ced3640 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 -- 172.21.15.32:0/531694609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 msgr2=0x564caa43f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 -- 172.21.15.32:0/531694609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564caa1757d0 con 0x564caa434770 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400ced3640 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400ded5640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3ff40a3d20 0x7f3ff40a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40000a5190 tx=0x7f4000085f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f400d6d4640 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa434770 0x564caa43ce10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f400400cca0 tx=0x7f400400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b7e65f100cdfea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f3ff37fe640 1 -- 172.21.15.32:0/531694609 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004005220 con 0x564caa434770 2024-04-16T05:37:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f3ff37fe640 1 -- 172.21.15.32:0/531694609 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4004004c80 con 0x564caa434770 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.655+0000 7f3ff37fe640 1 -- 172.21.15.32:0/531694609 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040108f0 con 0x564caa434770 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 -- 172.21.15.32:0/531694609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa434770 msgr2=0x564caa43ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa434770 0x564caa43ce10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f400400cca0 tx=0x7f400400e720 comp rx=0 tx=0).stop 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 -- 172.21.15.32:0/531694609 shutdown_connections 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa434770 0x564caa43ce10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 --2- 172.21.15.32:0/531694609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa433e50 0x564caa434230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 -- 172.21.15.32:0/531694609 >> 172.21.15.32:0/531694609 conn(0x564caa42e580 msgr2=0x564caa42e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 -- 172.21.15.32:0/531694609 shutdown_connections 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 -- 172.21.15.32:0/531694609 wait complete. 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 -- start start 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.656+0000 7f40125f4740 1 Processor -- start 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f40125f4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa43a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f40125f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa43a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f40125f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f40125f4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564caa440140 con 0x564caa433e50 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f40125f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564caa3522c0 con 0x564caa43d350 2024-04-16T05:37:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f40125f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564caa4402c0 con 0x564caa434770 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f40125f4740 10 client.0 _ll_register_callbacks cb 0x564caa339090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa43a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa43a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36844/0 (socket says 172.21.15.32:36844) 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 -- 172.21.15.32:0/3360340766 learned_addr learned my addr 172.21.15.32:0/3360340766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400d6d4640 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa43a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 -- 172.21.15.32:0/3360340766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 msgr2=0x564caa43a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa43a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 -- 172.21.15.32:0/3360340766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 msgr2=0x564caa43ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 -- 172.21.15.32:0/3360340766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564caa0ab3f0 con 0x564caa433e50 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f400ced3640 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa43a3d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005c20 tx=0x7f3ff8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29c90cc9f5d349a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.657+0000 7f3ff0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3360340766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8049060 con 0x564caa433e50 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3360340766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f3ff8005040 con 0x564caa433e50 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 1 -- 172.21.15.32:0/3360340766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4004004c80 con 0x564caa433e50 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3360340766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ff8005240 con 0x564caa433e50 2024-04-16T05:37:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 10 client.16245 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 1 -- 172.21.15.32:0/3360340766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564caa06b0f0 con 0x564caa433e50 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3360340766 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4004004c80 con 0x564caa433e50 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3360340766 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3ff80035f0 con 0x564caa433e50 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 10 client.16245 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 20 client.16245 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564caa44d690 0x564caa44fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 4 client.16245 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f40125f4740 1 -- 172.21.15.32:0/3360340766 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564caa3a6ba0 con 0x564caa44d690 2024-04-16T05:37:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.658+0000 7f400d6d4640 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564caa44d690 0x564caa44fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.659+0000 7f400d6d4640 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564caa44d690 0x564caa44fa80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f516b80178be45d server_cookie=9f2e834650a174b4 in_seq=0 out_seq=0 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.659+0000 7f3ff0ff9640 10 client.16245 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3360340766 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564caa3a6ba0 con 0x564caa44d690 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f3ff0ff9640 10 client.16245 handle_command_reply: tid=0 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 client.16245 shutdown 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 -- 172.21.15.32:0/3360340766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 msgr2=0x564caa43a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa43a3d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005c20 tx=0x7f3ff8009260 comp rx=0 tx=0).stop 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 -- 172.21.15.32:0/3360340766 shutdown_connections 2024-04-16T05:37:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa43d350 0x564caa43ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564caa44d690 0x564caa44fa80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa43a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 --2- 172.21.15.32:0/3360340766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa43a3d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f3ff0ff9640 0 client.16245 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 -- 172.21.15.32:0/3360340766 >> 172.21.15.32:0/3360340766 conn(0x564caa42e580 msgr2=0x564caa4302d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 -- 172.21.15.32:0/3360340766 shutdown_connections 2024-04-16T05:37:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 1 -- 172.21.15.32:0/3360340766 wait complete. 2024-04-16T05:37:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.660+0000 7f40125f4740 20 client.16245 trim_cache size 0 max 16384 2024-04-16T05:37:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f40125f4740 1 Processor -- start 2024-04-16T05:37:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f40125f4740 1 -- start start 2024-04-16T05:37:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f40125f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa433e50 0x564caa449000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f40125f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa42d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f40125f4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa43d350 0x564caa43ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f40125f4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40081b1cb0 con 0x564caa43d350 2024-04-16T05:37:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f40125f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f40081af580 con 0x564caa433e50 2024-04-16T05:37:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f40125f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f40081ad3d0 con 0x564caa434770 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa43d350 0x564caa43ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa43d350 0x564caa43ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36854/0 (socket says 172.21.15.32:36854) 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 -- 172.21.15.32:0/1861602499 learned_addr learned my addr 172.21.15.32:0/1861602499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400d6d4640 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa42d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 -- 172.21.15.32:0/1861602499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 msgr2=0x564caa42d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa42d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 -- 172.21.15.32:0/1861602499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa433e50 msgr2=0x564caa449000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa433e50 0x564caa449000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 -- 172.21.15.32:0/1861602499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564caa4500e0 con 0x564caa43d350 2024-04-16T05:37:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400d6d4640 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa42d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f400ced3640 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa43d350 0x564caa43ac70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3ff802cfd0 tx=0x7f3ff8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c55ad780888eb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.678+0000 7f3ff2ffd640 1 -- 172.21.15.32:0/1861602499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8049070 con 0x564caa43d350 2024-04-16T05:37:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f3ff2ffd640 1 -- 172.21.15.32:0/1861602499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff8009520 con 0x564caa43d350 2024-04-16T05:37:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f3ff2ffd640 1 -- 172.21.15.32:0/1861602499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8000e60 con 0x564caa43d350 2024-04-16T05:37:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 -- 172.21.15.32:0/1861602499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa43d350 msgr2=0x564caa43ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa43d350 0x564caa43ac70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3ff802cfd0 tx=0x7f3ff8005fd0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 -- 172.21.15.32:0/1861602499 shutdown_connections 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa43d350 0x564caa43ac70 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa434770 0x564caa42d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 --2- 172.21.15.32:0/1861602499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa433e50 0x564caa449000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 -- 172.21.15.32:0/1861602499 >> 172.21.15.32:0/1861602499 conn(0x564caa42e580 msgr2=0x564caa439310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 -- 172.21.15.32:0/1861602499 shutdown_connections 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 -- 172.21.15.32:0/1861602499 wait complete. 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.679+0000 7f40125f4740 1 -- start start 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f40125f4740 1 Processor -- start 2024-04-16T05:37:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f40125f4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa4457c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f40125f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa434770 0x564caa442ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f40125f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa43d350 0x564caa4430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f40125f4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40081ada40 con 0x564caa433e50 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f40125f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f40081aeeb0 con 0x564caa434770 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f40125f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f40081ae110 con 0x564caa43d350 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f40125f4740 10 client.0 _ll_register_callbacks cb 0x7f3ff8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa4457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa4457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36868/0 (socket says 172.21.15.32:36868) 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 -- 172.21.15.32:0/3221055392 learned_addr learned my addr 172.21.15.32:0/3221055392 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa43d350 0x564caa4430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400d6d4640 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa434770 0x564caa442ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 -- 172.21.15.32:0/3221055392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa43d350 msgr2=0x564caa4430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa43d350 0x564caa4430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 -- 172.21.15.32:0/3221055392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa434770 msgr2=0x564caa442ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa434770 0x564caa442ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.680+0000 7f400ced3640 1 -- 172.21.15.32:0/3221055392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564caa4437d0 con 0x564caa433e50 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f400ced3640 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa4457c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3ff802c970 tx=0x7f3ff802cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddaf6581d5a85176 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f3ff0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3221055392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8049040 con 0x564caa433e50 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3221055392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f3ff80055f0 con 0x564caa433e50 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 1 -- 172.21.15.32:0/3221055392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff8009520 con 0x564caa433e50 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3221055392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ff80046c0 con 0x564caa433e50 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 10 client.16251 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 1 -- 172.21.15.32:0/3221055392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564caa4b3a10 con 0x564caa433e50 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3221055392 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff8009520 con 0x564caa433e50 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3221055392 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3ff8042590 con 0x564caa433e50 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 10 client.16251 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 20 client.16251 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564caa4b4070 0x564caa4b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 4 client.16251 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.681+0000 7f40125f4740 1 -- 172.21.15.32:0/3221055392 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564caa3a0a10 con 0x564caa4b4070 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f400d6d4640 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564caa4b4070 0x564caa4b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f3ff0ff9640 10 client.16251 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f400d6d4640 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564caa4b4070 0x564caa4b6460 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad01a3e1d96a35 server_cookie=40d04fa454f93a7c in_seq=0 out_seq=0 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f3ff0ff9640 1 -- 172.21.15.32:0/3221055392 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564caa3a0a10 con 0x564caa4b4070 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f3ff0ff9640 10 client.16251 handle_command_reply: tid=0 2024-04-16T05:37:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 client.16251 shutdown 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 -- 172.21.15.32:0/3221055392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 msgr2=0x564caa4457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa4457c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3ff802c970 tx=0x7f3ff802cdd0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 -- 172.21.15.32:0/3221055392 shutdown_connections 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564caa4b4070 0x564caa4b6460 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564caa43d350 0x564caa4430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564caa434770 0x564caa442ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f3ff0ff9640 0 client.16251 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 --2- 172.21.15.32:0/3221055392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564caa433e50 0x564caa4457c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 -- 172.21.15.32:0/3221055392 >> 172.21.15.32:0/3221055392 conn(0x564caa42e580 msgr2=0x564caa42d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 -- 172.21.15.32:0/3221055392 shutdown_connections 2024-04-16T05:37:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.682+0000 7f40125f4740 1 -- 172.21.15.32:0/3221055392 wait complete. 2024-04-16T05:37:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f40125f4740 20 client.16251 trim_cache size 0 max 16384 2024-04-16T05:37:11.693 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:37:11.693 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:37:11.693 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:37:11.693 INFO:teuthology.orchestra.run.smithi032.stdout: "4869a531-4bdd-43eb-a1ad-06753c739078": { 2024-04-16T05:37:11.694 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:37:11.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "4869a531-4bdd-43eb-a1ad-06753c739078", 2024-04-16T05:37:11.694 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:37:11.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:37:11.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:37:11.694 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:37:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 -- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3ff40a3d20 msgr2=0x7f3ff40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3ff40a3d20 0x7f3ff40a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40000a5190 tx=0x7f4000085f90 comp rx=0 tx=0).stop 2024-04-16T05:37:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 -- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008198730 msgr2=0x7f4008198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008198730 0x7f4008198bb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f400000eaa0 tx=0x7f400000ef60 comp rx=0 tx=0).stop 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 -- 172.21.15.32:0/273066140 shutdown_connections 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3ff40a3d20 0x7f3ff40a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4008198730 0x7f4008198bb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4008107360 0x7f400819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 --2- 172.21.15.32:0/273066140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4008100ba0 0x7f400819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 -- 172.21.15.32:0/273066140 >> 172.21.15.32:0/273066140 conn(0x7f4008076190 msgr2=0x7f40080feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 -- 172.21.15.32:0/273066140 shutdown_connections 2024-04-16T05:37:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:11.683+0000 7f400f938640 1 -- 172.21.15.32:0/273066140 wait complete. 2024-04-16T05:37:11.697 INFO:tasks.cephfs.filesystem:scrub status for tag:4869a531-4bdd-43eb-a1ad-06753c739078 - {'path': '/', 'tag': '4869a531-4bdd-43eb-a1ad-06753c739078', 'options': 'recursive,force'} 2024-04-16T05:37:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:11 smithi032 ceph-mon[32351]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:37:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:11 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:37:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:11 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:11 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:37:11.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:11 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:11 smithi134 ceph-mon[36153]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:37:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:11 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:37:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:11 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:11 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:37:12.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:11 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:11 smithi082 ceph-mon[36195]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:37:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:11 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:37:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:11 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:11 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:37:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:11 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:13.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:12 smithi134 ceph-mon[36153]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:37:13.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:12 smithi082 ceph-mon[36195]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:37:13.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:12 smithi032 ceph-mon[32351]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:37:15.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:15 smithi134 ceph-mon[36153]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-16T05:37:15.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:15 smithi082 ceph-mon[36195]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-16T05:37:15.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:15 smithi032 ceph-mon[32351]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-16T05:37:16.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:16 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:37:16.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:16 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:37:16.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:16 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:37:17.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:17 smithi134 ceph-mon[36153]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-16T05:37:17.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:17 smithi082 ceph-mon[36195]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-16T05:37:17.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:17 smithi032 ceph-mon[32351]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-16T05:37:19.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:19 smithi134 ceph-mon[36153]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:37:19.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:19 smithi082 ceph-mon[36195]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:37:19.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:19 smithi032 ceph-mon[32351]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:37:21.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:21 smithi134 ceph-mon[36153]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-16T05:37:21.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:21 smithi082 ceph-mon[36195]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-16T05:37:21.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:21 smithi032 ceph-mon[32351]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-16T05:37:23.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:23 smithi134 ceph-mon[36153]: pgmap v76: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:23.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:23 smithi082 ceph-mon[36195]: pgmap v76: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:23.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:23 smithi032 ceph-mon[32351]: pgmap v76: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:25.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:25 smithi134 ceph-mon[36153]: pgmap v77: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:25.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:25 smithi082 ceph-mon[36195]: pgmap v77: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:25.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:25 smithi032 ceph-mon[32351]: pgmap v77: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:27.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:27 smithi134 ceph-mon[36153]: pgmap v78: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:27.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:27 smithi082 ceph-mon[36195]: pgmap v78: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:27.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:27 smithi032 ceph-mon[32351]: pgmap v78: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:29.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:29 smithi134 ceph-mon[36153]: pgmap v79: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:29.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:29 smithi082 ceph-mon[36195]: pgmap v79: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:29.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:29 smithi032 ceph-mon[32351]: pgmap v79: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:31.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:31 smithi082 ceph-mon[36195]: pgmap v80: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:31.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:31 smithi032 ceph-mon[32351]: pgmap v80: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:31.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:31 smithi134 ceph-mon[36153]: pgmap v80: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:33.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:33 smithi082 ceph-mon[36195]: pgmap v81: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:33.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:33 smithi032 ceph-mon[32351]: pgmap v81: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:33.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:33 smithi134 ceph-mon[36153]: pgmap v81: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:37:35.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:35 smithi082 ceph-mon[36195]: pgmap v82: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:35.436 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:35 smithi134 ceph-mon[36153]: pgmap v82: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:35.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:35 smithi032 ceph-mon[32351]: pgmap v82: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:37.401 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:37 smithi082 ceph-mon[36195]: pgmap v83: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:37.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:37 smithi032 ceph-mon[32351]: pgmap v83: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:37.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:37 smithi134 ceph-mon[36153]: pgmap v83: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:37:39.400 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:39 smithi082 ceph-mon[36195]: pgmap v84: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:37:39.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:39 smithi032 ceph-mon[32351]: pgmap v84: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:37:39.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:39 smithi134 ceph-mon[36153]: pgmap v84: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:37:41.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:41 smithi032 ceph-mon[32351]: pgmap v85: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:41.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:41 smithi134 ceph-mon[36153]: pgmap v85: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:41.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:41 smithi082 ceph-mon[36195]: pgmap v85: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:41.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:37:41.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:37:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.820+0000 7f91a7ee4640 1 -- 172.21.15.32:0/489149940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0101700 msgr2=0x7f91a010e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.820+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/489149940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0101700 0x7f91a010e450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9190009900 tx=0x7f919002bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 -- 172.21.15.32:0/489149940 shutdown_connections 2024-04-16T05:37:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/489149940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91a010e990 0x7f91a0110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/489149940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0101700 0x7f91a010e450 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/489149940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91a0100de0 0x7f91a01011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 -- 172.21.15.32:0/489149940 >> 172.21.15.32:0/489149940 conn(0x7f91a0077280 msgr2=0x7f91a0077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 -- 172.21.15.32:0/489149940 shutdown_connections 2024-04-16T05:37:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 -- 172.21.15.32:0/489149940 wait complete. 2024-04-16T05:37:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 Processor -- start 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 -- start start 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0100de0 0x7f91a01a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91a0101700 0x7f91a01a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91a010e990 0x7f91a019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01145e0 con 0x7f91a0100de0 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0114460 con 0x7f91a010e990 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.821+0000 7f91a7ee4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0114760 con 0x7f91a0101700 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0100de0 0x7f91a01a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0100de0 0x7f91a01a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46326/0 (socket says 172.21.15.32:46326) 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 -- 172.21.15.32:0/3133882922 learned_addr learned my addr 172.21.15.32:0/3133882922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a547f640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91a0101700 0x7f91a01a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a6481640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91a010e990 0x7f91a019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 -- 172.21.15.32:0/3133882922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91a0101700 msgr2=0x7f91a01a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91a0101700 0x7f91a01a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 -- 172.21.15.32:0/3133882922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91a010e990 msgr2=0x7f91a019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91a010e990 0x7f91a019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 -- 172.21.15.32:0/3133882922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a019b180 con 0x7f91a0100de0 2024-04-16T05:37:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a5c80640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0100de0 0x7f91a01a0850 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f919c00d950 tx=0x7f919c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f9196ffd640 1 -- 172.21.15.32:0/3133882922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c017070 con 0x7f91a0100de0 2024-04-16T05:37:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f9196ffd640 1 -- 172.21.15.32:0/3133882922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919c009c30 con 0x7f91a0100de0 2024-04-16T05:37:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a7ee4640 1 -- 172.21.15.32:0/3133882922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a019b470 con 0x7f91a0100de0 2024-04-16T05:37:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f9196ffd640 1 -- 172.21.15.32:0/3133882922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c00f6b0 con 0x7f91a0100de0 2024-04-16T05:37:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.822+0000 7f91a7ee4640 1 -- 172.21.15.32:0/3133882922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a01a7710 con 0x7f91a0100de0 2024-04-16T05:37:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.825+0000 7f9196ffd640 1 -- 172.21.15.32:0/3133882922 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f919c0109d0 con 0x7f91a0100de0 2024-04-16T05:37:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.825+0000 7f91aabad740 1 Processor -- start 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91aabad740 1 -- start start 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91aabad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91aabad740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91aabad740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3d8350 0x55b7eb3da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91aabad740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b7eb334180 con 0x55b7eb3cf770 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91aabad740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b7eb2d1360 con 0x55b7eb3d8350 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91aabad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b7eb330140 con 0x55b7eb3cee50 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a5c80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a5c80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46328/0 (socket says 172.21.15.32:46328) 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a5c80640 1 -- 172.21.15.32:0/2785046194 learned_addr learned my addr 172.21.15.32:0/2785046194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a6481640 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a547f640 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3d8350 0x55b7eb3da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f9196ffd640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f918c0a3d20 0x7f918c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a5c80640 1 -- 172.21.15.32:0/2785046194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 msgr2=0x55b7eb3cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a5c80640 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a5c80640 1 -- 172.21.15.32:0/2785046194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3d8350 msgr2=0x55b7eb3da740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a5c80640 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3d8350 0x55b7eb3da740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a5c80640 1 -- 172.21.15.32:0/2785046194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7eb1107d0 con 0x55b7eb3cf770 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.826+0000 7f91a6481640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f918c0a3d20 0x7f918c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.827+0000 7f9196ffd640 1 -- 172.21.15.32:0/3133882922 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f919c0d0790 con 0x7f91a0100de0 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.827+0000 7f91a5c80640 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d7e10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f919c0cdb10 tx=0x7f919c06de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6a26094827a31c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.827+0000 7f918bfff640 1 -- 172.21.15.32:0/2785046194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c020020 con 0x55b7eb3cf770 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.827+0000 7f918bfff640 1 -- 172.21.15.32:0/2785046194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919c09caf0 con 0x55b7eb3cf770 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.827+0000 7f918bfff640 1 -- 172.21.15.32:0/2785046194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c09d4e0 con 0x55b7eb3cf770 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 -- 172.21.15.32:0/2785046194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 msgr2=0x55b7eb3d7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91a6481640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f918c0a3d20 0x7f918c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91980094d0 tx=0x7f9198004a40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d7e10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f919c0cdb10 tx=0x7f919c06de30 comp rx=0 tx=0).stop 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 -- 172.21.15.32:0/2785046194 shutdown_connections 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3d8350 0x55b7eb3da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d7e10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 --2- 172.21.15.32:0/2785046194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 -- 172.21.15.32:0/2785046194 >> 172.21.15.32:0/2785046194 conn(0x55b7eb3c9580 msgr2=0x55b7eb3c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 -- 172.21.15.32:0/2785046194 shutdown_connections 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 -- 172.21.15.32:0/2785046194 wait complete. 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 -- start start 2024-04-16T05:37:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.828+0000 7f91aabad740 1 Processor -- start 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91aabad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91aabad740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91aabad740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3d8350 0x55b7eb3d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91aabad740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b7eb3db080 con 0x55b7eb3d8350 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91aabad740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b7eb2ed2c0 con 0x55b7eb3cf770 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91aabad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b7eb3db200 con 0x55b7eb3cee50 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91aabad740 10 client.0 _ll_register_callbacks cb 0x55b7eb2d4090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3d8350 0x55b7eb3d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3d8350 0x55b7eb3d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46332/0 (socket says 172.21.15.32:46332) 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 -- 172.21.15.32:0/2831200571 learned_addr learned my addr 172.21.15.32:0/2831200571 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a547f640 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a5c80640 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 -- 172.21.15.32:0/2831200571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 msgr2=0x55b7eb3d5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 -- 172.21.15.32:0/2831200571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cf770 msgr2=0x55b7eb3d5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 -- 172.21.15.32:0/2831200571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7eb0463f0 con 0x55b7eb3d8350 2024-04-16T05:37:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a5c80640 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91a6481640 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3d8350 0x55b7eb3d5dc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9198011bf0 tx=0x7f9198004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beee91324545a123 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91897fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91897fa640 1 -- 172.21.15.32:0/2831200571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198027440 con 0x55b7eb3d8350 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.829+0000 7f91897fa640 1 -- 172.21.15.32:0/2831200571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f919801b410 con 0x55b7eb3d8350 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 1 -- 172.21.15.32:0/2831200571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f919c09caf0 con 0x55b7eb3d8350 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91897fa640 1 -- 172.21.15.32:0/2831200571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f919801b610 con 0x55b7eb3d8350 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 10 client.16263 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 1 -- 172.21.15.32:0/2831200571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7eb0060f0 con 0x55b7eb3d8350 2024-04-16T05:37:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91897fa640 1 -- 172.21.15.32:0/2831200571 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919c09caf0 con 0x55b7eb3d8350 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91897fa640 1 -- 172.21.15.32:0/2831200571 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9198025070 con 0x55b7eb3d8350 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 10 client.16263 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 20 client.16263 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b7eb3e8670 0x55b7eb3eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 4 client.16263 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91aabad740 1 -- 172.21.15.32:0/2831200571 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7eb341ba0 con 0x55b7eb3e8670 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.830+0000 7f91a5c80640 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b7eb3e8670 0x55b7eb3eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.831+0000 7f91897fa640 10 client.16263 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.831+0000 7f91a5c80640 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b7eb3e8670 0x55b7eb3eaae0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1820747914fbf060 server_cookie=cc05f34005357244 in_seq=0 out_seq=0 2024-04-16T05:37:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91897fa640 1 -- 172.21.15.32:0/2831200571 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7eb341ba0 con 0x55b7eb3e8670 2024-04-16T05:37:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91897fa640 10 client.16263 handle_command_reply: tid=0 2024-04-16T05:37:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 client.16263 shutdown 2024-04-16T05:37:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 -- 172.21.15.32:0/2831200571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3d8350 msgr2=0x55b7eb3d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3d8350 0x55b7eb3d5dc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9198011bf0 tx=0x7f9198004ab0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 -- 172.21.15.32:0/2831200571 shutdown_connections 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3d8350 0x55b7eb3d5dc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cf770 0x55b7eb3d5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b7eb3e8670 0x55b7eb3eaae0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 --2- 172.21.15.32:0/2831200571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91897fa640 0 client.16263 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 -- 172.21.15.32:0/2831200571 >> 172.21.15.32:0/2831200571 conn(0x55b7eb3c9580 msgr2=0x55b7eb3cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 -- 172.21.15.32:0/2831200571 shutdown_connections 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 1 -- 172.21.15.32:0/2831200571 wait complete. 2024-04-16T05:37:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.832+0000 7f91aabad740 20 client.16263 trim_cache size 0 max 16384 2024-04-16T05:37:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91aabad740 1 Processor -- start 2024-04-16T05:37:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91aabad740 1 -- start start 2024-04-16T05:37:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91aabad740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91aabad740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91aabad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 0x55b7eb3c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91aabad740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01b3f50 con 0x55b7eb3cf770 2024-04-16T05:37:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91aabad740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01b1820 con 0x55b7eb3cee50 2024-04-16T05:37:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91aabad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01af670 con 0x55b7eb3d8350 2024-04-16T05:37:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a547f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a547f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3e3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46348/0 (socket says 172.21.15.32:46348) 2024-04-16T05:37:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a547f640 1 -- 172.21.15.32:0/200067791 learned_addr learned my addr 172.21.15.32:0/200067791 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a5c80640 1 --2- 172.21.15.32:0/200067791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a547f640 1 -- 172.21.15.32:0/200067791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 msgr2=0x55b7eb3c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a547f640 1 --2- 172.21.15.32:0/200067791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 0x55b7eb3c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a547f640 1 -- 172.21.15.32:0/200067791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 msgr2=0x55b7eb3d7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a547f640 1 --2- 172.21.15.32:0/200067791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.850+0000 7f91a547f640 1 -- 172.21.15.32:0/200067791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7eb3c93b0 con 0x55b7eb3cf770 2024-04-16T05:37:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f91a547f640 1 --2- 172.21.15.32:0/200067791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3e3f40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f919002c170 tx=0x7f9190005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a250fc265c96ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f918b7fe640 1 -- 172.21.15.32:0/200067791 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919002cbc0 con 0x55b7eb3cf770 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f918b7fe640 1 -- 172.21.15.32:0/200067791 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91900046c0 con 0x55b7eb3cf770 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f918b7fe640 1 -- 172.21.15.32:0/200067791 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190005290 con 0x55b7eb3cf770 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f91aabad740 1 -- 172.21.15.32:0/200067791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 msgr2=0x55b7eb3e3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f91aabad740 1 --2- 172.21.15.32:0/200067791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3e3f40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f919002c170 tx=0x7f9190005b50 comp rx=0 tx=0).stop 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f91aabad740 1 -- 172.21.15.32:0/200067791 shutdown_connections 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f91aabad740 1 --2- 172.21.15.32:0/200067791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 0x55b7eb3c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f91aabad740 1 --2- 172.21.15.32:0/200067791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3e3f40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f91aabad740 1 --2- 172.21.15.32:0/200067791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 0x55b7eb3d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.851+0000 7f91aabad740 1 -- 172.21.15.32:0/200067791 >> 172.21.15.32:0/200067791 conn(0x55b7eb3c9580 msgr2=0x55b7eb3d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 -- 172.21.15.32:0/200067791 shutdown_connections 2024-04-16T05:37:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 -- 172.21.15.32:0/200067791 wait complete. 2024-04-16T05:37:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 -- start start 2024-04-16T05:37:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 Processor -- start 2024-04-16T05:37:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 0x55b7eb3de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3de650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 0x55b7eb3deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01afce0 con 0x55b7eb3cf770 2024-04-16T05:37:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01b1150 con 0x55b7eb3cee50 2024-04-16T05:37:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01b03b0 con 0x55b7eb3d8350 2024-04-16T05:37:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91aabad740 10 client.0 _ll_register_callbacks cb 0x7f9198009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91a547f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3de650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91a547f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3de650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46352/0 (socket says 172.21.15.32:46352) 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.852+0000 7f91a547f640 1 -- 172.21.15.32:0/361191568 learned_addr learned my addr 172.21.15.32:0/361191568 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91a6481640 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 0x55b7eb3deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91a5c80640 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 0x55b7eb3de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91a547f640 1 -- 172.21.15.32:0/361191568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 msgr2=0x55b7eb3deb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91a547f640 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 0x55b7eb3deb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91a547f640 1 -- 172.21.15.32:0/361191568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 msgr2=0x55b7eb3de110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91a547f640 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 0x55b7eb3de110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91a547f640 1 -- 172.21.15.32:0/361191568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7eb44ded0 con 0x55b7eb3cf770 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91a547f640 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3de650 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9190048040 tx=0x7f9190004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21466ffe29abfeda server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91897fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91897fa640 1 -- 172.21.15.32:0/361191568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190005420 con 0x55b7eb3cf770 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91aabad740 1 -- 172.21.15.32:0/361191568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91900046c0 con 0x55b7eb3cf770 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91897fa640 1 -- 172.21.15.32:0/361191568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f91900055c0 con 0x55b7eb3cf770 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91897fa640 1 -- 172.21.15.32:0/361191568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f919005d050 con 0x55b7eb3cf770 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91aabad740 10 client.16269 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91aabad740 1 -- 172.21.15.32:0/361191568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7eb44f1f0 con 0x55b7eb3cf770 2024-04-16T05:37:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.853+0000 7f91897fa640 1 -- 172.21.15.32:0/361191568 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91900046c0 con 0x55b7eb3cf770 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91897fa640 1 -- 172.21.15.32:0/361191568 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9190034440 con 0x55b7eb3cf770 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91aabad740 10 client.16269 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91aabad740 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91aabad740 20 client.16269 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91aabad740 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b7eb44f9a0 0x55b7eb451dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91aabad740 4 client.16269 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91aabad740 1 -- 172.21.15.32:0/361191568 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7eb33ba10 con 0x55b7eb44f9a0 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91a5c80640 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b7eb44f9a0 0x55b7eb451dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91897fa640 10 client.16269 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91a5c80640 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b7eb44f9a0 0x55b7eb451dc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d9d352b21af87f server_cookie=d0f9cbef3b93a6ee in_seq=0 out_seq=0 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91897fa640 1 -- 172.21.15.32:0/361191568 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b7eb33ba10 con 0x55b7eb44f9a0 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91897fa640 10 client.16269 handle_command_reply: tid=0 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.854+0000 7f91aabad740 1 client.16269 shutdown 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 -- 172.21.15.32:0/361191568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 msgr2=0x55b7eb3de650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3de650 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9190048040 tx=0x7f9190004b20 comp rx=0 tx=0).stop 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 -- 172.21.15.32:0/361191568 shutdown_connections 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b7eb44f9a0 0x55b7eb451dc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7eb3d8350 0x55b7eb3deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7eb3cf770 0x55b7eb3de650 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91897fa640 0 client.16269 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 --2- 172.21.15.32:0/361191568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7eb3cee50 0x55b7eb3de110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 -- 172.21.15.32:0/361191568 >> 172.21.15.32:0/361191568 conn(0x55b7eb3c9580 msgr2=0x55b7eb3cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 -- 172.21.15.32:0/361191568 shutdown_connections 2024-04-16T05:37:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 1 -- 172.21.15.32:0/361191568 wait complete. 2024-04-16T05:37:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91aabad740 20 client.16269 trim_cache size 0 max 16384 2024-04-16T05:37:41.864 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:37:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:37:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:37:41.864 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:37:41.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 -- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f918c0a3d20 msgr2=0x7f918c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f918c0a3d20 0x7f918c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91980094d0 tx=0x7f9198004a40 comp rx=0 tx=0).stop 2024-04-16T05:37:41.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 -- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0100de0 msgr2=0x7f91a01a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0100de0 0x7f91a01a0850 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f919c00d950 tx=0x7f919c00de10 comp rx=0 tx=0).stop 2024-04-16T05:37:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 -- 172.21.15.32:0/3133882922 shutdown_connections 2024-04-16T05:37:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91a010e990 0x7f91a019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91a0101700 0x7f91a01a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f918c0a3d20 0x7f918c0a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 --2- 172.21.15.32:0/3133882922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91a0100de0 0x7f91a01a0850 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 -- 172.21.15.32:0/3133882922 >> 172.21.15.32:0/3133882922 conn(0x7f91a0077280 msgr2=0x7f91a0110040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 -- 172.21.15.32:0/3133882922 shutdown_connections 2024-04-16T05:37:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.855+0000 7f91a7ee4640 1 -- 172.21.15.32:0/3133882922 wait complete. 2024-04-16T05:37:41.867 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:37:41.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:37:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 -- 172.21.15.32:0/2271179203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648102c00 msgr2=0x7fb648106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 --2- 172.21.15.32:0/2271179203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648102c00 0x7fb648106c40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009900 tx=0x7fb63c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 -- 172.21.15.32:0/2271179203 shutdown_connections 2024-04-16T05:37:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 --2- 172.21.15.32:0/2271179203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648107320 0x7fb64810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 --2- 172.21.15.32:0/2271179203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648102c00 0x7fb648106c40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 --2- 172.21.15.32:0/2271179203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb648102250 0x7fb648102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 -- 172.21.15.32:0/2271179203 >> 172.21.15.32:0/2271179203 conn(0x7fb648079400 msgr2=0x7fb648079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 -- 172.21.15.32:0/2271179203 shutdown_connections 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.974+0000 7fb64e07b640 1 -- 172.21.15.32:0/2271179203 wait complete. 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb64e07b640 1 Processor -- start 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb64e07b640 1 -- start start 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb64e07b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648102250 0x7fb64819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb64e07b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648107320 0x7fb64819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb64e07b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb648198790 0x7fb648198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb64e07b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb648112430 con 0x7fb648107320 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb64e07b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481122b0 con 0x7fb648198790 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb64e07b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481125b0 con 0x7fb648102250 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.975+0000 7fb646ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648107320 0x7fb64819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb646ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648107320 0x7fb64819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46364/0 (socket says 172.21.15.32:46364) 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb646ffd640 1 -- 172.21.15.32:0/1910016109 learned_addr learned my addr 172.21.15.32:0/1910016109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb647fff640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb648198790 0x7fb648198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb646ffd640 1 -- 172.21.15.32:0/1910016109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648102250 msgr2=0x7fb64819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb646ffd640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648102250 0x7fb64819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb646ffd640 1 -- 172.21.15.32:0/1910016109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb648198790 msgr2=0x7fb648198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb646ffd640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb648198790 0x7fb648198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:41.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb646ffd640 1 -- 172.21.15.32:0/1910016109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb648199320 con 0x7fb648107320 2024-04-16T05:37:41.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb646ffd640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648107320 0x7fb64819eb60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb63c036040 tx=0x7fb63c02ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:41.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb644ff9640 1 -- 172.21.15.32:0/1910016109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c041070 con 0x7fb648107320 2024-04-16T05:37:41.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6481995b0 con 0x7fb648107320 2024-04-16T05:37:41.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb644ff9640 1 -- 172.21.15.32:0/1910016109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c0054e0 con 0x7fb648107320 2024-04-16T05:37:41.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb644ff9640 1 -- 172.21.15.32:0/1910016109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c034440 con 0x7fb648107320 2024-04-16T05:37:41.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.976+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6481a5760 con 0x7fb648107320 2024-04-16T05:37:41.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.977+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb648103720 con 0x7fb648107320 2024-04-16T05:37:41.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.978+0000 7fb644ff9640 1 -- 172.21.15.32:0/1910016109 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb63c044070 con 0x7fb648107320 2024-04-16T05:37:41.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.979+0000 7fb644ff9640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6300a3d20 0x7fb6300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:41.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.979+0000 7fb6477fe640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6300a3d20 0x7fb6300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:41.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.979+0000 7fb644ff9640 1 -- 172.21.15.32:0/1910016109 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fb63c0f2700 con 0x7fb648107320 2024-04-16T05:37:41.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.980+0000 7fb6477fe640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6300a3d20 0x7fb6300a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb638009710 tx=0x7fb638006c90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:41.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:41.984+0000 7fb644ff9640 1 -- 172.21.15.32:0/1910016109 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb63c0bec00 con 0x7fb648107320 2024-04-16T05:37:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.213+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb64810a550 con 0x7fb648107320 2024-04-16T05:37:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.214+0000 7fb644ff9640 1 -- 172.21.15.32:0/1910016109 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7fb63c004c50 con 0x7fb648107320 2024-04-16T05:37:42.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:37:42.213 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:37:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6300a3d20 msgr2=0x7fb6300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6300a3d20 0x7fb6300a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb638009710 tx=0x7fb638006c90 comp rx=0 tx=0).stop 2024-04-16T05:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648107320 msgr2=0x7fb64819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648107320 0x7fb64819eb60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb63c036040 tx=0x7fb63c02ce20 comp rx=0 tx=0).stop 2024-04-16T05:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 shutdown_connections 2024-04-16T05:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb648198790 0x7fb648198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb6300a3d20 0x7fb6300a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648107320 0x7fb64819eb60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 --2- 172.21.15.32:0/1910016109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648102250 0x7fb64819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 >> 172.21.15.32:0/1910016109 conn(0x7fb648079400 msgr2=0x7fb6481001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 shutdown_connections 2024-04-16T05:37:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.216+0000 7fb64e07b640 1 -- 172.21.15.32:0/1910016109 wait complete. 2024-04-16T05:37:42.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:37:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 -- 172.21.15.32:0/221167391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101ae0 msgr2=0x7f1eac105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 --2- 172.21.15.32:0/221167391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101ae0 0x7f1eac105b20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009900 tx=0x7f1ea002bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 -- 172.21.15.32:0/221167391 shutdown_connections 2024-04-16T05:37:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 --2- 172.21.15.32:0/221167391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac106200 0x7f1eac10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 --2- 172.21.15.32:0/221167391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101ae0 0x7f1eac105b20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 --2- 172.21.15.32:0/221167391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eac101130 0x7f1eac101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 -- 172.21.15.32:0/221167391 >> 172.21.15.32:0/221167391 conn(0x7f1eac0780b0 msgr2=0x7f1eac0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 -- 172.21.15.32:0/221167391 shutdown_connections 2024-04-16T05:37:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.335+0000 7f1eb2863640 1 -- 172.21.15.32:0/221167391 wait complete. 2024-04-16T05:37:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb2863640 1 Processor -- start 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb2863640 1 -- start start 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb2863640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101130 0x7f1eac102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb2863640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eac106200 0x7f1eac103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb2863640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac103640 0x7f1eac1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb2863640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac111290 con 0x7f1eac101130 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb2863640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac111110 con 0x7f1eac106200 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb2863640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac111410 con 0x7f1eac103640 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101130 0x7f1eac102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101130 0x7f1eac102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46390/0 (socket says 172.21.15.32:46390) 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eabfff640 1 -- 172.21.15.32:0/3676757509 learned_addr learned my addr 172.21.15.32:0/3676757509 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eac106200 0x7f1eac103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac103640 0x7f1eac1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eabfff640 1 -- 172.21.15.32:0/3676757509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac103640 msgr2=0x7f1eac1a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eabfff640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac103640 0x7f1eac1a6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eabfff640 1 -- 172.21.15.32:0/3676757509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eac106200 msgr2=0x7f1eac103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eabfff640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eac106200 0x7f1eac103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eabfff640 1 -- 172.21.15.32:0/3676757509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac1a7150 con 0x7f1eac101130 2024-04-16T05:37:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.336+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eac106200 0x7f1eac103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.337+0000 7f1eabfff640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101130 0x7f1eac102bc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00d950 tx=0x7f1e9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.337+0000 7f1ea97fa640 1 -- 172.21.15.32:0/3676757509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c017030 con 0x7f1eac101130 2024-04-16T05:37:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.337+0000 7f1ea97fa640 1 -- 172.21.15.32:0/3676757509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e9c004020 con 0x7f1eac101130 2024-04-16T05:37:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.337+0000 7f1eb2863640 1 -- 172.21.15.32:0/3676757509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac1a7440 con 0x7f1eac101130 2024-04-16T05:37:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.337+0000 7f1ea97fa640 1 -- 172.21.15.32:0/3676757509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0047a0 con 0x7f1eac101130 2024-04-16T05:37:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.337+0000 7f1eb2863640 1 -- 172.21.15.32:0/3676757509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac1a7960 con 0x7f1eac101130 2024-04-16T05:37:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.339+0000 7f1ea97fa640 1 -- 172.21.15.32:0/3676757509 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e9c0041d0 con 0x7f1eac101130 2024-04-16T05:37:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.339+0000 7f1eb5451740 1 Processor -- start 2024-04-16T05:37:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb5451740 1 -- start start 2024-04-16T05:37:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb5451740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5612822170e0 con 0x5612822b2780 2024-04-16T05:37:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb5451740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5612821b4520 con 0x5612822b1e60 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb5451740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612822130a0 con 0x5612822bb360 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1ea97fa640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e940a3d50 0x7f1e940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822bae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46396/0 (socket says 172.21.15.32:46396) 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eabfff640 1 -- 172.21.15.32:0/1013847732 learned_addr learned my addr 172.21.15.32:0/1013847732 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e940a3d50 0x7f1e940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1ea97fa640 1 -- 172.21.15.32:0/3676757509 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1e9c0d05d0 con 0x7f1eac101130 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eabfff640 1 -- 172.21.15.32:0/1013847732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 msgr2=0x5612822bd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eabfff640 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822bd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eabfff640 1 -- 172.21.15.32:0/1013847732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 msgr2=0x5612822b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eabfff640 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.340+0000 7f1eabfff640 1 -- 172.21.15.32:0/1013847732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612821dc8d0 con 0x5612822b2780 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eabfff640 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822bae20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c005fd0 tx=0x7f1e9c0051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8747f9ea6fb39398 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e940a3d50 0x7f1e940a6210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4005fd0 tx=0x7f1ea4009500 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1e927fc640 1 -- 172.21.15.32:0/1013847732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c09c230 con 0x5612822b2780 2024-04-16T05:37:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1e927fc640 1 -- 172.21.15.32:0/1013847732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e9c09c930 con 0x5612822b2780 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1e927fc640 1 -- 172.21.15.32:0/1013847732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0cddf0 con 0x5612822b2780 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eb5451740 1 -- 172.21.15.32:0/1013847732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 msgr2=0x5612822bae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eb5451740 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822bae20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c005fd0 tx=0x7f1e9c0051f0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eb5451740 1 -- 172.21.15.32:0/1013847732 shutdown_connections 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eb5451740 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822bd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eb5451740 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822bae20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eb5451740 1 --2- 172.21.15.32:0/1013847732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.341+0000 7f1eb5451740 1 -- 172.21.15.32:0/1013847732 >> 172.21.15.32:0/1013847732 conn(0x5612822ac590 msgr2=0x5612822ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 -- 172.21.15.32:0/1013847732 shutdown_connections 2024-04-16T05:37:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 -- 172.21.15.32:0/1013847732 wait complete. 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 -- start start 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 Processor -- start 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822b8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5612822bdfa0 con 0x5612822b2780 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5612821d04d0 con 0x5612822b1e60 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612822be120 con 0x5612822bb360 2024-04-16T05:37:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eb5451740 10 client.0 _ll_register_callbacks cb 0x5612821b52c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46406/0 (socket says 172.21.15.32:46406) 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.342+0000 7f1eab7fe640 1 -- 172.21.15.32:0/2443041586 learned_addr learned my addr 172.21.15.32:0/2443041586 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eabfff640 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eab7fe640 1 -- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 msgr2=0x5612822b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eab7fe640 1 -- 172.21.15.32:0/2443041586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 msgr2=0x5612822b8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822b8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eab7fe640 1 -- 172.21.15.32:0/2443041586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561281f293f0 con 0x5612822b2780 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8960 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0005940 tx=0x7f1ea0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afff465c80e3b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2443041586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0005230 con 0x5612822b2780 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2443041586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1ea00053d0 con 0x5612822b2780 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eb5451740 1 -- 172.21.15.32:0/2443041586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e9c09c930 con 0x5612822b2780 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2443041586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ea0004490 con 0x5612822b2780 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eb5451740 10 client.16287 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1eb5451740 1 -- 172.21.15.32:0/2443041586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561281ff37d0 con 0x5612822b2780 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.343+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2443041586 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e9c09c930 con 0x5612822b2780 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.344+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2443041586 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ea0003850 con 0x5612822b2780 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.344+0000 7f1eb5451740 10 client.16287 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.344+0000 7f1eb5451740 10 client.16287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.344+0000 7f1eb5451740 20 client.16287 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.344+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5612822cb360 0x5612822cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.344+0000 7f1eb5451740 4 client.16287 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.344+0000 7f1eb5451740 1 -- 172.21.15.32:0/2443041586 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561282224b00 con 0x5612822cb360 2024-04-16T05:37:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.344+0000 7f1eabfff640 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5612822cb360 0x5612822cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.345+0000 7f1e8b7fe640 10 client.16287 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.345+0000 7f1eabfff640 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5612822cb360 0x5612822cd7a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62efc5624b51e20e server_cookie=27045e7383991335 in_seq=0 out_seq=0 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.345+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2443041586 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561282224b00 con 0x5612822cb360 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.345+0000 7f1e8b7fe640 10 client.16287 handle_command_reply: tid=0 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.345+0000 7f1eb5451740 1 client.16287 shutdown 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 -- 172.21.15.32:0/2443041586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 msgr2=0x5612822b8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8960 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0005940 tx=0x7f1ea0005e30 comp rx=0 tx=0).stop 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 -- 172.21.15.32:0/2443041586 shutdown_connections 2024-04-16T05:37:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5612822cb360 0x5612822cd7a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1e8b7fe640 0 client.16287 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:37:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8960 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2443041586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822b8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 -- 172.21.15.32:0/2443041586 >> 172.21.15.32:0/2443041586 conn(0x5612822ac590 msgr2=0x5612822ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 -- 172.21.15.32:0/2443041586 shutdown_connections 2024-04-16T05:37:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 1 -- 172.21.15.32:0/2443041586 wait complete. 2024-04-16T05:37:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.346+0000 7f1eb5451740 20 client.16287 trim_cache size 0 max 16384 2024-04-16T05:37:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eb5451740 1 Processor -- start 2024-04-16T05:37:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eb5451740 1 -- start start 2024-04-16T05:37:42.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822bab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eb5451740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b3e50 con 0x5612822b2780 2024-04-16T05:37:42.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eb5451740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b1720 con 0x5612822b1e60 2024-04-16T05:37:42.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eb5451740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1af570 con 0x5612822bb360 2024-04-16T05:37:42.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46416/0 (socket says 172.21.15.32:46416) 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.359+0000 7f1eab7fe640 1 -- 172.21.15.32:0/3132274575 learned_addr learned my addr 172.21.15.32:0/3132274575 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1eabfff640 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822bab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1eab7fe640 1 -- 172.21.15.32:0/3132274575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 msgr2=0x5612822d3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822d3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1eab7fe640 1 -- 172.21.15.32:0/3132274575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 msgr2=0x5612822bab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822bab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1eab7fe640 1 -- 172.21.15.32:0/3132274575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612822be140 con 0x5612822b2780 2024-04-16T05:37:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8cc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0000c00 tx=0x7f1ea0009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c745e3f05de3c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1e91ffb640 1 -- 172.21.15.32:0/3132274575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0003ff0 con 0x5612822b2780 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1e91ffb640 1 -- 172.21.15.32:0/3132274575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea002cd40 con 0x5612822b2780 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.360+0000 7f1e91ffb640 1 -- 172.21.15.32:0/3132274575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00046c0 con 0x5612822b2780 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- 172.21.15.32:0/3132274575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 msgr2=0x5612822b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8cc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0000c00 tx=0x7f1ea0009230 comp rx=0 tx=0).stop 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- 172.21.15.32:0/3132274575 shutdown_connections 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822bb360 0x5612822d3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822b2780 0x5612822b8cc0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 --2- 172.21.15.32:0/3132274575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b1e60 0x5612822bab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- 172.21.15.32:0/3132274575 >> 172.21.15.32:0/3132274575 conn(0x5612822ac590 msgr2=0x5612822ad7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- 172.21.15.32:0/3132274575 shutdown_connections 2024-04-16T05:37:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- 172.21.15.32:0/3132274575 wait complete. 2024-04-16T05:37:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- start start 2024-04-16T05:37:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 Processor -- start 2024-04-16T05:37:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822b1e60 0x5612822c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b2780 0x5612822b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822bb360 0x5612822b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1afbe0 con 0x5612822bb360 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b1050 con 0x5612822b2780 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b02b0 con 0x5612822b1e60 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb5451740 10 client.0 _ll_register_callbacks cb 0x7f1ea0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.361+0000 7f1eb0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822bb360 0x5612822b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822bb360 0x5612822b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46420/0 (socket says 172.21.15.32:46420) 2024-04-16T05:37:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb0e00640 1 -- 172.21.15.32:0/2532677371 learned_addr learned my addr 172.21.15.32:0/2532677371 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eab7fe640 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b2780 0x5612822b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eabfff640 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822b1e60 0x5612822c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb0e00640 1 -- 172.21.15.32:0/2532677371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822b1e60 msgr2=0x5612822c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822b1e60 0x5612822c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb0e00640 1 -- 172.21.15.32:0/2532677371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b2780 msgr2=0x5612822b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b2780 0x5612822b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb0e00640 1 -- 172.21.15.32:0/2532677371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612823308e0 con 0x5612822bb360 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb0e00640 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822bb360 0x5612822b5600 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40147c0 tx=0x7f1ea4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131f8d00ac4658e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2532677371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4011e90 con 0x5612822bb360 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb5451740 1 -- 172.21.15.32:0/2532677371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea002cd40 con 0x5612822bb360 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2532677371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1ea4004b20 con 0x5612822bb360 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2532677371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ea4033050 con 0x5612822bb360 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb5451740 10 client.16293 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1eb5451740 1 -- 172.21.15.32:0/2532677371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561282331a60 con 0x5612822bb360 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.362+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2532677371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea002cd40 con 0x5612822bb360 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2532677371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ea4020040 con 0x5612822bb360 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1eb5451740 10 client.16293 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1eb5451740 10 client.16293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:37:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1eb5451740 20 client.16293 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x561282332180 0x561282334570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1eb5451740 4 client.16293 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1eb5451740 1 -- 172.21.15.32:0/2532677371 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56128221e970 con 0x561282332180 2024-04-16T05:37:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1eabfff640 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x561282332180 0x561282334570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1e8b7fe640 10 client.16293 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:37:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.363+0000 7f1eabfff640 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x561282332180 0x561282334570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b7724a0482fca41 server_cookie=8ad3386eb6f13562 in_seq=0 out_seq=0 2024-04-16T05:37:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/2532677371 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56128221e970 con 0x561282332180 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1e8b7fe640 10 client.16293 handle_command_reply: tid=0 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 client.16293 shutdown 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 -- 172.21.15.32:0/2532677371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822bb360 msgr2=0x5612822b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822bb360 0x5612822b5600 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40147c0 tx=0x7f1ea4004a40 comp rx=0 tx=0).stop 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 -- 172.21.15.32:0/2532677371 shutdown_connections 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x561282332180 0x561282334570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612822bb360 0x5612822b5600 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612822b2780 0x5612822b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1e8b7fe640 0 client.16293 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:37:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 --2- 172.21.15.32:0/2532677371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612822b1e60 0x5612822c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 -- 172.21.15.32:0/2532677371 >> 172.21.15.32:0/2532677371 conn(0x5612822ac590 msgr2=0x5612822adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 -- 172.21.15.32:0/2532677371 shutdown_connections 2024-04-16T05:37:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 1 -- 172.21.15.32:0/2532677371 wait complete. 2024-04-16T05:37:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb5451740 20 client.16293 trim_cache size 0 max 16384 2024-04-16T05:37:42.372 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:37:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb2863640 1 -- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e940a3d50 msgr2=0x7f1e940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb2863640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e940a3d50 0x7f1e940a6210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4005fd0 tx=0x7f1ea4009500 comp rx=0 tx=0).stop 2024-04-16T05:37:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb2863640 1 -- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101130 msgr2=0x7f1eac102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.364+0000 7f1eb2863640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101130 0x7f1eac102bc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00d950 tx=0x7f1e9c00de10 comp rx=0 tx=0).stop 2024-04-16T05:37:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.365+0000 7f1eb2863640 1 -- 172.21.15.32:0/3676757509 shutdown_connections 2024-04-16T05:37:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.365+0000 7f1eb2863640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac103640 0x7f1eac1a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.365+0000 7f1eb2863640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eac106200 0x7f1eac103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.365+0000 7f1eb2863640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1e940a3d50 0x7f1e940a6210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.365+0000 7f1eb2863640 1 --2- 172.21.15.32:0/3676757509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1eac101130 0x7f1eac102bc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.365+0000 7f1eb2863640 1 -- 172.21.15.32:0/3676757509 >> 172.21.15.32:0/3676757509 conn(0x7f1eac0780b0 msgr2=0x7f1eac069560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.365+0000 7f1eb2863640 1 -- 172.21.15.32:0/3676757509 shutdown_connections 2024-04-16T05:37:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.365+0000 7f1eb2863640 1 -- 172.21.15.32:0/3676757509 wait complete. 2024-04-16T05:37:42.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:37:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.483+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/1873413217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c58102220 msgr2=0x7f9c58102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.483+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/1873413217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c58102220 0x7f9c58102600 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009900 tx=0x7f9c5002bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/1873413217 shutdown_connections 2024-04-16T05:37:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/1873413217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c581072f0 0x7f9c5810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/1873413217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c58102bd0 0x7f9c58106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/1873413217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c58102220 0x7f9c58102600 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/1873413217 >> 172.21.15.32:0/1873413217 conn(0x7f9c580fdac0 msgr2=0x7f9c580ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/1873413217 shutdown_connections 2024-04-16T05:37:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/1873413217 wait complete. 2024-04-16T05:37:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 Processor -- start 2024-04-16T05:37:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.484+0000 7f9c5cb8d640 1 -- start start 2024-04-16T05:37:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c5cb8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c58102220 0x7f9c581a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c5cb8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c581072f0 0x7f9c581a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c5cb8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5819aa90 0x7f9c5819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c5cb8d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581123b0 con 0x7f9c581072f0 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c5cb8d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58112230 con 0x7f9c58102220 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c5cb8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58112530 con 0x7f9c5819aa90 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c581072f0 0x7f9c581a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c581072f0 0x7f9c581a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46432/0 (socket says 172.21.15.32:46432) 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 -- 172.21.15.32:0/2510199515 learned_addr learned my addr 172.21.15.32:0/2510199515 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c5659c640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c58102220 0x7f9c581a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5819aa90 0x7f9c5819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 -- 172.21.15.32:0/2510199515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5819aa90 msgr2=0x7f9c5819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5819aa90 0x7f9c5819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 -- 172.21.15.32:0/2510199515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c58102220 msgr2=0x7f9c581a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c58102220 0x7f9c581a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 -- 172.21.15.32:0/2510199515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5819b5b0 con 0x7f9c581072f0 2024-04-16T05:37:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c581072f0 0x7f9c581a0d30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000d770 tx=0x7f9c4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c4f7fe640 1 -- 172.21.15.32:0/2510199515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40017070 con 0x7f9c581072f0 2024-04-16T05:37:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/2510199515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5819b8a0 con 0x7f9c581072f0 2024-04-16T05:37:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c4f7fe640 1 -- 172.21.15.32:0/2510199515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c40009c30 con 0x7f9c581072f0 2024-04-16T05:37:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.485+0000 7f9c4f7fe640 1 -- 172.21.15.32:0/2510199515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000f620 con 0x7f9c581072f0 2024-04-16T05:37:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.486+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/2510199515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c581a7a00 con 0x7f9c581072f0 2024-04-16T05:37:42.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.488+0000 7f9c4f7fe640 1 -- 172.21.15.32:0/2510199515 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9c4001a070 con 0x7f9c581072f0 2024-04-16T05:37:42.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.488+0000 7f9c5f771740 1 Processor -- start 2024-04-16T05:37:42.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.488+0000 7f9c5f771740 1 -- start start 2024-04-16T05:37:42.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.488+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f5d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.488+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f65e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.488+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f68750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.488+0000 7f9c5f771740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c1b5ec20e0 con 0x55c1b5f66360 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.488+0000 7f9c5f771740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c1b5e5f520 con 0x55c1b5f5d780 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c5f771740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1b5ebe0a0 con 0x55c1b5f5ce60 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f68750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f68750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46446/0 (socket says 172.21.15.32:46446) 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 -- 172.21.15.32:0/3379536109 learned_addr learned my addr 172.21.15.32:0/3379536109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c5659c640 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f65e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c4f7fe640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c440a3d20 0x7f9c440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c440a3d20 0x7f9c440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f5d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c4f7fe640 1 -- 172.21.15.32:0/2510199515 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f9c400d0660 con 0x7f9c581072f0 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 -- 172.21.15.32:0/3379536109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 msgr2=0x55c1b5f5d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f5d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 -- 172.21.15.32:0/3379536109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 msgr2=0x55c1b5f65e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f65e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 -- 172.21.15.32:0/3379536109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1b5e878d0 con 0x55c1b5f66360 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f5d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c5659c640 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.489+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f68750 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9c400cda10 tx=0x7f9c40010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fcd20a7a91d7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c4cff9640 1 -- 172.21.15.32:0/3379536109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40021070 con 0x55c1b5f66360 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c4cff9640 1 -- 172.21.15.32:0/3379536109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c40009de0 con 0x55c1b5f66360 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c4cff9640 1 -- 172.21.15.32:0/3379536109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40098930 con 0x55c1b5f66360 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c440a3d20 0x7f9c440a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800dfb0 tx=0x7f9c4800b2c0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c5f771740 1 -- 172.21.15.32:0/3379536109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 msgr2=0x55c1b5f68750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f68750 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9c400cda10 tx=0x7f9c40010cd0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c5f771740 1 -- 172.21.15.32:0/3379536109 shutdown_connections 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f68750 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3379536109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f5d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.490+0000 7f9c5f771740 1 -- 172.21.15.32:0/3379536109 >> 172.21.15.32:0/3379536109 conn(0x55c1b5f57590 msgr2=0x55c1b5f57970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 -- 172.21.15.32:0/3379536109 shutdown_connections 2024-04-16T05:37:42.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 -- 172.21.15.32:0/3379536109 wait complete. 2024-04-16T05:37:42.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 -- start start 2024-04-16T05:37:42.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 Processor -- start 2024-04-16T05:37:42.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f63410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5d780 0x55c1b5f63950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f63e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c1b5f68fa0 con 0x55c1b5f66360 2024-04-16T05:37:42.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c1b5e7b4d0 con 0x55c1b5f5ce60 2024-04-16T05:37:42.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1b5f69120 con 0x55c1b5f5d780 2024-04-16T05:37:42.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5f771740 10 client.0 _ll_register_callbacks cb 0x55c1b5e602c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:42.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c56d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f63e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c56d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f63e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46456/0 (socket says 172.21.15.32:46456) 2024-04-16T05:37:42.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c56d9d640 1 -- 172.21.15.32:0/3051593113 learned_addr learned my addr 172.21.15.32:0/3051593113 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c5659c640 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f63410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.491+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5d780 0x55c1b5f63950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c56d9d640 1 -- 172.21.15.32:0/3051593113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5d780 msgr2=0x55c1b5f63950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5d780 0x55c1b5f63950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c56d9d640 1 -- 172.21.15.32:0/3051593113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 msgr2=0x55c1b5f63410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f63410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c56d9d640 1 -- 172.21.15.32:0/3051593113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1b5bd43f0 con 0x55c1b5f66360 2024-04-16T05:37:42.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f63e90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9c48011e70 tx=0x7f9c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433c92c7ee9fcfbb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:42.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3051593113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4801c630 con 0x55c1b5f66360 2024-04-16T05:37:42.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3051593113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f9c4801c7d0 con 0x55c1b5f66360 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c5f771740 1 -- 172.21.15.32:0/3051593113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c40009de0 con 0x55c1b5f66360 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3051593113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c4802f050 con 0x55c1b5f66360 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c5f771740 10 client.16305 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c5f771740 1 -- 172.21.15.32:0/3051593113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1b5c9e7d0 con 0x55c1b5f66360 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3051593113 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c40009de0 con 0x55c1b5f66360 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.492+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3051593113 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c48019b60 con 0x55c1b5f66360 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c5f771740 10 client.16305 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c5f771740 10 client.16305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:37:42.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c5f771740 20 client.16305 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:42.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c1b5f762b0 0x55c1b5f78700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c5f771740 4 client.16305 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:42.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c5f771740 1 -- 172.21.15.32:0/3051593113 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1b5ecfb00 con 0x55c1b5f762b0 2024-04-16T05:37:42.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c5659c640 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c1b5f762b0 0x55c1b5f78700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c5659c640 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c1b5f762b0 0x55c1b5f78700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f82b57f7eaa1dec server_cookie=355d8b4ccb53654c in_seq=0 out_seq=0 2024-04-16T05:37:42.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.493+0000 7f9c3dffb640 10 client.16305 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:37:42.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3051593113 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1b5ecfb00 con 0x55c1b5f762b0 2024-04-16T05:37:42.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c3dffb640 10 client.16305 handle_command_reply: tid=0 2024-04-16T05:37:42.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 client.16305 shutdown 2024-04-16T05:37:42.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 -- 172.21.15.32:0/3051593113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 msgr2=0x55c1b5f63e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f63e90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9c48011e70 tx=0x7f9c48004a40 comp rx=0 tx=0).stop 2024-04-16T05:37:42.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 -- 172.21.15.32:0/3051593113 shutdown_connections 2024-04-16T05:37:42.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c1b5f762b0 0x55c1b5f78700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f63e90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5d780 0x55c1b5f63950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3051593113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f63410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c3dffb640 0 client.16305 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:37:42.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 -- 172.21.15.32:0/3051593113 >> 172.21.15.32:0/3051593113 conn(0x55c1b5f57590 msgr2=0x55c1b5f592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 -- 172.21.15.32:0/3051593113 shutdown_connections 2024-04-16T05:37:42.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 1 -- 172.21.15.32:0/3051593113 wait complete. 2024-04-16T05:37:42.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.494+0000 7f9c5f771740 20 client.16305 trim_cache size 0 max 16384 2024-04-16T05:37:42.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.507+0000 7f9c5f771740 1 Processor -- start 2024-04-16T05:37:42.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.507+0000 7f9c5f771740 1 -- start start 2024-04-16T05:37:42.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f71ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f5d780 0x55c1b5f62570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f66360 0x55c1b5f590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c5f771740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581b3e80 con 0x55c1b5f5d780 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c5f771740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581b1750 con 0x55c1b5f5ce60 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c5f771740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581af5a0 con 0x55c1b5f66360 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f5d780 0x55c1b5f62570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f5d780 0x55c1b5f62570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46470/0 (socket says 172.21.15.32:46470) 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 -- 172.21.15.32:0/192615278 learned_addr learned my addr 172.21.15.32:0/192615278 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 -- 172.21.15.32:0/192615278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f66360 msgr2=0x55c1b5f590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/192615278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f66360 0x55c1b5f590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 -- 172.21.15.32:0/192615278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 msgr2=0x55c1b5f71ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/192615278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f71ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 -- 172.21.15.32:0/192615278 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1b5f74e00 con 0x55c1b5f5d780 2024-04-16T05:37:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/192615278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f5d780 0x55c1b5f62570 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9c4009bfc0 tx=0x7f9c4009ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c38630f02fdadef server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c3ffff640 1 -- 172.21.15.32:0/192615278 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40021040 con 0x55c1b5f5d780 2024-04-16T05:37:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c3ffff640 1 -- 172.21.15.32:0/192615278 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c400b5070 con 0x55c1b5f5d780 2024-04-16T05:37:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.508+0000 7f9c3ffff640 1 -- 172.21.15.32:0/192615278 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400d02c0 con 0x55c1b5f5d780 2024-04-16T05:37:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 -- 172.21.15.32:0/192615278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f5d780 msgr2=0x55c1b5f62570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 --2- 172.21.15.32:0/192615278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f5d780 0x55c1b5f62570 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9c4009bfc0 tx=0x7f9c4009ca80 comp rx=0 tx=0).stop 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 -- 172.21.15.32:0/192615278 shutdown_connections 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 --2- 172.21.15.32:0/192615278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f66360 0x55c1b5f590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 --2- 172.21.15.32:0/192615278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f5d780 0x55c1b5f62570 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 --2- 172.21.15.32:0/192615278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f71ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 -- 172.21.15.32:0/192615278 >> 172.21.15.32:0/192615278 conn(0x55c1b5f57590 msgr2=0x55c1b5f589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 -- 172.21.15.32:0/192615278 shutdown_connections 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 -- 172.21.15.32:0/192615278 wait complete. 2024-04-16T05:37:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 -- start start 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 Processor -- start 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f6e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.509+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f69c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5f771740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f6a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5f771740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581afc10 con 0x55c1b5f66360 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5f771740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581b1080 con 0x55c1b5f5d780 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5f771740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581b02e0 con 0x55c1b5f5ce60 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5f771740 10 client.0 _ll_register_callbacks cb 0x7f9c4800d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:42.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f6a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f6a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46480/0 (socket says 172.21.15.32:46480) 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 -- 172.21.15.32:0/3473475114 learned_addr learned my addr 172.21.15.32:0/3473475114 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5659c640 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f6e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c55d9b640 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f69c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 -- 172.21.15.32:0/3473475114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 msgr2=0x55c1b5f6e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f6e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 -- 172.21.15.32:0/3473475114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 msgr2=0x55c1b5f69c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f69c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 -- 172.21.15.32:0/3473475114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1b5fdb840 con 0x55c1b5f66360 2024-04-16T05:37:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c56d9d640 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f6a3d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9c48024440 tx=0x7f9c480244c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4273ba4c6dfc4be server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3473475114 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48037020 con 0x55c1b5f66360 2024-04-16T05:37:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3473475114 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f9c48004c40 con 0x55c1b5f66360 2024-04-16T05:37:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5f771740 1 -- 172.21.15.32:0/3473475114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c400b5070 con 0x55c1b5f66360 2024-04-16T05:37:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3473475114 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c48042050 con 0x55c1b5f66360 2024-04-16T05:37:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5f771740 10 client.16311 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:42.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.510+0000 7f9c5f771740 1 -- 172.21.15.32:0/3473475114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1b5fdc9a0 con 0x55c1b5f66360 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3473475114 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c400b5070 con 0x55c1b5f66360 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3473475114 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c4801c630 con 0x55c1b5f66360 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c5f771740 10 client.16311 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c5f771740 10 client.16311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c5f771740 20 client.16311 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c1b5fdd0e0 0x55c1b5fdf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c5f771740 4 client.16311 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c5f771740 1 -- 172.21.15.32:0/3473475114 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1b5ec9970 con 0x55c1b5fdd0e0 2024-04-16T05:37:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c5659c640 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c1b5fdd0e0 0x55c1b5fdf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c3dffb640 10 client.16311 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:37:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.511+0000 7f9c5659c640 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c1b5fdd0e0 0x55c1b5fdf500 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ca0c93fad765bee server_cookie=12f83965811fdcf3 in_seq=0 out_seq=0 2024-04-16T05:37:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c3dffb640 1 -- 172.21.15.32:0/3473475114 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1b5ec9970 con 0x55c1b5fdd0e0 2024-04-16T05:37:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c3dffb640 10 client.16311 handle_command_reply: tid=0 2024-04-16T05:37:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 client.16311 shutdown 2024-04-16T05:37:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 -- 172.21.15.32:0/3473475114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 msgr2=0x55c1b5f6a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f6a3d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9c48024440 tx=0x7f9c480244c0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 -- 172.21.15.32:0/3473475114 shutdown_connections 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55c1b5fdd0e0 0x55c1b5fdf500 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1b5f66360 0x55c1b5f6a3d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1b5f5d780 0x55c1b5f69c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 --2- 172.21.15.32:0/3473475114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1b5f5ce60 0x55c1b5f6e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c3dffb640 0 client.16311 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 -- 172.21.15.32:0/3473475114 >> 172.21.15.32:0/3473475114 conn(0x55c1b5f57590 msgr2=0x55c1b5f589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 -- 172.21.15.32:0/3473475114 shutdown_connections 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 1 -- 172.21.15.32:0/3473475114 wait complete. 2024-04-16T05:37:42.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5f771740 20 client.16311 trim_cache size 0 max 16384 2024-04-16T05:37:42.522 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:37:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c440a3d20 msgr2=0x7f9c440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c440a3d20 0x7f9c440a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800dfb0 tx=0x7f9c4800b2c0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c581072f0 msgr2=0x7f9c581a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c581072f0 0x7f9c581a0d30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000d770 tx=0x7f9c4000dc30 comp rx=0 tx=0).stop 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/2510199515 shutdown_connections 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5819aa90 0x7f9c5819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c440a3d20 0x7f9c440a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c581072f0 0x7f9c581a0d30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 --2- 172.21.15.32:0/2510199515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c58102220 0x7f9c581a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/2510199515 >> 172.21.15.32:0/2510199515 conn(0x7f9c580fdac0 msgr2=0x7f9c580ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/2510199515 shutdown_connections 2024-04-16T05:37:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.512+0000 7f9c5cb8d640 1 -- 172.21.15.32:0/2510199515 wait complete. 2024-04-16T05:37:42.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:37:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 -- 172.21.15.32:0/2112230244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c06a950 msgr2=0x7f899c06ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 --2- 172.21.15.32:0/2112230244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c06a950 0x7f899c06ad30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f898c009900 tx=0x7f898c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:37:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 -- 172.21.15.32:0/2112230244 shutdown_connections 2024-04-16T05:37:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 --2- 172.21.15.32:0/2112230244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f899c1073c0 0x7f899c10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 --2- 172.21.15.32:0/2112230244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899c06b300 0x7f899c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 --2- 172.21.15.32:0/2112230244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c06a950 0x7f899c06ad30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 -- 172.21.15.32:0/2112230244 >> 172.21.15.32:0/2112230244 conn(0x7f899c077cf0 msgr2=0x7f899c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 -- 172.21.15.32:0/2112230244 shutdown_connections 2024-04-16T05:37:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.633+0000 7f89a1b29640 1 -- 172.21.15.32:0/2112230244 wait complete. 2024-04-16T05:37:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.634+0000 7f89a1b29640 1 Processor -- start 2024-04-16T05:37:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.634+0000 7f89a1b29640 1 -- start start 2024-04-16T05:37:42.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.634+0000 7f89a1b29640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f899c06a950 0x7f899c101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.634+0000 7f89a1b29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c1073c0 0x7f899c101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.634+0000 7f89a1b29640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899c101ca0 0x7f899c1a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.634+0000 7f89a1b29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1123e0 con 0x7f899c1073c0 2024-04-16T05:37:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.634+0000 7f89a1b29640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f899c112260 con 0x7f899c06a950 2024-04-16T05:37:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.634+0000 7f89a1b29640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f899c112560 con 0x7f899c101ca0 2024-04-16T05:37:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c1073c0 0x7f899c101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c1073c0 0x7f899c101760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46510/0 (socket says 172.21.15.32:46510) 2024-04-16T05:37:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 -- 172.21.15.32:0/3771099636 learned_addr learned my addr 172.21.15.32:0/3771099636 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899b7fe640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f899c06a950 0x7f899c101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899bfff640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899c101ca0 0x7f899c1a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 -- 172.21.15.32:0/3771099636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899c101ca0 msgr2=0x7f899c1a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899c101ca0 0x7f899c1a6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 -- 172.21.15.32:0/3771099636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f899c06a950 msgr2=0x7f899c101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f899c06a950 0x7f899c101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 -- 172.21.15.32:0/3771099636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c1a7150 con 0x7f899c1073c0 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f899affd640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c1073c0 0x7f899c101760 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8990005fd0 tx=0x7f899000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f8998ff9640 1 -- 172.21.15.32:0/3771099636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990017070 con 0x7f899c1073c0 2024-04-16T05:37:42.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f89a1b29640 1 -- 172.21.15.32:0/3771099636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c1a7440 con 0x7f899c1073c0 2024-04-16T05:37:42.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f8998ff9640 1 -- 172.21.15.32:0/3771099636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8990009c30 con 0x7f899c1073c0 2024-04-16T05:37:42.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f89a1b29640 1 -- 172.21.15.32:0/3771099636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c1a79e0 con 0x7f899c1073c0 2024-04-16T05:37:42.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.635+0000 7f8998ff9640 1 -- 172.21.15.32:0/3771099636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990020060 con 0x7f899c1073c0 2024-04-16T05:37:42.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.637+0000 7f8998ff9640 1 -- 172.21.15.32:0/3771099636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8990003d10 con 0x7f899c1073c0 2024-04-16T05:37:42.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f89a47ea740 1 Processor -- start 2024-04-16T05:37:42.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f89a47ea740 1 -- start start 2024-04-16T05:37:42.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1626240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb162ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1631750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f89a47ea740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ccb158b0e0 con 0x55ccb1625e60 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f89a47ea740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ccb1528520 con 0x55ccb162f360 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f89a47ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ccb15870a0 con 0x55ccb1626780 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1626240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f8998ff9640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f89840a3d20 0x7f89840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1626240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46520/0 (socket says 172.21.15.32:46520) 2024-04-16T05:37:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 -- 172.21.15.32:0/525054922 learned_addr learned my addr 172.21.15.32:0/525054922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899b7fe640 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb162ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f89840a3d20 0x7f89840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f8998ff9640 1 -- 172.21.15.32:0/3771099636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f89900d0420 con 0x7f899c1073c0 2024-04-16T05:37:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899affd640 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1631750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 -- 172.21.15.32:0/525054922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 msgr2=0x55ccb162ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb162ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 -- 172.21.15.32:0/525054922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 msgr2=0x55ccb1631750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1631750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899bfff640 1 -- 172.21.15.32:0/525054922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb15508d0 con 0x55ccb1625e60 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899b7fe640 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb162ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.638+0000 7f899affd640 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1631750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.639+0000 7f899bfff640 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1626240 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8994014a60 tx=0x7f8994014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892a7bb8e399c6b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.639+0000 7f8981ffb640 1 -- 172.21.15.32:0/525054922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899401ec50 con 0x55ccb1625e60 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.639+0000 7f8981ffb640 1 -- 172.21.15.32:0/525054922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8994023070 con 0x55ccb1625e60 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.639+0000 7f8981ffb640 1 -- 172.21.15.32:0/525054922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994005650 con 0x55ccb1625e60 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.639+0000 7f89a47ea740 1 -- 172.21.15.32:0/525054922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 msgr2=0x55ccb1626240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.639+0000 7f899bfff640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f89840a3d20 0x7f89840a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8994022000 tx=0x7f899401e000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.639+0000 7f89a47ea740 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1626240 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8994014a60 tx=0x7f8994014f20 comp rx=0 tx=0).stop 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 -- 172.21.15.32:0/525054922 shutdown_connections 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1631750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb162ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 --2- 172.21.15.32:0/525054922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1626240 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 -- 172.21.15.32:0/525054922 >> 172.21.15.32:0/525054922 conn(0x55ccb1620590 msgr2=0x55ccb1620970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 -- 172.21.15.32:0/525054922 shutdown_connections 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 -- 172.21.15.32:0/525054922 wait complete. 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 -- start start 2024-04-16T05:37:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 Processor -- start 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb162c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1626780 0x55ccb162c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb162f360 0x55ccb162ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ccb1632000 con 0x55ccb1625e60 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ccb15444d0 con 0x55ccb1626780 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ccb1632180 con 0x55ccb162f360 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f89a47ea740 10 client.0 _ll_register_callbacks cb 0x55ccb15292c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.640+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb162c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb162c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46526/0 (socket says 172.21.15.32:46526) 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899b7fe640 1 -- 172.21.15.32:0/3918858447 learned_addr learned my addr 172.21.15.32:0/3918858447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899affd640 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1626780 0x55ccb162c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899bfff640 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb162f360 0x55ccb162ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899b7fe640 1 -- 172.21.15.32:0/3918858447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb162f360 msgr2=0x55ccb162ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899b7fe640 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb162f360 0x55ccb162ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899b7fe640 1 -- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1626780 msgr2=0x55ccb162c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899b7fe640 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1626780 0x55ccb162c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899b7fe640 1 -- 172.21.15.32:0/3918858447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb129d3f0 con 0x55ccb1625e60 2024-04-16T05:37:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899affd640 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1626780 0x55ccb162c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f899b7fe640 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb162c3c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f898c02c170 tx=0x7f898c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3268be1fab91ca9c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f897affd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f897affd640 1 -- 172.21.15.32:0/3918858447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0043b0 con 0x55ccb1625e60 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f897affd640 1 -- 172.21.15.32:0/3918858447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f898c0046c0 con 0x55ccb1625e60 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f89a47ea740 1 -- 172.21.15.32:0/3918858447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8994023070 con 0x55ccb1625e60 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f897affd640 1 -- 172.21.15.32:0/3918858447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f898c0048c0 con 0x55ccb1625e60 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f89a47ea740 10 client.16323 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f89a47ea740 1 -- 172.21.15.32:0/3918858447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ccb13677d0 con 0x55ccb1625e60 2024-04-16T05:37:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.641+0000 7f897affd640 1 -- 172.21.15.32:0/3918858447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8994023070 con 0x55ccb1625e60 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f897affd640 1 -- 172.21.15.32:0/3918858447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f898c0037b0 con 0x55ccb1625e60 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f89a47ea740 10 client.16323 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f89a47ea740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f89a47ea740 20 client.16323 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f89a47ea740 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ccb163f370 0x55ccb1641800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f89a47ea740 4 client.16323 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f89a47ea740 1 -- 172.21.15.32:0/3918858447 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccb1598b00 con 0x55ccb163f370 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f899affd640 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ccb163f370 0x55ccb1641800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f899affd640 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ccb163f370 0x55ccb1641800 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f689ba23bf5976 server_cookie=e2190a18ab6b2ea3 in_seq=0 out_seq=0 2024-04-16T05:37:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.642+0000 7f897affd640 10 client.16323 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f897affd640 1 -- 172.21.15.32:0/3918858447 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccb1598b00 con 0x55ccb163f370 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f897affd640 10 client.16323 handle_command_reply: tid=0 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 client.16323 shutdown 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 -- 172.21.15.32:0/3918858447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 msgr2=0x55ccb162c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb162c3c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f898c02c170 tx=0x7f898c004040 comp rx=0 tx=0).stop 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 -- 172.21.15.32:0/3918858447 shutdown_connections 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ccb163f370 0x55ccb1641800 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb162f360 0x55ccb162ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1626780 0x55ccb162c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 --2- 172.21.15.32:0/3918858447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb162c3c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f897affd640 0 client.16323 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 -- 172.21.15.32:0/3918858447 >> 172.21.15.32:0/3918858447 conn(0x55ccb1620590 msgr2=0x55ccb16222e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 -- 172.21.15.32:0/3918858447 shutdown_connections 2024-04-16T05:37:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 1 -- 172.21.15.32:0/3918858447 wait complete. 2024-04-16T05:37:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.643+0000 7f89a47ea740 20 client.16323 trim_cache size 0 max 16384 2024-04-16T05:37:42.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f89a47ea740 1 Processor -- start 2024-04-16T05:37:42.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f89a47ea740 1 -- start start 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1625e60 0x55ccb162eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb163b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb162f360 0x55ccb161f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f89a47ea740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1b3e60 con 0x55ccb162f360 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f89a47ea740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1b1730 con 0x55ccb1625e60 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f89a47ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1af580 con 0x55ccb1626780 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb162f360 0x55ccb161f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb162f360 0x55ccb161f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46536/0 (socket says 172.21.15.32:46536) 2024-04-16T05:37:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 -- 172.21.15.32:0/1044675254 learned_addr learned my addr 172.21.15.32:0/1044675254 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899b7fe640 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1625e60 0x55ccb162eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899affd640 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb163b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 -- 172.21.15.32:0/1044675254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 msgr2=0x55ccb163b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb163b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 -- 172.21.15.32:0/1044675254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1625e60 msgr2=0x55ccb162eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1625e60 0x55ccb162eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 -- 172.21.15.32:0/1044675254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb16265b0 con 0x55ccb162f360 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899b7fe640 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1625e60 0x55ccb162eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.657+0000 7f899bfff640 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb162f360 0x55ccb161f5b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8994005b20 tx=0x7f8994005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7902d3c26d2f8218 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89817fa640 1 -- 172.21.15.32:0/1044675254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994037070 con 0x55ccb162f360 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89817fa640 1 -- 172.21.15.32:0/1044675254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8994023040 con 0x55ccb162f360 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89817fa640 1 -- 172.21.15.32:0/1044675254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89940144f0 con 0x55ccb162f360 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 -- 172.21.15.32:0/1044675254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb162f360 msgr2=0x55ccb161f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb162f360 0x55ccb161f5b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8994005b20 tx=0x7f8994005b50 comp rx=0 tx=0).stop 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 -- 172.21.15.32:0/1044675254 shutdown_connections 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb162f360 0x55ccb161f5b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb163b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 --2- 172.21.15.32:0/1044675254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb1625e60 0x55ccb162eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 -- 172.21.15.32:0/1044675254 >> 172.21.15.32:0/1044675254 conn(0x55ccb1620590 msgr2=0x55ccb161eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 -- 172.21.15.32:0/1044675254 shutdown_connections 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 -- 172.21.15.32:0/1044675254 wait complete. 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.658+0000 7f89a47ea740 1 -- start start 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f89a47ea740 1 Processor -- start 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1637aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb1632e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f89a47ea740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1633550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f89a47ea740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1afbf0 con 0x55ccb1625e60 2024-04-16T05:37:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f89a47ea740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1b1060 con 0x55ccb162f360 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f89a47ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1b02c0 con 0x55ccb1626780 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f89a47ea740 10 client.0 _ll_register_callbacks cb 0x7f898c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1637aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1637aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46546/0 (socket says 172.21.15.32:46546) 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 -- 172.21.15.32:0/2908157238 learned_addr learned my addr 172.21.15.32:0/2908157238 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899bfff640 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1633550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899affd640 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb1632e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 -- 172.21.15.32:0/2908157238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 msgr2=0x55ccb1632e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb1632e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 -- 172.21.15.32:0/2908157238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 msgr2=0x55ccb1633550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1633550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 -- 172.21.15.32:0/2908157238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb16a4940 con 0x55ccb1625e60 2024-04-16T05:37:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f899b7fe640 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1637aa0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f898c003b80 tx=0x7f898c003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a188d308358ccb32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.659+0000 7f897affd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f897affd640 1 -- 172.21.15.32:0/2908157238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0042b0 con 0x55ccb1625e60 2024-04-16T05:37:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f897affd640 1 -- 172.21.15.32:0/2908157238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f898c004450 con 0x55ccb1625e60 2024-04-16T05:37:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 1 -- 172.21.15.32:0/2908157238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8994023040 con 0x55ccb1625e60 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f897affd640 1 -- 172.21.15.32:0/2908157238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f898c05e050 con 0x55ccb1625e60 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 10 client.16329 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 1 -- 172.21.15.32:0/2908157238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ccb16a5b00 con 0x55ccb1625e60 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f897affd640 1 -- 172.21.15.32:0/2908157238 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8994023040 con 0x55ccb1625e60 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f897affd640 1 -- 172.21.15.32:0/2908157238 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f898c055580 con 0x55ccb1625e60 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 10 client.16329 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 20 client.16329 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ccb16a6280 0x55ccb16a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 4 client.16329 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f89a47ea740 1 -- 172.21.15.32:0/2908157238 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccb1592970 con 0x55ccb16a6280 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.660+0000 7f899affd640 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ccb16a6280 0x55ccb16a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f897affd640 10 client.16329 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f899affd640 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ccb16a6280 0x55ccb16a86a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab990424e75cdee7 server_cookie=9725b4537c2f4138 in_seq=0 out_seq=0 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f897affd640 1 -- 172.21.15.32:0/2908157238 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccb1592970 con 0x55ccb16a6280 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f897affd640 10 client.16329 handle_command_reply: tid=0 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 client.16329 shutdown 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 -- 172.21.15.32:0/2908157238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 msgr2=0x55ccb1637aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1637aa0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f898c003b80 tx=0x7f898c003fa0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 -- 172.21.15.32:0/2908157238 shutdown_connections 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ccb162f360 0x55ccb1633550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ccb16a6280 0x55ccb16a86a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb1626780 0x55ccb1632e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 --2- 172.21.15.32:0/2908157238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccb1625e60 0x55ccb1637aa0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f897affd640 0 client.16329 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 -- 172.21.15.32:0/2908157238 >> 172.21.15.32:0/2908157238 conn(0x55ccb1620590 msgr2=0x55ccb161f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 -- 172.21.15.32:0/2908157238 shutdown_connections 2024-04-16T05:37:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 1 -- 172.21.15.32:0/2908157238 wait complete. 2024-04-16T05:37:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.661+0000 7f89a47ea740 20 client.16329 trim_cache size 0 max 16384 2024-04-16T05:37:42.670 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:37:42.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 -- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f89840a3d20 msgr2=0x7f89840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f89840a3d20 0x7f89840a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8994022000 tx=0x7f899401e000 comp rx=0 tx=0).stop 2024-04-16T05:37:42.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 -- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c1073c0 msgr2=0x7f899c101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:42.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c1073c0 0x7f899c101760 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8990005fd0 tx=0x7f899000dde0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 -- 172.21.15.32:0/3771099636 shutdown_connections 2024-04-16T05:37:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899c101ca0 0x7f899c1a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f89840a3d20 0x7f89840a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c1073c0 0x7f899c101760 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 --2- 172.21.15.32:0/3771099636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f899c06a950 0x7f899c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 -- 172.21.15.32:0/3771099636 >> 172.21.15.32:0/3771099636 conn(0x7f899c077cf0 msgr2=0x7f899c0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 -- 172.21.15.32:0/3771099636 shutdown_connections 2024-04-16T05:37:42.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:42.662+0000 7f89a1b29640 1 -- 172.21.15.32:0/3771099636 wait complete. 2024-04-16T05:37:43.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:43 smithi032 ceph-mon[32351]: pgmap v86: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:37:43.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:43 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1910016109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:43.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:43 smithi134 ceph-mon[36153]: pgmap v86: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:37:43.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:43 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1910016109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:43.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:43 smithi082 ceph-mon[36195]: pgmap v86: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:37:43.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:43 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1910016109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:37:43.673 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:37:43.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:37:43.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:37:43.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 -- 172.21.15.32:0/441316656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 msgr2=0x7faafc10e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 --2- 172.21.15.32:0/441316656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 0x7faafc10e000 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009ca0 tx=0x7faaf402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 -- 172.21.15.32:0/441316656 shutdown_connections 2024-04-16T05:37:43.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 --2- 172.21.15.32:0/441316656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 0x7faafc10e000 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 --2- 172.21.15.32:0/441316656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faafc102230 0x7faafc105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 --2- 172.21.15.32:0/441316656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faafc101880 0x7faafc101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 -- 172.21.15.32:0/441316656 >> 172.21.15.32:0/441316656 conn(0x7faafc077490 msgr2=0x7faafc0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 -- 172.21.15.32:0/441316656 shutdown_connections 2024-04-16T05:37:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.797+0000 7fab023ba640 1 -- 172.21.15.32:0/441316656 wait complete. 2024-04-16T05:37:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab023ba640 1 Processor -- start 2024-04-16T05:37:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab023ba640 1 -- start start 2024-04-16T05:37:43.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab023ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faafc101880 0x7faafc0fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab023ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 0x7faafc0fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab023ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faafc0fd350 0x7faafc1a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab023ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1114f0 con 0x7faafc106430 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab023ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faafc111370 con 0x7faafc0fd350 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab023ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faafc111670 con 0x7faafc101880 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 0x7faafc0fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 0x7faafc0fce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46578/0 (socket says 172.21.15.32:46578) 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7faaf3fff640 1 -- 172.21.15.32:0/2676766960 learned_addr learned my addr 172.21.15.32:0/2676766960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7faafbfff640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faafc101880 0x7faafc0fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7fab00957640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faafc0fd350 0x7faafc1a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7faaf3fff640 1 -- 172.21.15.32:0/2676766960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faafc101880 msgr2=0x7faafc0fc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7faaf3fff640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faafc101880 0x7faafc0fc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7faaf3fff640 1 -- 172.21.15.32:0/2676766960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faafc0fd350 msgr2=0x7faafc1a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.798+0000 7faaf3fff640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faafc0fd350 0x7faafc1a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.799+0000 7faaf3fff640 1 -- 172.21.15.32:0/2676766960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc1a73b0 con 0x7faafc106430 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.799+0000 7faafbfff640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faafc101880 0x7faafc0fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.799+0000 7faaf3fff640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 0x7faafc0fce10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faae4005fd0 tx=0x7faae400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.799+0000 7faaf9ffb640 1 -- 172.21.15.32:0/2676766960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4017070 con 0x7faafc106430 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.799+0000 7faaf9ffb640 1 -- 172.21.15.32:0/2676766960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae4009c30 con 0x7faafc106430 2024-04-16T05:37:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.799+0000 7fab023ba640 1 -- 172.21.15.32:0/2676766960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc1a76a0 con 0x7faafc106430 2024-04-16T05:37:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.799+0000 7faaf9ffb640 1 -- 172.21.15.32:0/2676766960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4010460 con 0x7faafc106430 2024-04-16T05:37:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.799+0000 7fab023ba640 1 -- 172.21.15.32:0/2676766960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc1a7bf0 con 0x7faafc106430 2024-04-16T05:37:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.801+0000 7faaf9ffb640 1 -- 172.21.15.32:0/2676766960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faae4003e70 con 0x7faafc106430 2024-04-16T05:37:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.801+0000 7fab04f9e740 1 Processor -- start 2024-04-16T05:37:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.801+0000 7fab04f9e740 1 -- start start 2024-04-16T05:37:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b33d0 0x55dcc29bba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab04f9e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dcc2917e90 con 0x55dcc29b2ab0 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab04f9e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dcc28b54c0 con 0x55dcc29bbfb0 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab04f9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcc2913e50 con 0x55dcc29b33d0 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7faaf9ffb640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faae80a3d50 0x7faae80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46588/0 (socket says 172.21.15.32:46588) 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 -- 172.21.15.32:0/2265557470 learned_addr learned my addr 172.21.15.32:0/2265557470 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7faaf3fff640 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7faafbfff640 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b33d0 0x55dcc29bba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faae80a3d50 0x7faae80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 -- 172.21.15.32:0/2265557470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b33d0 msgr2=0x55dcc29bba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b33d0 0x55dcc29bba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 -- 172.21.15.32:0/2265557470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 msgr2=0x55dcc29be3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7faaf9ffb640 1 -- 172.21.15.32:0/2676766960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7faae40d07d0 con 0x7faafc106430 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29be3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7fab00957640 1 -- 172.21.15.32:0/2265557470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc26f27e0 con 0x55dcc29b2ab0 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.802+0000 7faafbfff640 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b33d0 0x55dcc29bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.803+0000 7fab00957640 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b2e90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7faaf4004af0 tx=0x7faaf402ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289bb4c46a837b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.803+0000 7fab00957640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faae80a3d50 0x7faae80a6210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faaf4005e30 tx=0x7faaf4048000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.803+0000 7faaf27fc640 1 -- 172.21.15.32:0/2265557470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4039040 con 0x55dcc29b2ab0 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.803+0000 7faaf27fc640 1 -- 172.21.15.32:0/2265557470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf4034920 con 0x55dcc29b2ab0 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.803+0000 7faaf27fc640 1 -- 172.21.15.32:0/2265557470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4043660 con 0x55dcc29b2ab0 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.803+0000 7fab04f9e740 1 -- 172.21.15.32:0/2265557470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b2ab0 msgr2=0x55dcc29b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.803+0000 7fab04f9e740 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b2e90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7faaf4004af0 tx=0x7faaf402ceb0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.803+0000 7fab04f9e740 1 -- 172.21.15.32:0/2265557470 shutdown_connections 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29be3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b33d0 0x55dcc29bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 --2- 172.21.15.32:0/2265557470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b2e90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 -- 172.21.15.32:0/2265557470 >> 172.21.15.32:0/2265557470 conn(0x55dcc29ad1e0 msgr2=0x55dcc29ad5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 -- 172.21.15.32:0/2265557470 shutdown_connections 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 -- 172.21.15.32:0/2265557470 wait complete. 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 -- start start 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 Processor -- start 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29bbfb0 0x55dcc29b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dcc29bed40 con 0x55dcc29b33d0 2024-04-16T05:37:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dcc28a0410 con 0x55dcc29b2ab0 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcc29beec0 con 0x55dcc29bbfb0 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.804+0000 7fab04f9e740 10 client.0 _ll_register_callbacks cb 0x55dcc289bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46598/0 (socket says 172.21.15.32:46598) 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 -- 172.21.15.32:0/1630059878 learned_addr learned my addr 172.21.15.32:0/1630059878 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7fab00957640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29bbfb0 0x55dcc29b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faafbfff640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 -- 172.21.15.32:0/1630059878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29bbfb0 msgr2=0x55dcc29b9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29bbfb0 0x55dcc29b9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 -- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29b2ab0 msgr2=0x55dcc29b9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 -- 172.21.15.32:0/1630059878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc2752f70 con 0x55dcc29b33d0 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7fab00957640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29bbfb0 0x55dcc29b9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faafbfff640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faaf3fff640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b9570 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7faae4098890 tx=0x7faae409be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1786ac68ba017d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faadb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faadb7fe640 1 -- 172.21.15.32:0/1630059878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40a07e0 con 0x55dcc29b33d0 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faadb7fe640 1 -- 172.21.15.32:0/1630059878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7faae40d0470 con 0x55dcc29b33d0 2024-04-16T05:37:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7fab04f9e740 1 -- 172.21.15.32:0/1630059878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf4034920 con 0x55dcc29b33d0 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7faadb7fe640 1 -- 172.21.15.32:0/1630059878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faae40a50a0 con 0x55dcc29b33d0 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7fab04f9e740 10 client.16341 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.805+0000 7fab04f9e740 1 -- 172.21.15.32:0/1630059878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dcc25ea170 con 0x55dcc29b33d0 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7faadb7fe640 1 -- 172.21.15.32:0/1630059878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf4034920 con 0x55dcc29b33d0 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7faadb7fe640 1 -- 172.21.15.32:0/1630059878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faae40a4c10 con 0x55dcc29b33d0 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7fab04f9e740 10 client.16341 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7fab04f9e740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7fab04f9e740 20 client.16341 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7fab04f9e740 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dcc29cc410 0x55dcc29ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7fab04f9e740 4 client.16341 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7fab04f9e740 1 -- 172.21.15.32:0/1630059878 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcc29258b0 con 0x55dcc29cc410 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7faafbfff640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dcc29cc410 0x55dcc29ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7faadb7fe640 10 client.16341 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.806+0000 7faafbfff640 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dcc29cc410 0x55dcc29ce840 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c3ab4b933888ff server_cookie=b0867cfe58d91f54 in_seq=0 out_seq=0 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.807+0000 7faadb7fe640 1 -- 172.21.15.32:0/1630059878 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcc29258b0 con 0x55dcc29cc410 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.807+0000 7faadb7fe640 10 client.16341 handle_command_reply: tid=0 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.807+0000 7fab04f9e740 1 client.16341 shutdown 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 -- 172.21.15.32:0/1630059878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 msgr2=0x55dcc29b9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b9570 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7faae4098890 tx=0x7faae409be50 comp rx=0 tx=0).stop 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 -- 172.21.15.32:0/1630059878 shutdown_connections 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dcc29cc410 0x55dcc29ce840 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29bbfb0 0x55dcc29b9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b9570 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 --2- 172.21.15.32:0/1630059878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29b2ab0 0x55dcc29b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7faadb7fe640 0 client.16341 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 -- 172.21.15.32:0/1630059878 >> 172.21.15.32:0/1630059878 conn(0x55dcc29ad1e0 msgr2=0x55dcc29aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 -- 172.21.15.32:0/1630059878 shutdown_connections 2024-04-16T05:37:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 1 -- 172.21.15.32:0/1630059878 wait complete. 2024-04-16T05:37:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.808+0000 7fab04f9e740 20 client.16341 trim_cache size 0 max 16384 2024-04-16T05:37:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab04f9e740 1 Processor -- start 2024-04-16T05:37:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab04f9e740 1 -- start start 2024-04-16T05:37:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 0x55dcc29c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29d4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab04f9e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b3da0 con 0x55dcc29b33d0 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab04f9e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b1670 con 0x55dcc29bbfb0 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab04f9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1af4c0 con 0x55dcc29b2ab0 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46608/0 (socket says 172.21.15.32:46608) 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7faaf3fff640 1 -- 172.21.15.32:0/3870143303 learned_addr learned my addr 172.21.15.32:0/3870143303 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7fab00957640 1 --2- 172.21.15.32:0/3870143303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29d4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7faaf3fff640 1 -- 172.21.15.32:0/3870143303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 msgr2=0x55dcc29c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7faaf3fff640 1 --2- 172.21.15.32:0/3870143303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 0x55dcc29c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7faaf3fff640 1 -- 172.21.15.32:0/3870143303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 msgr2=0x55dcc29d4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7faaf3fff640 1 --2- 172.21.15.32:0/3870143303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29d4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.826+0000 7faaf3fff640 1 -- 172.21.15.32:0/3870143303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc29b75a0 con 0x55dcc29b33d0 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7faaf3fff640 1 --2- 172.21.15.32:0/3870143303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b98d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7faae40038c0 tx=0x7faae409c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a65beee321fa65c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7faaf1ffb640 1 -- 172.21.15.32:0/3870143303 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40a07e0 con 0x55dcc29b33d0 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7faaf1ffb640 1 -- 172.21.15.32:0/3870143303 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae4015020 con 0x55dcc29b33d0 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7faaf1ffb640 1 -- 172.21.15.32:0/3870143303 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40d0470 con 0x55dcc29b33d0 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 -- 172.21.15.32:0/3870143303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 msgr2=0x55dcc29b98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 --2- 172.21.15.32:0/3870143303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b98d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7faae40038c0 tx=0x7faae409c5b0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 -- 172.21.15.32:0/3870143303 shutdown_connections 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 --2- 172.21.15.32:0/3870143303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29d4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 --2- 172.21.15.32:0/3870143303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29b98d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 --2- 172.21.15.32:0/3870143303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 0x55dcc29c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 -- 172.21.15.32:0/3870143303 >> 172.21.15.32:0/3870143303 conn(0x55dcc29ad1e0 msgr2=0x55dcc29b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 -- 172.21.15.32:0/3870143303 shutdown_connections 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.827+0000 7fab04f9e740 1 -- 172.21.15.32:0/3870143303 wait complete. 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 1 -- start start 2024-04-16T05:37:43.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 1 Processor -- start 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 0x55dcc29c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29c0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1afb30 con 0x55dcc29b33d0 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b0fa0 con 0x55dcc29bbfb0 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b0200 con 0x55dcc29b2ab0 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab04f9e740 10 client.0 _ll_register_callbacks cb 0x7faae4003810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29c0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46624/0 (socket says 172.21.15.32:46624) 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faaf3fff640 1 -- 172.21.15.32:0/427665150 learned_addr learned my addr 172.21.15.32:0/427665150 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faafbfff640 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 0x55dcc29c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faaf3fff640 1 -- 172.21.15.32:0/427665150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 msgr2=0x55dcc29c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faaf3fff640 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 0x55dcc29c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faaf3fff640 1 -- 172.21.15.32:0/427665150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 msgr2=0x55dcc29c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7fab00957640 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faaf3fff640 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.828+0000 7faaf3fff640 1 -- 172.21.15.32:0/427665150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc29c13c0 con 0x55dcc29b33d0 2024-04-16T05:37:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7faaf3fff640 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29c0790 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7faae409c3c0 tx=0x7faae40980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eea5db73dbf13c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7faadb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7faadb7fe640 1 -- 172.21.15.32:0/427665150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40bb2f0 con 0x55dcc29b33d0 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 1 -- 172.21.15.32:0/427665150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae4015020 con 0x55dcc29b33d0 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7faadb7fe640 1 -- 172.21.15.32:0/427665150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7faae40b5bb0 con 0x55dcc29b33d0 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7faadb7fe640 1 -- 172.21.15.32:0/427665150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faae40a07e0 con 0x55dcc29b33d0 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 10 client.16347 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 1 -- 172.21.15.32:0/427665150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dcc2a338d0 con 0x55dcc29b33d0 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7faadb7fe640 1 -- 172.21.15.32:0/427665150 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae4015020 con 0x55dcc29b33d0 2024-04-16T05:37:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7faadb7fe640 1 -- 172.21.15.32:0/427665150 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faae4098180 con 0x55dcc29b33d0 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 10 client.16347 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 20 client.16347 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dcc2a33da0 0x55dcc2a36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 4 client.16347 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.829+0000 7fab04f9e740 1 -- 172.21.15.32:0/427665150 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dcc291f720 con 0x55dcc2a33da0 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.830+0000 7faafbfff640 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dcc2a33da0 0x55dcc2a36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.830+0000 7faadb7fe640 10 client.16347 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.830+0000 7faafbfff640 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dcc2a33da0 0x55dcc2a36190 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d62e561b9f7862a2 server_cookie=9029518350aa39ee in_seq=0 out_seq=0 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7faadb7fe640 1 -- 172.21.15.32:0/427665150 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcc291f720 con 0x55dcc2a33da0 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7faadb7fe640 10 client.16347 handle_command_reply: tid=0 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 client.16347 shutdown 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 -- 172.21.15.32:0/427665150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 msgr2=0x55dcc29c0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29c0790 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7faae409c3c0 tx=0x7faae40980d0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 -- 172.21.15.32:0/427665150 shutdown_connections 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dcc29bbfb0 0x55dcc29c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcc29b33d0 0x55dcc29c0790 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dcc2a33da0 0x55dcc2a36190 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 --2- 172.21.15.32:0/427665150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcc29b2ab0 0x55dcc29c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7faadb7fe640 0 client.16347 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 -- 172.21.15.32:0/427665150 >> 172.21.15.32:0/427665150 conn(0x55dcc29ad1e0 msgr2=0x55dcc29b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 -- 172.21.15.32:0/427665150 shutdown_connections 2024-04-16T05:37:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 1 -- 172.21.15.32:0/427665150 wait complete. 2024-04-16T05:37:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.831+0000 7fab04f9e740 20 client.16347 trim_cache size 0 max 16384 2024-04-16T05:37:43.838 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:37:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:37:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "eb775d57-aff1-47cc-9b09-9eb499166532", 2024-04-16T05:37:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:37:43.839 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 -- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faae80a3d50 msgr2=0x7faae80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faae80a3d50 0x7faae80a6210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faaf4005e30 tx=0x7faaf4048000 comp rx=0 tx=0).stop 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 -- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 msgr2=0x7faafc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 0x7faafc0fce10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faae4005fd0 tx=0x7faae400dde0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 -- 172.21.15.32:0/2676766960 shutdown_connections 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faafc0fd350 0x7faafc1a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faae80a3d50 0x7faae80a6210 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc106430 0x7faafc0fce10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 --2- 172.21.15.32:0/2676766960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faafc101880 0x7faafc0fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 -- 172.21.15.32:0/2676766960 >> 172.21.15.32:0/2676766960 conn(0x7faafc077490 msgr2=0x7faafc100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 -- 172.21.15.32:0/2676766960 shutdown_connections 2024-04-16T05:37:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.832+0000 7fab023ba640 1 -- 172.21.15.32:0/2676766960 wait complete. 2024-04-16T05:37:43.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:37:43.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:37:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 -- 172.21.15.32:0/913085283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868107360 msgr2=0x7f986810ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 --2- 172.21.15.32:0/913085283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868107360 0x7f986810ef70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9860009ca0 tx=0x7f986002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 -- 172.21.15.32:0/913085283 shutdown_connections 2024-04-16T05:37:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 --2- 172.21.15.32:0/913085283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868107360 0x7f986810ef70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 --2- 172.21.15.32:0/913085283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9868101550 0x7f9868106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 --2- 172.21.15.32:0/913085283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9868100ba0 0x7f9868100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 -- 172.21.15.32:0/913085283 >> 172.21.15.32:0/913085283 conn(0x7f9868076190 msgr2=0x7f98680765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 -- 172.21.15.32:0/913085283 shutdown_connections 2024-04-16T05:37:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.952+0000 7f986d295640 1 -- 172.21.15.32:0/913085283 wait complete. 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f986d295640 1 Processor -- start 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f986d295640 1 -- start start 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f986d295640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9868100ba0 0x7f986819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f986d295640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9868107360 0x7f986819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f986d295640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868198790 0x7f9868198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f986d295640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112450 con 0x7f9868198790 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f986d295640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98681122d0 con 0x7f9868107360 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f986d295640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98681125d0 con 0x7f9868100ba0 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868198790 0x7f9868198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868198790 0x7f9868198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46630/0 (socket says 172.21.15.32:46630) 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 -- 172.21.15.32:0/3607953447 learned_addr learned my addr 172.21.15.32:0/3607953447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f9866ffd640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9868100ba0 0x7f986819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f985ffff640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9868107360 0x7f986819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 -- 172.21.15.32:0/3607953447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9868100ba0 msgr2=0x7f986819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9868100ba0 0x7f986819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 -- 172.21.15.32:0/3607953447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9868107360 msgr2=0x7f986819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9868107360 0x7f986819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 -- 172.21.15.32:0/3607953447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9868199320 con 0x7f9868198790 2024-04-16T05:37:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f9866ffd640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9868100ba0 0x7f986819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:43.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.953+0000 7f98677fe640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868198790 0x7f9868198c40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f986002c700 tx=0x7f9860032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.954+0000 7f9864ff9640 1 -- 172.21.15.32:0/3607953447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860039040 con 0x7f9868198790 2024-04-16T05:37:43.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.954+0000 7f9864ff9640 1 -- 172.21.15.32:0/3607953447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9860044070 con 0x7f9868198790 2024-04-16T05:37:43.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.954+0000 7f986d295640 1 -- 172.21.15.32:0/3607953447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98681995b0 con 0x7f9868198790 2024-04-16T05:37:43.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.954+0000 7f9864ff9640 1 -- 172.21.15.32:0/3607953447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600042a0 con 0x7f9868198790 2024-04-16T05:37:43.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.954+0000 7f986d295640 1 -- 172.21.15.32:0/3607953447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a5790 con 0x7f9868198790 2024-04-16T05:37:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.956+0000 7f9864ff9640 1 -- 172.21.15.32:0/3607953447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9860004440 con 0x7f9868198790 2024-04-16T05:37:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.956+0000 7f986fe7d740 1 Processor -- start 2024-04-16T05:37:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.956+0000 7f986fe7d740 1 -- start start 2024-04-16T05:37:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531260 0x561280531640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531b80 0x56128053a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 0x56128053cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f986fe7d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561280492180 con 0x561280531260 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f986fe7d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56128042f360 con 0x56128053a760 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f986fe7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56128048e140 con 0x561280531b80 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531260 0x561280531640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531260 0x561280531640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46640/0 (socket says 172.21.15.32:46640) 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 -- 172.21.15.32:0/2826035258 learned_addr learned my addr 172.21.15.32:0/2826035258 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f9864ff9640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98540a3da0 0x7f98540a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f9866ffd640 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531b80 0x56128053a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 0x56128053cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f98677fe640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98540a3da0 0x7f98540a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 -- 172.21.15.32:0/2826035258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531b80 msgr2=0x56128053a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531b80 0x56128053a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 -- 172.21.15.32:0/2826035258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 msgr2=0x56128053cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 0x56128053cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 -- 172.21.15.32:0/2826035258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128026e7d0 con 0x561280531260 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f9866ffd640 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531b80 0x56128053a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f9864ff9640 1 -- 172.21.15.32:0/3607953447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f98600f2e90 con 0x7f9868198790 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985ffff640 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531260 0x561280531640 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9850004650 tx=0x7f98500094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b799d830cf08a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.957+0000 7f985d7fa640 1 -- 172.21.15.32:0/2826035258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985001b070 con 0x561280531260 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f98677fe640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98540a3da0 0x7f98540a6260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f98600bb000 tx=0x7f98600095c0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f985d7fa640 1 -- 172.21.15.32:0/2826035258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9850010960 con 0x561280531260 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f985d7fa640 1 -- 172.21.15.32:0/2826035258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500137a0 con 0x561280531260 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 -- 172.21.15.32:0/2826035258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531260 msgr2=0x561280531640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531260 0x561280531640 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9850004650 tx=0x7f98500094a0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 -- 172.21.15.32:0/2826035258 shutdown_connections 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 0x56128053cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531b80 0x56128053a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2826035258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531260 0x561280531640 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 -- 172.21.15.32:0/2826035258 >> 172.21.15.32:0/2826035258 conn(0x56128052b990 msgr2=0x56128052bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 -- 172.21.15.32:0/2826035258 shutdown_connections 2024-04-16T05:37:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.958+0000 7f986fe7d740 1 -- 172.21.15.32:0/2826035258 wait complete. 2024-04-16T05:37:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 1 -- start start 2024-04-16T05:37:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 1 Processor -- start 2024-04-16T05:37:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531260 0x561280537860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x561280537da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 0x5612805382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56128053d550 con 0x561280531b80 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56128044b2c0 con 0x56128053a760 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56128053d6d0 con 0x561280531260 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f986fe7d740 10 client.0 _ll_register_callbacks cb 0x561280432090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f9866ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x561280537da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f9866ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x561280537da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46656/0 (socket says 172.21.15.32:46656) 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f9866ffd640 1 -- 172.21.15.32:0/1359770691 learned_addr learned my addr 172.21.15.32:0/1359770691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f985ffff640 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531260 0x561280537860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f985ffff640 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 0x5612805382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f9866ffd640 1 -- 172.21.15.32:0/1359770691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531260 msgr2=0x561280537860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f9866ffd640 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531260 0x561280537860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f9866ffd640 1 -- 172.21.15.32:0/1359770691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 msgr2=0x5612805382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f9866ffd640 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 0x5612805382e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.959+0000 7f9866ffd640 1 -- 172.21.15.32:0/1359770691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612801a43f0 con 0x561280531b80 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f9866ffd640 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x561280537da0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f98580113e0 tx=0x7f98580118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c866ae9e791fabc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f984e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f984e7fc640 1 -- 172.21.15.32:0/1359770691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858012710 con 0x561280531b80 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f984e7fc640 1 -- 172.21.15.32:0/1359770691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f98580128b0 con 0x561280531b80 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 1 -- 172.21.15.32:0/1359770691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9850010960 con 0x561280531b80 2024-04-16T05:37:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f984e7fc640 1 -- 172.21.15.32:0/1359770691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9858012ab0 con 0x561280531b80 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 10 client.16359 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 1 -- 172.21.15.32:0/1359770691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5612801640f0 con 0x561280531b80 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f984e7fc640 1 -- 172.21.15.32:0/1359770691 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9850010960 con 0x561280531b80 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f984e7fc640 1 -- 172.21.15.32:0/1359770691 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f985801a3d0 con 0x561280531b80 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 10 client.16359 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 20 client.16359 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56128054ac70 0x56128054d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 4 client.16359 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:37:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.960+0000 7f986fe7d740 1 -- 172.21.15.32:0/1359770691 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56128049fba0 con 0x56128054ac70 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.961+0000 7f985ffff640 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56128054ac70 0x56128054d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.961+0000 7f985ffff640 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56128054ac70 0x56128054d0b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f9896e1f7664b server_cookie=41e7d95ab13f6ddc in_seq=0 out_seq=0 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.961+0000 7f984e7fc640 10 client.16359 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f984e7fc640 1 -- 172.21.15.32:0/1359770691 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56128049fba0 con 0x56128054ac70 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f984e7fc640 10 client.16359 handle_command_reply: tid=0 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 client.16359 shutdown 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 -- 172.21.15.32:0/1359770691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 msgr2=0x561280537da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x561280537da0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f98580113e0 tx=0x7f98580118a0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 -- 172.21.15.32:0/1359770691 shutdown_connections 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128053a760 0x5612805382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x561280537da0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56128054ac70 0x56128054d0b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1359770691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561280531260 0x561280537860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 -- 172.21.15.32:0/1359770691 >> 172.21.15.32:0/1359770691 conn(0x56128052b990 msgr2=0x561280526760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f984e7fc640 0 client.16359 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 -- 172.21.15.32:0/1359770691 shutdown_connections 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 1 -- 172.21.15.32:0/1359770691 wait complete. 2024-04-16T05:37:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.962+0000 7f986fe7d740 20 client.16359 trim_cache size 0 max 16384 2024-04-16T05:37:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.979+0000 7f986fe7d740 1 Processor -- start 2024-04-16T05:37:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.979+0000 7f986fe7d740 1 -- start start 2024-04-16T05:37:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 0x561280546410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 0x561280538100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f986fe7d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f98681af9b0 con 0x561280531b80 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f986fe7d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ad280 con 0x561280531260 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f986fe7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ab0d0 con 0x56128053a760 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805367a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46660/0 (socket says 172.21.15.32:46660) 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 -- 172.21.15.32:0/1216759584 learned_addr learned my addr 172.21.15.32:0/1216759584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 -- 172.21.15.32:0/1216759584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 msgr2=0x561280538100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 --2- 172.21.15.32:0/1216759584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 0x561280538100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 -- 172.21.15.32:0/1216759584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 msgr2=0x561280546410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 --2- 172.21.15.32:0/1216759584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 0x561280546410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 -- 172.21.15.32:0/1216759584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128053cec0 con 0x561280531b80 2024-04-16T05:37:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f9866ffd640 1 --2- 172.21.15.32:0/1216759584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805367a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f985801a840 tx=0x7f985800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225a00dbfc757a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f985cff9640 1 -- 172.21.15.32:0/1216759584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985802d070 con 0x561280531b80 2024-04-16T05:37:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f985cff9640 1 -- 172.21.15.32:0/1216759584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9858012e50 con 0x561280531b80 2024-04-16T05:37:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.980+0000 7f985cff9640 1 -- 172.21.15.32:0/1216759584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858005040 con 0x561280531b80 2024-04-16T05:37:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 -- 172.21.15.32:0/1216759584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 msgr2=0x5612805367a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1216759584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805367a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f985801a840 tx=0x7f985800bfd0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 -- 172.21.15.32:0/1216759584 shutdown_connections 2024-04-16T05:37:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1216759584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 0x561280538100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1216759584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805367a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 --2- 172.21.15.32:0/1216759584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 0x561280546410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 -- 172.21.15.32:0/1216759584 >> 172.21.15.32:0/1216759584 conn(0x56128052b990 msgr2=0x561280526760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 -- 172.21.15.32:0/1216759584 shutdown_connections 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 -- 172.21.15.32:0/1216759584 wait complete. 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 -- start start 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.981+0000 7f986fe7d740 1 Processor -- start 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 0x561280542e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f986fe7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 0x561280540910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f986fe7d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ab740 con 0x561280531b80 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f986fe7d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98681acbb0 con 0x561280531260 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f986fe7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98681abe10 con 0x56128053a760 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f986fe7d740 10 client.0 _ll_register_callbacks cb 0x7f985800b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46672/0 (socket says 172.21.15.32:46672) 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 -- 172.21.15.32:0/2074161761 learned_addr learned my addr 172.21.15.32:0/2074161761 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:37:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f985ffff640 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 0x561280542e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f985ffff640 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 0x561280540910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 -- 172.21.15.32:0/2074161761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 msgr2=0x561280540910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 0x561280540910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 -- 172.21.15.32:0/2074161761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 msgr2=0x561280542e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 0x561280542e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 -- 172.21.15.32:0/2074161761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612805b0240 con 0x561280531b80 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f9866ffd640 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805401d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f985801a8c0 tx=0x7f985801af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a579089ec48cfc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f984e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:37:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.982+0000 7f984e7fc640 1 -- 172.21.15.32:0/2074161761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985802d070 con 0x561280531b80 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f984e7fc640 1 -- 172.21.15.32:0/2074161761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f985801aa30 con 0x561280531b80 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 1 -- 172.21.15.32:0/2074161761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9858012e50 con 0x561280531b80 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f984e7fc640 1 -- 172.21.15.32:0/2074161761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9858012510 con 0x561280531b80 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 10 client.16365 fetch_fsmap learned FSMap version 25 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 1 -- 172.21.15.32:0/2074161761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5612805b13b0 con 0x561280531b80 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f984e7fc640 1 -- 172.21.15.32:0/2074161761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9858012e50 con 0x561280531b80 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f984e7fc640 1 -- 172.21.15.32:0/2074161761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f985803a030 con 0x561280531b80 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 10 client.16365 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:37:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 20 client.16365 populate_metadata read hostname 'smithi032' 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612805b1880 0x5612805b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 4 client.16365 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f986fe7d740 1 -- 172.21.15.32:0/2074161761 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561280499a10 con 0x5612805b1880 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.983+0000 7f985ffff640 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612805b1880 0x5612805b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f984e7fc640 10 client.16365 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f985ffff640 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612805b1880 0x5612805b3c70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc662468ba3e09d server_cookie=667d5ae345c01ffd in_seq=0 out_seq=0 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f984e7fc640 1 -- 172.21.15.32:0/2074161761 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561280499a10 con 0x5612805b1880 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f984e7fc640 10 client.16365 handle_command_reply: tid=0 2024-04-16T05:37:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 client.16365 shutdown 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 -- 172.21.15.32:0/2074161761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 msgr2=0x5612805401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805401d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f985801a8c0 tx=0x7f985801af30 comp rx=0 tx=0).stop 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 -- 172.21.15.32:0/2074161761 shutdown_connections 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612805b1880 0x5612805b3c70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128053a760 0x561280540910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561280531b80 0x5612805401d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 --2- 172.21.15.32:0/2074161761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561280531260 0x561280542e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f984e7fc640 0 client.16365 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:37:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 -- 172.21.15.32:0/2074161761 >> 172.21.15.32:0/2074161761 conn(0x56128052b990 msgr2=0x561280526760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 -- 172.21.15.32:0/2074161761 shutdown_connections 2024-04-16T05:37:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 1 -- 172.21.15.32:0/2074161761 wait complete. 2024-04-16T05:37:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.984+0000 7f986fe7d740 20 client.16365 trim_cache size 0 max 16384 2024-04-16T05:37:43.992 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:37:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:37:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:37:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "eb775d57-aff1-47cc-9b09-9eb499166532": { 2024-04-16T05:37:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:37:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "eb775d57-aff1-47cc-9b09-9eb499166532", 2024-04-16T05:37:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:37:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:37:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:37:43.993 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:37:43.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 -- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98540a3da0 msgr2=0x7f98540a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98540a3da0 0x7f98540a6260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f98600bb000 tx=0x7f98600095c0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 -- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868198790 msgr2=0x7f9868198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868198790 0x7f9868198c40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f986002c700 tx=0x7f9860032f20 comp rx=0 tx=0).stop 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 -- 172.21.15.32:0/3607953447 shutdown_connections 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98540a3da0 0x7f98540a6260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9868198790 0x7f9868198c40 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9868107360 0x7f986819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 --2- 172.21.15.32:0/3607953447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9868100ba0 0x7f986819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 -- 172.21.15.32:0/3607953447 >> 172.21.15.32:0/3607953447 conn(0x7f9868076190 msgr2=0x7f98680fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 -- 172.21.15.32:0/3607953447 shutdown_connections 2024-04-16T05:37:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:37:43.985+0000 7f986d295640 1 -- 172.21.15.32:0/3607953447 wait complete. 2024-04-16T05:37:43.996 INFO:tasks.cephfs.filesystem:scrub status for tag:eb775d57-aff1-47cc-9b09-9eb499166532 - {'path': '/', 'tag': 'eb775d57-aff1-47cc-9b09-9eb499166532', 'options': 'recursive,force'} 2024-04-16T05:37:45.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:45 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:37:45.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:45 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:45.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:45 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:37:45.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:45 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:45.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:45 smithi032 ceph-mon[32351]: pgmap v87: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:45.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:45 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:37:45.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:45 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:45.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:45 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:37:45.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:45 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:45.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:45 smithi134 ceph-mon[36153]: pgmap v87: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:45.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:45 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:37:45.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:45 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:45.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:45 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:37:45.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:45 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:37:45.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:45 smithi082 ceph-mon[36195]: pgmap v87: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:46.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:46 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:37:46.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:46 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:37:46.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:46 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:37:47.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:47 smithi032 ceph-mon[32351]: pgmap v88: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:47.551 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:47 smithi082 ceph-mon[36195]: pgmap v88: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:47.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:47 smithi134 ceph-mon[36153]: pgmap v88: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:37:49.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:49 smithi032 ceph-mon[32351]: pgmap v89: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:49.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:49 smithi134 ceph-mon[36153]: pgmap v89: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:49 smithi082 ceph-mon[36195]: pgmap v89: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:37:51.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:51 smithi032 ceph-mon[32351]: pgmap v90: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:51.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:51 smithi134 ceph-mon[36153]: pgmap v90: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:51.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:51 smithi082 ceph-mon[36195]: pgmap v90: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:53.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:53 smithi032 ceph-mon[32351]: pgmap v91: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:37:53.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:53 smithi134 ceph-mon[36153]: pgmap v91: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:37:53.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:53 smithi082 ceph-mon[36195]: pgmap v91: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:37:55.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:55 smithi032 ceph-mon[32351]: pgmap v92: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:55.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:55 smithi134 ceph-mon[36153]: pgmap v92: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:55.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:55 smithi082 ceph-mon[36195]: pgmap v92: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:57.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:57 smithi032 ceph-mon[32351]: pgmap v93: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:57.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:57 smithi134 ceph-mon[36153]: pgmap v93: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:57.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:57 smithi082 ceph-mon[36195]: pgmap v93: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:37:59.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:37:59 smithi032 ceph-mon[32351]: pgmap v94: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:37:59.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:37:59 smithi134 ceph-mon[36153]: pgmap v94: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:37:59.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:37:59 smithi082 ceph-mon[36195]: pgmap v94: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:01.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:01 smithi032 ceph-mon[32351]: pgmap v95: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:01.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:01 smithi134 ceph-mon[36153]: pgmap v95: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:01.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:01 smithi082 ceph-mon[36195]: pgmap v95: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:03.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:03 smithi032 ceph-mon[32351]: pgmap v96: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:03 smithi134 ceph-mon[36153]: pgmap v96: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:03.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:03 smithi082 ceph-mon[36195]: pgmap v96: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:04.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:04 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:38:04.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:04 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:38:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:04 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:38:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:05 smithi032 ceph-mon[32351]: pgmap v97: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:38:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:05 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:38:05.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:05 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:38:05.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:05 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:38:05.535 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:05 smithi134 ceph-mon[36153]: pgmap v97: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:38:05.536 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:05 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:38:05.536 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:05 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:38:05.536 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:05 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:38:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:05 smithi082 ceph-mon[36195]: pgmap v97: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:38:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:05 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:38:05.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:05 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:38:05.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:05 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:38:07.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:07 smithi032 ceph-mon[32351]: pgmap v98: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:38:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:07 smithi134 ceph-mon[36153]: pgmap v98: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:38:07.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:07 smithi082 ceph-mon[36195]: pgmap v98: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:38:09.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:09 smithi032 ceph-mon[32351]: pgmap v99: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:09 smithi134 ceph-mon[36153]: pgmap v99: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:09.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:09 smithi082 ceph-mon[36195]: pgmap v99: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:11.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:11 smithi032 ceph-mon[32351]: pgmap v100: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:38:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:11 smithi134 ceph-mon[36153]: pgmap v100: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:38:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:11 smithi082 ceph-mon[36195]: pgmap v100: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:38:13.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:13 smithi032 ceph-mon[32351]: pgmap v101: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:13.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:13 smithi134 ceph-mon[36153]: pgmap v101: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:13.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:13 smithi082 ceph-mon[36195]: pgmap v101: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:38:13.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:38:13.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:38:14.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 -- 172.21.15.32:0/2241036494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101bf0 msgr2=0x7f7c8c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 --2- 172.21.15.32:0/2241036494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101bf0 0x7f7c8c10dd30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009cd0 tx=0x7f7c8402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 -- 172.21.15.32:0/2241036494 shutdown_connections 2024-04-16T05:38:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 --2- 172.21.15.32:0/2241036494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101bf0 0x7f7c8c10dd30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 --2- 172.21.15.32:0/2241036494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c101250 0x7f7c8c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 --2- 172.21.15.32:0/2241036494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c1070e0 0x7f7c8c1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 -- 172.21.15.32:0/2241036494 >> 172.21.15.32:0/2241036494 conn(0x7f7c8c076d00 msgr2=0x7f7c8c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 -- 172.21.15.32:0/2241036494 shutdown_connections 2024-04-16T05:38:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.119+0000 7f7c912c7640 1 -- 172.21.15.32:0/2241036494 wait complete. 2024-04-16T05:38:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.120+0000 7f7c912c7640 1 Processor -- start 2024-04-16T05:38:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.120+0000 7f7c912c7640 1 -- start start 2024-04-16T05:38:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.120+0000 7f7c912c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101250 0x7f7c8c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.120+0000 7f7c912c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c101bf0 0x7f7c8c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.120+0000 7f7c912c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c1070e0 0x7f7c8c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.120+0000 7f7c912c7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1112e0 con 0x7f7c8c101250 2024-04-16T05:38:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.120+0000 7f7c912c7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c111160 con 0x7f7c8c1070e0 2024-04-16T05:38:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c912c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c111460 con 0x7f7c8c101bf0 2024-04-16T05:38:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101250 0x7f7c8c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101250 0x7f7c8c19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36566/0 (socket says 172.21.15.32:36566) 2024-04-16T05:38:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 -- 172.21.15.32:0/3411249869 learned_addr learned my addr 172.21.15.32:0/3411249869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c1070e0 0x7f7c8c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8a7fc640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c101bf0 0x7f7c8c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 -- 172.21.15.32:0/3411249869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c101bf0 msgr2=0x7f7c8c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c101bf0 0x7f7c8c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 -- 172.21.15.32:0/3411249869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c1070e0 msgr2=0x7f7c8c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c1070e0 0x7f7c8c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 -- 172.21.15.32:0/3411249869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c199000 con 0x7f7c8c101250 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c1070e0 0x7f7c8c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8a7fc640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c101bf0 0x7f7c8c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101250 0x7f7c8c19e630 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00d950 tx=0x7f7c7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c73fff640 1 -- 172.21.15.32:0/3411249869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c017030 con 0x7f7c8c101250 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c73fff640 1 -- 172.21.15.32:0/3411249869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c004020 con 0x7f7c8c101250 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c912c7640 1 -- 172.21.15.32:0/3411249869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c1992f0 con 0x7f7c8c101250 2024-04-16T05:38:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c73fff640 1 -- 172.21.15.32:0/3411249869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0047a0 con 0x7f7c8c101250 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.121+0000 7f7c912c7640 1 -- 172.21.15.32:0/3411249869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c1a55b0 con 0x7f7c8c101250 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c73fff640 1 -- 172.21.15.32:0/3411249869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c7c0041d0 con 0x7f7c8c101250 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c93f82740 1 Processor -- start 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c93f82740 1 -- start start 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57c2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 0x560cc57cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c93f82740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560cc5727180 con 0x560cc57c1e50 2024-04-16T05:38:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c93f82740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560cc56c4360 con 0x560cc57c2770 2024-04-16T05:38:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c93f82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560cc5723140 con 0x560cc57cb350 2024-04-16T05:38:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c73fff640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7c740a3d20 0x7f7c740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c8b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c8b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36568/0 (socket says 172.21.15.32:36568) 2024-04-16T05:38:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c8b7fe640 1 -- 172.21.15.32:0/2400286208 learned_addr learned my addr 172.21.15.32:0/2400286208 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c8a7fc640 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 0x560cc57cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7c740a3d20 0x7f7c740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.124+0000 7f7c73fff640 1 -- 172.21.15.32:0/3411249869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7c7c0d05f0 con 0x7f7c8c101250 2024-04-16T05:38:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c8affd640 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c8b7fe640 1 -- 172.21.15.32:0/2400286208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 msgr2=0x560cc57cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 0x560cc57cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c8b7fe640 1 -- 172.21.15.32:0/2400286208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 msgr2=0x560cc57cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c8b7fe640 1 -- 172.21.15.32:0/2400286208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc55037d0 con 0x560cc57c1e50 2024-04-16T05:38:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57c2230 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009430 tx=0x7f7c84032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec50030340b40bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c717fa640 1 -- 172.21.15.32:0/2400286208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c840328f0 con 0x560cc57c1e50 2024-04-16T05:38:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c717fa640 1 -- 172.21.15.32:0/2400286208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c84004520 con 0x560cc57c1e50 2024-04-16T05:38:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.125+0000 7f7c717fa640 1 -- 172.21.15.32:0/2400286208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84039070 con 0x560cc57c1e50 2024-04-16T05:38:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 -- 172.21.15.32:0/2400286208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 msgr2=0x560cc57c2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7c740a3d20 0x7f7c740a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c8404c000 tx=0x7f7c84048000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57c2230 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009430 tx=0x7f7c84032f40 comp rx=0 tx=0).stop 2024-04-16T05:38:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 -- 172.21.15.32:0/2400286208 shutdown_connections 2024-04-16T05:38:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 0x560cc57cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 --2- 172.21.15.32:0/2400286208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57c2230 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 -- 172.21.15.32:0/2400286208 >> 172.21.15.32:0/2400286208 conn(0x560cc57bc580 msgr2=0x560cc57bc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 -- 172.21.15.32:0/2400286208 shutdown_connections 2024-04-16T05:38:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 -- 172.21.15.32:0/2400286208 wait complete. 2024-04-16T05:38:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 -- start start 2024-04-16T05:38:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.126+0000 7f7c93f82740 1 Processor -- start 2024-04-16T05:38:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c1e50 0x560cc57c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57cb350 0x560cc57c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c93f82740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560cc57ce140 con 0x560cc57cb350 2024-04-16T05:38:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c93f82740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560cc56e02c0 con 0x560cc57c2770 2024-04-16T05:38:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c93f82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560cc57ce2c0 con 0x560cc57c1e50 2024-04-16T05:38:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c93f82740 10 client.0 _ll_register_callbacks cb 0x560cc56c7090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57cb350 0x560cc57c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57cb350 0x560cc57c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36580/0 (socket says 172.21.15.32:36580) 2024-04-16T05:38:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 -- 172.21.15.32:0/96473339 learned_addr learned my addr 172.21.15.32:0/96473339 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8affd640 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c1e50 0x560cc57c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8a7fc640 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 -- 172.21.15.32:0/96473339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c1e50 msgr2=0x560cc57c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c1e50 0x560cc57c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 -- 172.21.15.32:0/96473339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 msgr2=0x560cc57c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 -- 172.21.15.32:0/96473339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc54393f0 con 0x560cc57cb350 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8affd640 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c1e50 0x560cc57c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57cb350 0x560cc57c8e30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7c84032380 tx=0x7f7c84032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff49e2309a35a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c6eff5640 1 -- 172.21.15.32:0/96473339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84039040 con 0x560cc57cb350 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c93f82740 1 -- 172.21.15.32:0/96473339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c84004520 con 0x560cc57cb350 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.127+0000 7f7c6eff5640 1 -- 172.21.15.32:0/96473339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7c84004240 con 0x560cc57cb350 2024-04-16T05:38:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c6eff5640 1 -- 172.21.15.32:0/96473339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c84060050 con 0x560cc57cb350 2024-04-16T05:38:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c93f82740 10 client.16377 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c93f82740 1 -- 172.21.15.32:0/96473339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560cc53f90f0 con 0x560cc57cb350 2024-04-16T05:38:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c6eff5640 1 -- 172.21.15.32:0/96473339 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c84004520 con 0x560cc57cb350 2024-04-16T05:38:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c6eff5640 1 -- 172.21.15.32:0/96473339 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c84043920 con 0x560cc57cb350 2024-04-16T05:38:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c93f82740 10 client.16377 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c93f82740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c93f82740 20 client.16377 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c93f82740 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560cc57db590 0x560cc57dd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c93f82740 4 client.16377 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c93f82740 1 -- 172.21.15.32:0/96473339 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cc5734ba0 con 0x560cc57db590 2024-04-16T05:38:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.128+0000 7f7c8affd640 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560cc57db590 0x560cc57dd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.129+0000 7f7c8affd640 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560cc57db590 0x560cc57dd980 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1ec0c99b8e3bb0 server_cookie=30d3d45302e28e60 in_seq=0 out_seq=0 2024-04-16T05:38:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.129+0000 7f7c6eff5640 10 client.16377 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.129+0000 7f7c6eff5640 1 -- 172.21.15.32:0/96473339 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cc5734ba0 con 0x560cc57db590 2024-04-16T05:38:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.129+0000 7f7c6eff5640 10 client.16377 handle_command_reply: tid=0 2024-04-16T05:38:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.129+0000 7f7c93f82740 1 client.16377 shutdown 2024-04-16T05:38:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 -- 172.21.15.32:0/96473339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57cb350 msgr2=0x560cc57c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57cb350 0x560cc57c8e30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7c84032380 tx=0x7f7c84032b00 comp rx=0 tx=0).stop 2024-04-16T05:38:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 -- 172.21.15.32:0/96473339 shutdown_connections 2024-04-16T05:38:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57cb350 0x560cc57c8e30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560cc57db590 0x560cc57dd980 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 --2- 172.21.15.32:0/96473339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c1e50 0x560cc57c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 -- 172.21.15.32:0/96473339 >> 172.21.15.32:0/96473339 conn(0x560cc57bc580 msgr2=0x560cc57be2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c6eff5640 0 client.16377 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 -- 172.21.15.32:0/96473339 shutdown_connections 2024-04-16T05:38:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 1 -- 172.21.15.32:0/96473339 wait complete. 2024-04-16T05:38:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.130+0000 7f7c93f82740 20 client.16377 trim_cache size 0 max 16384 2024-04-16T05:38:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.147+0000 7f7c93f82740 1 Processor -- start 2024-04-16T05:38:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.147+0000 7f7c93f82740 1 -- start start 2024-04-16T05:38:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57d7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 0x560cc57bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c93f82740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1b1c40 con 0x560cc57c1e50 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c93f82740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1af510 con 0x560cc57c2770 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c93f82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1ad360 con 0x560cc57cb350 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36586/0 (socket says 172.21.15.32:36586) 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 -- 172.21.15.32:0/3479565725 learned_addr learned my addr 172.21.15.32:0/3479565725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 0x560cc57bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 -- 172.21.15.32:0/3479565725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 msgr2=0x560cc57bb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 0x560cc57bb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8a7fc640 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 -- 172.21.15.32:0/3479565725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 msgr2=0x560cc57d7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57d7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 -- 172.21.15.32:0/3479565725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc57cd740 con 0x560cc57c1e50 2024-04-16T05:38:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57caae0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0ce000 tx=0x7f7c7c0d0910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48674bf449bb2165 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c70ff9640 1 -- 172.21.15.32:0/3479565725 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0a0600 con 0x560cc57c1e50 2024-04-16T05:38:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c70ff9640 1 -- 172.21.15.32:0/3479565725 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c0a4ec0 con 0x560cc57c1e50 2024-04-16T05:38:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.148+0000 7f7c70ff9640 1 -- 172.21.15.32:0/3479565725 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0a4a30 con 0x560cc57c1e50 2024-04-16T05:38:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 -- 172.21.15.32:0/3479565725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 msgr2=0x560cc57caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57caae0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0ce000 tx=0x7f7c7c0d0910 comp rx=0 tx=0).stop 2024-04-16T05:38:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 -- 172.21.15.32:0/3479565725 shutdown_connections 2024-04-16T05:38:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57cb350 0x560cc57bb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57c2770 0x560cc57d7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3479565725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57caae0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 -- 172.21.15.32:0/3479565725 >> 172.21.15.32:0/3479565725 conn(0x560cc57bc580 msgr2=0x560cc57baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 -- 172.21.15.32:0/3479565725 shutdown_connections 2024-04-16T05:38:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 -- 172.21.15.32:0/3479565725 wait complete. 2024-04-16T05:38:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 -- start start 2024-04-16T05:38:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.149+0000 7f7c93f82740 1 Processor -- start 2024-04-16T05:38:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c2770 0x560cc57d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57cb350 0x560cc57d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1ad9d0 con 0x560cc57c1e50 2024-04-16T05:38:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1aee40 con 0x560cc57cb350 2024-04-16T05:38:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1ae0a0 con 0x560cc57c2770 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 10 client.0 _ll_register_callbacks cb 0x7f7c84009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57d32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36594/0 (socket says 172.21.15.32:36594) 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 -- 172.21.15.32:0/3682930373 learned_addr learned my addr 172.21.15.32:0/3682930373 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57cb350 0x560cc57d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 -- 172.21.15.32:0/3682930373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c2770 msgr2=0x560cc57d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8a7fc640 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c2770 0x560cc57d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c2770 0x560cc57d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 -- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57cb350 msgr2=0x560cc57d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57cb350 0x560cc57d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 -- 172.21.15.32:0/3682930373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc57d0ff0 con 0x560cc57c1e50 2024-04-16T05:38:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8b7fe640 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57cb350 0x560cc57d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c8affd640 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57d32c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0a4990 tx=0x7f7c7c0a0600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05df98c31e01c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c6eff5640 1 -- 172.21.15.32:0/3682930373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c09a550 con 0x560cc57c1e50 2024-04-16T05:38:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c6eff5640 1 -- 172.21.15.32:0/3682930373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7c7c0bb170 con 0x560cc57c1e50 2024-04-16T05:38:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 1 -- 172.21.15.32:0/3682930373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c7c0a4ec0 con 0x560cc57c1e50 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c6eff5640 1 -- 172.21.15.32:0/3682930373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c7c0988c0 con 0x560cc57c1e50 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.150+0000 7f7c93f82740 10 client.16383 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c93f82740 1 -- 172.21.15.32:0/3682930373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560cc58412f0 con 0x560cc57c1e50 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c6eff5640 1 -- 172.21.15.32:0/3682930373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c0a4ec0 con 0x560cc57c1e50 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c6eff5640 1 -- 172.21.15.32:0/3682930373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c7c0a0160 con 0x560cc57c1e50 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c93f82740 10 client.16383 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c93f82740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c93f82740 20 client.16383 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560cc5841990 0x560cc5843db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c93f82740 4 client.16383 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c93f82740 1 -- 172.21.15.32:0/3682930373 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560cc572ea10 con 0x560cc5841990 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c8a7fc640 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560cc5841990 0x560cc5843db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c8a7fc640 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560cc5841990 0x560cc5843db0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4532c10879747389 server_cookie=c6715e86602740c6 in_seq=0 out_seq=0 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.151+0000 7f7c6eff5640 10 client.16383 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c6eff5640 1 -- 172.21.15.32:0/3682930373 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560cc572ea10 con 0x560cc5841990 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c6eff5640 10 client.16383 handle_command_reply: tid=0 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 client.16383 shutdown 2024-04-16T05:38:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 -- 172.21.15.32:0/3682930373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 msgr2=0x560cc57d32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57d32c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0a4990 tx=0x7f7c7c0a0600 comp rx=0 tx=0).stop 2024-04-16T05:38:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 -- 172.21.15.32:0/3682930373 shutdown_connections 2024-04-16T05:38:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cc57cb350 0x560cc57d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560cc5841990 0x560cc5843db0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc57c2770 0x560cc57d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 --2- 172.21.15.32:0/3682930373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560cc57c1e50 0x560cc57d32c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c6eff5640 0 client.16383 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 -- 172.21.15.32:0/3682930373 >> 172.21.15.32:0/3682930373 conn(0x560cc57bc580 msgr2=0x560cc57bb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 -- 172.21.15.32:0/3682930373 shutdown_connections 2024-04-16T05:38:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 1 -- 172.21.15.32:0/3682930373 wait complete. 2024-04-16T05:38:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c93f82740 20 client.16383 trim_cache size 0 max 16384 2024-04-16T05:38:14.162 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:38:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:38:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:38:14.163 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:38:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c912c7640 1 -- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7c740a3d20 msgr2=0x7f7c740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c912c7640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7c740a3d20 0x7f7c740a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c8404c000 tx=0x7f7c84048000 comp rx=0 tx=0).stop 2024-04-16T05:38:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c912c7640 1 -- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101250 msgr2=0x7f7c8c19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.152+0000 7f7c912c7640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101250 0x7f7c8c19e630 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00d950 tx=0x7f7c7c00de10 comp rx=0 tx=0).stop 2024-04-16T05:38:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.153+0000 7f7c912c7640 1 -- 172.21.15.32:0/3411249869 shutdown_connections 2024-04-16T05:38:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.153+0000 7f7c912c7640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c1070e0 0x7f7c8c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.153+0000 7f7c912c7640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c101bf0 0x7f7c8c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.153+0000 7f7c912c7640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7c740a3d20 0x7f7c740a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.153+0000 7f7c912c7640 1 --2- 172.21.15.32:0/3411249869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c8c101250 0x7f7c8c19e630 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.153+0000 7f7c912c7640 1 -- 172.21.15.32:0/3411249869 >> 172.21.15.32:0/3411249869 conn(0x7f7c8c076d00 msgr2=0x7f7c8c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.153+0000 7f7c912c7640 1 -- 172.21.15.32:0/3411249869 shutdown_connections 2024-04-16T05:38:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.153+0000 7f7c912c7640 1 -- 172.21.15.32:0/3411249869 wait complete. 2024-04-16T05:38:14.165 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:38:14.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:38:14.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 -- 172.21.15.32:0/1148299051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c106220 msgr2=0x7f005c10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 --2- 172.21.15.32:0/1148299051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c106220 0x7f005c10ddf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0054009ca0 tx=0x7f005402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 -- 172.21.15.32:0/1148299051 shutdown_connections 2024-04-16T05:38:14.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 --2- 172.21.15.32:0/1148299051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c106220 0x7f005c10ddf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 --2- 172.21.15.32:0/1148299051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c101b00 0x7f005c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 --2- 172.21.15.32:0/1148299051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c101150 0x7f005c101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 -- 172.21.15.32:0/1148299051 >> 172.21.15.32:0/1148299051 conn(0x7f005c078000 msgr2=0x7f005c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 -- 172.21.15.32:0/1148299051 shutdown_connections 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 -- 172.21.15.32:0/1148299051 wait complete. 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 Processor -- start 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.274+0000 7f0062396640 1 -- start start 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0062396640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c101150 0x7f005c107e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0062396640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106220 0x7f005c1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0062396640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c108910 0x7f005c1a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0062396640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1112e0 con 0x7f005c108910 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0062396640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f005c111160 con 0x7f005c101150 2024-04-16T05:38:14.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0062396640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f005c111460 con 0x7f005c106220 2024-04-16T05:38:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c108910 0x7f005c1a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c108910 0x7f005c1a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36614/0 (socket says 172.21.15.32:36614) 2024-04-16T05:38:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 -- 172.21.15.32:0/2448753974 learned_addr learned my addr 172.21.15.32:0/2448753974 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f005bfff640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c101150 0x7f005c107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 -- 172.21.15.32:0/2448753974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106220 msgr2=0x7f005c1083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f005b7fe640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106220 0x7f005c1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106220 0x7f005c1083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 -- 172.21.15.32:0/2448753974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c101150 msgr2=0x7f005c107e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c101150 0x7f005c107e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 -- 172.21.15.32:0/2448753974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c1a7130 con 0x7f005c108910 2024-04-16T05:38:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f005bfff640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c101150 0x7f005c107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f005b7fe640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106220 0x7f005c1083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0060933640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c108910 0x7f005c1a6a90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f005402c700 tx=0x7f0054032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f00597fa640 1 -- 172.21.15.32:0/2448753974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054041020 con 0x7f005c108910 2024-04-16T05:38:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c1a73c0 con 0x7f005c108910 2024-04-16T05:38:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f00597fa640 1 -- 172.21.15.32:0/2448753974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0054034cd0 con 0x7f005c108910 2024-04-16T05:38:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f00597fa640 1 -- 172.21.15.32:0/2448753974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054039040 con 0x7f005c108910 2024-04-16T05:38:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.275+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1a78b0 con 0x7f005c108910 2024-04-16T05:38:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.276+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c068e20 con 0x7f005c108910 2024-04-16T05:38:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.278+0000 7f00597fa640 1 -- 172.21.15.32:0/2448753974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0054044020 con 0x7f005c108910 2024-04-16T05:38:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.278+0000 7f00597fa640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00440a3d20 0x7f00440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.278+0000 7f005bfff640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00440a3d20 0x7f00440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.278+0000 7f00597fa640 1 -- 172.21.15.32:0/2448753974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f00540f2930 con 0x7f005c108910 2024-04-16T05:38:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.279+0000 7f005bfff640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00440a3d20 0x7f00440a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f004c009b40 tx=0x7f004c005e50 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.283+0000 7f00597fa640 1 -- 172.21.15.32:0/2448753974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f005404b020 con 0x7f005c108910 2024-04-16T05:38:14.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.510+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f005c101ba0 con 0x7f005c108910 2024-04-16T05:38:14.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.511+0000 7f00597fa640 1 -- 172.21.15.32:0/2448753974 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f00540beda0 con 0x7f005c108910 2024-04-16T05:38:14.510 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:38:14.510 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:38:14.510 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:38:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.512+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00440a3d20 msgr2=0x7f00440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.512+0000 7f0062396640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00440a3d20 0x7f00440a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f004c009b40 tx=0x7f004c005e50 comp rx=0 tx=0).stop 2024-04-16T05:38:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.512+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c108910 msgr2=0x7f005c1a6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.512+0000 7f0062396640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c108910 0x7f005c1a6a90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f005402c700 tx=0x7f0054032f20 comp rx=0 tx=0).stop 2024-04-16T05:38:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.513+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 shutdown_connections 2024-04-16T05:38:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.513+0000 7f0062396640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00440a3d20 0x7f00440a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.513+0000 7f0062396640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c108910 0x7f005c1a6a90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.513+0000 7f0062396640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106220 0x7f005c1083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.513+0000 7f0062396640 1 --2- 172.21.15.32:0/2448753974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c101150 0x7f005c107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.513+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 >> 172.21.15.32:0/2448753974 conn(0x7f005c078000 msgr2=0x7f005c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.513+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 shutdown_connections 2024-04-16T05:38:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.513+0000 7f0062396640 1 -- 172.21.15.32:0/2448753974 wait complete. 2024-04-16T05:38:14.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:38:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 -- 172.21.15.32:0/2245262852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac0108110 msgr2=0x7faac01084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 --2- 172.21.15.32:0/2245262852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac0108110 0x7faac01084f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faabc009930 tx=0x7faabc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 -- 172.21.15.32:0/2245262852 shutdown_connections 2024-04-16T05:38:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 --2- 172.21.15.32:0/2245262852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faac006b020 0x7faac010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 --2- 172.21.15.32:0/2245262852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac006a660 0x7faac006aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 --2- 172.21.15.32:0/2245262852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac0108110 0x7faac01084f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 -- 172.21.15.32:0/2245262852 >> 172.21.15.32:0/2245262852 conn(0x7faac0077cf0 msgr2=0x7faac0078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 -- 172.21.15.32:0/2245262852 shutdown_connections 2024-04-16T05:38:14.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.632+0000 7faac73a8640 1 -- 172.21.15.32:0/2245262852 wait complete. 2024-04-16T05:38:14.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac73a8640 1 Processor -- start 2024-04-16T05:38:14.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac73a8640 1 -- start start 2024-04-16T05:38:14.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac73a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac006a660 0x7faac019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac73a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac006b020 0x7faac019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac73a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faac0108110 0x7faac0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac73a8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faac0112380 con 0x7faac006b020 2024-04-16T05:38:14.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac73a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faac0112200 con 0x7faac0108110 2024-04-16T05:38:14.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac73a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faac0112500 con 0x7faac006a660 2024-04-16T05:38:14.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac006b020 0x7faac019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac006b020 0x7faac019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36622/0 (socket says 172.21.15.32:36622) 2024-04-16T05:38:14.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 -- 172.21.15.32:0/304826681 learned_addr learned my addr 172.21.15.32:0/304826681 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac5144640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac006a660 0x7faac019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac5945640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faac0108110 0x7faac0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 -- 172.21.15.32:0/304826681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac006a660 msgr2=0x7faac019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac006a660 0x7faac019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 -- 172.21.15.32:0/304826681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faac0108110 msgr2=0x7faac0198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faac0108110 0x7faac0198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 -- 172.21.15.32:0/304826681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac0198de0 con 0x7faac006b020 2024-04-16T05:38:14.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac5945640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faac0108110 0x7faac0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.633+0000 7faac4943640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac006b020 0x7faac019e9a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faab000d950 tx=0x7faab000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.634+0000 7faab67fc640 1 -- 172.21.15.32:0/304826681 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0017030 con 0x7faac006b020 2024-04-16T05:38:14.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.634+0000 7faab67fc640 1 -- 172.21.15.32:0/304826681 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab0004020 con 0x7faac006b020 2024-04-16T05:38:14.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.634+0000 7faac73a8640 1 -- 172.21.15.32:0/304826681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac01990d0 con 0x7faac006b020 2024-04-16T05:38:14.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.634+0000 7faab67fc640 1 -- 172.21.15.32:0/304826681 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00047a0 con 0x7faac006b020 2024-04-16T05:38:14.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.634+0000 7faac73a8640 1 -- 172.21.15.32:0/304826681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac01a5320 con 0x7faac006b020 2024-04-16T05:38:14.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faab67fc640 1 -- 172.21.15.32:0/304826681 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faab00041d0 con 0x7faac006b020 2024-04-16T05:38:14.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faac9f91740 1 Processor -- start 2024-04-16T05:38:14.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faac9f91740 1 -- start start 2024-04-16T05:38:14.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faac9f91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe190e60 0x5623fe191240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faac9f91740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe199e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faac9f91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe19a360 0x5623fe19c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faac9f91740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5623fe0f60e0 con 0x5623fe191780 2024-04-16T05:38:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faac9f91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5623fe093520 con 0x5623fe190e60 2024-04-16T05:38:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.636+0000 7faac9f91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5623fe0f20a0 con 0x5623fe19a360 2024-04-16T05:38:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faab67fc640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faaac0a3d20 0x7faaac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36632/0 (socket says 172.21.15.32:36632) 2024-04-16T05:38:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 -- 172.21.15.32:0/336063930 learned_addr learned my addr 172.21.15.32:0/336063930 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5945640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faaac0a3d20 0x7faaac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faab67fc640 1 -- 172.21.15.32:0/304826681 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7faab00d06a0 con 0x7faac006b020 2024-04-16T05:38:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5945640 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe190e60 0x5623fe191240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac4943640 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe19a360 0x5623fe19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 -- 172.21.15.32:0/336063930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe19a360 msgr2=0x5623fe19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe19a360 0x5623fe19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 -- 172.21.15.32:0/336063930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe190e60 msgr2=0x5623fe191240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe190e60 0x5623fe191240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 -- 172.21.15.32:0/336063930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623fe0bb8d0 con 0x5623fe191780 2024-04-16T05:38:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5144640 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe199e20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7faabc009930 tx=0x7faabc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee9a7ca80678165 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faac5945640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faaac0a3d20 0x7faaac0a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faab8009270 tx=0x7faab8009200 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faaab7fe640 1 -- 172.21.15.32:0/336063930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc02ca80 con 0x5623fe191780 2024-04-16T05:38:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faaab7fe640 1 -- 172.21.15.32:0/336063930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc02cc20 con 0x5623fe191780 2024-04-16T05:38:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.637+0000 7faaab7fe640 1 -- 172.21.15.32:0/336063930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc004920 con 0x5623fe191780 2024-04-16T05:38:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 -- 172.21.15.32:0/336063930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 msgr2=0x5623fe199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe199e20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7faabc009930 tx=0x7faabc005d50 comp rx=0 tx=0).stop 2024-04-16T05:38:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 -- 172.21.15.32:0/336063930 shutdown_connections 2024-04-16T05:38:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe19a360 0x5623fe19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe199e20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 --2- 172.21.15.32:0/336063930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe190e60 0x5623fe191240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 -- 172.21.15.32:0/336063930 >> 172.21.15.32:0/336063930 conn(0x5623fe18b590 msgr2=0x5623fe18b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 -- 172.21.15.32:0/336063930 shutdown_connections 2024-04-16T05:38:14.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 -- 172.21.15.32:0/336063930 wait complete. 2024-04-16T05:38:14.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.638+0000 7faac9f91740 1 -- start start 2024-04-16T05:38:14.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac9f91740 1 Processor -- start 2024-04-16T05:38:14.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac9f91740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe197430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac9f91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe197970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac9f91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac9f91740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5623fe19cfa0 con 0x5623fe190e60 2024-04-16T05:38:14.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac9f91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5623fe0af4d0 con 0x5623fe19a360 2024-04-16T05:38:14.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac9f91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5623fe19d120 con 0x5623fe191780 2024-04-16T05:38:14.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac9f91740 10 client.0 _ll_register_callbacks cb 0x5623fe0942c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:14.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5144640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5144640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36638/0 (socket says 172.21.15.32:36638) 2024-04-16T05:38:14.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5144640 1 -- 172.21.15.32:0/1276826023 learned_addr learned my addr 172.21.15.32:0/1276826023 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5945640 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac4943640 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5144640 1 -- 172.21.15.32:0/1276826023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 msgr2=0x5623fe197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5144640 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5144640 1 -- 172.21.15.32:0/1276826023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 msgr2=0x5623fe197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5144640 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5144640 1 -- 172.21.15.32:0/1276826023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623fde083f0 con 0x5623fe190e60 2024-04-16T05:38:14.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.639+0000 7faac5945640 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac5144640 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe197430 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7faabc02cfd0 tx=0x7faabc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f4b91451740d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faaa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:14.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faaa8ff9640 1 -- 172.21.15.32:0/1276826023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc0035f0 con 0x5623fe190e60 2024-04-16T05:38:14.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faaa8ff9640 1 -- 172.21.15.32:0/1276826023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7faabc003790 con 0x5623fe190e60 2024-04-16T05:38:14.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 1 -- 172.21.15.32:0/1276826023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faabc02cc20 con 0x5623fe190e60 2024-04-16T05:38:14.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faaa8ff9640 1 -- 172.21.15.32:0/1276826023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faabc003990 con 0x5623fe190e60 2024-04-16T05:38:14.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 10 client.16401 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:14.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 1 -- 172.21.15.32:0/1276826023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5623fded27d0 con 0x5623fe190e60 2024-04-16T05:38:14.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faaa8ff9640 1 -- 172.21.15.32:0/1276826023 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc02cc20 con 0x5623fe190e60 2024-04-16T05:38:14.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faaa8ff9640 1 -- 172.21.15.32:0/1276826023 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faabc034c80 con 0x5623fe190e60 2024-04-16T05:38:14.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 10 client.16401 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:14.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 10 client.16401 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:38:14.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 20 client.16401 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:14.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5623fe1aa0d0 0x5623fe1ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 4 client.16401 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.640+0000 7faac9f91740 1 -- 172.21.15.32:0/1276826023 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623fe103b00 con 0x5623fe1aa0d0 2024-04-16T05:38:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.641+0000 7faac4943640 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5623fe1aa0d0 0x5623fe1ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.641+0000 7faac4943640 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5623fe1aa0d0 0x5623fe1ac4c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15acf6d6c64fb709 server_cookie=d95b10636db916b4 in_seq=0 out_seq=0 2024-04-16T05:38:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.641+0000 7faaa8ff9640 10 client.16401 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:38:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faaa8ff9640 1 -- 172.21.15.32:0/1276826023 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623fe103b00 con 0x5623fe1aa0d0 2024-04-16T05:38:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faaa8ff9640 10 client.16401 handle_command_reply: tid=0 2024-04-16T05:38:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 client.16401 shutdown 2024-04-16T05:38:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 -- 172.21.15.32:0/1276826023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 msgr2=0x5623fe197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe197430 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7faabc02cfd0 tx=0x7faabc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 -- 172.21.15.32:0/1276826023 shutdown_connections 2024-04-16T05:38:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5623fe1aa0d0 0x5623fe1ac4c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 --2- 172.21.15.32:0/1276826023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe197430 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faaa8ff9640 0 client.16401 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:38:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 -- 172.21.15.32:0/1276826023 >> 172.21.15.32:0/1276826023 conn(0x5623fe18b590 msgr2=0x5623fe18d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 -- 172.21.15.32:0/1276826023 shutdown_connections 2024-04-16T05:38:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 1 -- 172.21.15.32:0/1276826023 wait complete. 2024-04-16T05:38:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.642+0000 7faac9f91740 20 client.16401 trim_cache size 0 max 16384 2024-04-16T05:38:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.655+0000 7faac9f91740 1 Processor -- start 2024-04-16T05:38:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.655+0000 7faac9f91740 1 -- start start 2024-04-16T05:38:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac9f91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe190e60 0x5623fe1865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac9f91740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe1b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac9f91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe1b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac9f91740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faac01b1ae0 con 0x5623fe191780 2024-04-16T05:38:14.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac9f91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faac01af3b0 con 0x5623fe19a360 2024-04-16T05:38:14.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac9f91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ad200 con 0x5623fe190e60 2024-04-16T05:38:14.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe1b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe1b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36646/0 (socket says 172.21.15.32:36646) 2024-04-16T05:38:14.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 -- 172.21.15.32:0/711622665 learned_addr learned my addr 172.21.15.32:0/711622665 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 -- 172.21.15.32:0/711622665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe190e60 msgr2=0x5623fe1865c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe190e60 0x5623fe1865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac5945640 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe1b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 -- 172.21.15.32:0/711622665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 msgr2=0x5623fe1b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe1b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 -- 172.21.15.32:0/711622665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623fe19d150 con 0x5623fe191780 2024-04-16T05:38:14.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac5945640 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe1b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faac4943640 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe1b1b20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faab0009610 tx=0x7faab00985e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9205a7c51ab4cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faaaaffd640 1 -- 172.21.15.32:0/711622665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00b5070 con 0x5623fe191780 2024-04-16T05:38:14.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faaaaffd640 1 -- 172.21.15.32:0/711622665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab00a06b0 con 0x5623fe191780 2024-04-16T05:38:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.656+0000 7faaaaffd640 1 -- 172.21.15.32:0/711622665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab009c560 con 0x5623fe191780 2024-04-16T05:38:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 -- 172.21.15.32:0/711622665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 msgr2=0x5623fe1b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe1b1b20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faab0009610 tx=0x7faab00985e0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 -- 172.21.15.32:0/711622665 shutdown_connections 2024-04-16T05:38:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe1b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe191780 0x5623fe1b1b20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 --2- 172.21.15.32:0/711622665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe190e60 0x5623fe1865c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 -- 172.21.15.32:0/711622665 >> 172.21.15.32:0/711622665 conn(0x5623fe18b590 msgr2=0x5623fe196460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 -- 172.21.15.32:0/711622665 shutdown_connections 2024-04-16T05:38:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 -- 172.21.15.32:0/711622665 wait complete. 2024-04-16T05:38:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 -- start start 2024-04-16T05:38:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.657+0000 7faac9f91740 1 Processor -- start 2024-04-16T05:38:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac9f91740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe1a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac9f91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac9f91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac9f91740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ad870 con 0x5623fe190e60 2024-04-16T05:38:14.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac9f91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faac01aece0 con 0x5623fe19a360 2024-04-16T05:38:14.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac9f91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faac01adf40 con 0x5623fe191780 2024-04-16T05:38:14.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac9f91740 10 client.0 _ll_register_callbacks cb 0x7faabc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:14.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe1a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe1a28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36652/0 (socket says 172.21.15.32:36652) 2024-04-16T05:38:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 -- 172.21.15.32:0/2298592624 learned_addr learned my addr 172.21.15.32:0/2298592624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5945640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac4943640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 -- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 msgr2=0x5623fe19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 -- 172.21.15.32:0/2298592624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 msgr2=0x5623fe19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 -- 172.21.15.32:0/2298592624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623fe19e9d0 con 0x5623fe190e60 2024-04-16T05:38:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5945640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac4943640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faac5144640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe1a28d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faabc02ce50 tx=0x7faabc02c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34ac5d116daed1b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faaa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.658+0000 7faaa8ff9640 1 -- 172.21.15.32:0/2298592624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc05c030 con 0x5623fe190e60 2024-04-16T05:38:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faaa8ff9640 1 -- 172.21.15.32:0/2298592624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7faabc03d610 con 0x5623fe190e60 2024-04-16T05:38:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 1 -- 172.21.15.32:0/2298592624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab00a06b0 con 0x5623fe190e60 2024-04-16T05:38:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faaa8ff9640 1 -- 172.21.15.32:0/2298592624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faabc003b30 con 0x5623fe190e60 2024-04-16T05:38:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 10 client.16407 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 1 -- 172.21.15.32:0/2298592624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5623fe210390 con 0x5623fe190e60 2024-04-16T05:38:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faaa8ff9640 1 -- 172.21.15.32:0/2298592624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab00a06b0 con 0x5623fe190e60 2024-04-16T05:38:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faaa8ff9640 1 -- 172.21.15.32:0/2298592624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faabc0344d0 con 0x5623fe190e60 2024-04-16T05:38:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 10 client.16407 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 10 client.16407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:38:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 20 client.16407 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5623fe210b90 0x5623fe212fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 4 client.16407 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac9f91740 1 -- 172.21.15.32:0/2298592624 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623fe0fd970 con 0x5623fe210b90 2024-04-16T05:38:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.659+0000 7faac4943640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5623fe210b90 0x5623fe212fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac4943640 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5623fe210b90 0x5623fe212fb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fde0f1977ec24dd server_cookie=bc26d092bff651ae in_seq=0 out_seq=0 2024-04-16T05:38:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faaa8ff9640 10 client.16407 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:38:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faaa8ff9640 1 -- 172.21.15.32:0/2298592624 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623fe0fd970 con 0x5623fe210b90 2024-04-16T05:38:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faaa8ff9640 10 client.16407 handle_command_reply: tid=0 2024-04-16T05:38:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 client.16407 shutdown 2024-04-16T05:38:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 -- 172.21.15.32:0/2298592624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 msgr2=0x5623fe1a28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe1a28d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faabc02ce50 tx=0x7faabc02c360 comp rx=0 tx=0).stop 2024-04-16T05:38:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 -- 172.21.15.32:0/2298592624 shutdown_connections 2024-04-16T05:38:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5623fe210b90 0x5623fe212fb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5623fe19a360 0x5623fe19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623fe191780 0x5623fe19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faaa8ff9640 0 client.16407 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:38:14.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 --2- 172.21.15.32:0/2298592624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623fe190e60 0x5623fe1a28d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 -- 172.21.15.32:0/2298592624 >> 172.21.15.32:0/2298592624 conn(0x5623fe18b590 msgr2=0x5623fe18a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 -- 172.21.15.32:0/2298592624 shutdown_connections 2024-04-16T05:38:14.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 1 -- 172.21.15.32:0/2298592624 wait complete. 2024-04-16T05:38:14.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.660+0000 7faac9f91740 20 client.16407 trim_cache size 0 max 16384 2024-04-16T05:38:14.683 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:38:14.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 -- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faaac0a3d20 msgr2=0x7faaac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faaac0a3d20 0x7faaac0a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faab8009270 tx=0x7faab8009200 comp rx=0 tx=0).stop 2024-04-16T05:38:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 -- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac006b020 msgr2=0x7faac019e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac006b020 0x7faac019e9a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faab000d950 tx=0x7faab000de10 comp rx=0 tx=0).stop 2024-04-16T05:38:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 -- 172.21.15.32:0/304826681 shutdown_connections 2024-04-16T05:38:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faac0108110 0x7faac0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7faaac0a3d20 0x7faaac0a61e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac006b020 0x7faac019e9a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 --2- 172.21.15.32:0/304826681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac006a660 0x7faac019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 -- 172.21.15.32:0/304826681 >> 172.21.15.32:0/304826681 conn(0x7faac0077cf0 msgr2=0x7faac00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 -- 172.21.15.32:0/304826681 shutdown_connections 2024-04-16T05:38:14.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.661+0000 7faac73a8640 1 -- 172.21.15.32:0/304826681 wait complete. 2024-04-16T05:38:14.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:38:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 -- 172.21.15.32:0/2354779809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850107380 msgr2=0x7f785010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 --2- 172.21.15.32:0/2354779809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850107380 0x7f785010ef00 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7848009ca0 tx=0x7f784802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 -- 172.21.15.32:0/2354779809 shutdown_connections 2024-04-16T05:38:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 --2- 172.21.15.32:0/2354779809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850107380 0x7f785010ef00 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 --2- 172.21.15.32:0/2354779809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7850101120 0x7f7850106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 --2- 172.21.15.32:0/2354779809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7850100770 0x7f7850100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 -- 172.21.15.32:0/2354779809 >> 172.21.15.32:0/2354779809 conn(0x7f7850077430 msgr2=0x7f78500fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 -- 172.21.15.32:0/2354779809 shutdown_connections 2024-04-16T05:38:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.796+0000 7f7857538640 1 -- 172.21.15.32:0/2354779809 wait complete. 2024-04-16T05:38:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7857538640 1 Processor -- start 2024-04-16T05:38:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7857538640 1 -- start start 2024-04-16T05:38:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7857538640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7850100770 0x7f78501016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7857538640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7850107380 0x7f7850101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7857538640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850102160 0x7f78501a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7857538640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78501123d0 con 0x7f7850102160 2024-04-16T05:38:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7857538640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7850112250 con 0x7f7850100770 2024-04-16T05:38:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7857538640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7850112550 con 0x7f7850107380 2024-04-16T05:38:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7855ad5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850102160 0x7f78501a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7855ad5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850102160 0x7f78501a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36682/0 (socket says 172.21.15.32:36682) 2024-04-16T05:38:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7855ad5640 1 -- 172.21.15.32:0/2074781160 learned_addr learned my addr 172.21.15.32:0/2074781160 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.797+0000 7f7854ad3640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7850107380 0x7f7850101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7855ad5640 1 -- 172.21.15.32:0/2074781160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7850107380 msgr2=0x7f7850101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7855ad5640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7850107380 0x7f7850101c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7855ad5640 1 -- 172.21.15.32:0/2074781160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7850100770 msgr2=0x7f78501016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f78552d4640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7850100770 0x7f78501016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7855ad5640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7850100770 0x7f78501016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7855ad5640 1 -- 172.21.15.32:0/2074781160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78501a7120 con 0x7f7850102160 2024-04-16T05:38:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7854ad3640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7850107380 0x7f7850101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7855ad5640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850102160 0x7f78501a6a20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f784802c700 tx=0x7f7848032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f78467fc640 1 -- 172.21.15.32:0/2074781160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848039040 con 0x7f7850102160 2024-04-16T05:38:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f78467fc640 1 -- 172.21.15.32:0/2074781160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7848044070 con 0x7f7850102160 2024-04-16T05:38:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7857538640 1 -- 172.21.15.32:0/2074781160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78501a73b0 con 0x7f7850102160 2024-04-16T05:38:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f78467fc640 1 -- 172.21.15.32:0/2074781160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78480042a0 con 0x7f7850102160 2024-04-16T05:38:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.798+0000 7f7857538640 1 -- 172.21.15.32:0/2074781160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78501a78f0 con 0x7f7850102160 2024-04-16T05:38:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.800+0000 7f78467fc640 1 -- 172.21.15.32:0/2074781160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7848004440 con 0x7f7850102160 2024-04-16T05:38:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f785a1f5740 1 Processor -- start 2024-04-16T05:38:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f785a1f5740 1 -- start start 2024-04-16T05:38:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e99431240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 0x556e99439e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9943c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f785a1f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556e993960e0 con 0x556e9943a360 2024-04-16T05:38:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f785a1f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556e99333520 con 0x556e99430e60 2024-04-16T05:38:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f785a1f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e993920a0 con 0x556e99431780 2024-04-16T05:38:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f78467fc640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f783c0a3dd0 0x7f783c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9943c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7855ad5640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f783c0a3dd0 0x7f783c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9943c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36684/0 (socket says 172.21.15.32:36684) 2024-04-16T05:38:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 -- 172.21.15.32:0/3445011115 learned_addr learned my addr 172.21.15.32:0/3445011115 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e99431240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f78552d4640 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 0x556e99439e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f78467fc640 1 -- 172.21.15.32:0/2074781160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f78480f2e00 con 0x7f7850102160 2024-04-16T05:38:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 -- 172.21.15.32:0/3445011115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 msgr2=0x556e99439e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 0x556e99439e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 -- 172.21.15.32:0/3445011115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 msgr2=0x556e99431240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e99431240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.801+0000 7f7854ad3640 1 -- 172.21.15.32:0/3445011115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e9935b8d0 con 0x556e9943a360 2024-04-16T05:38:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.802+0000 7f7855ad5640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f783c0a3dd0 0x7f783c0a6290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78480f03d0 tx=0x7f78480df040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.802+0000 7f7854ad3640 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9943c750 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7840014900 tx=0x7f784000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1597757ec1b8a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.802+0000 7f783b7fe640 1 -- 172.21.15.32:0/3445011115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840004030 con 0x556e9943a360 2024-04-16T05:38:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.802+0000 7f783b7fe640 1 -- 172.21.15.32:0/3445011115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7840005500 con 0x556e9943a360 2024-04-16T05:38:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.802+0000 7f783b7fe640 1 -- 172.21.15.32:0/3445011115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840004a70 con 0x556e9943a360 2024-04-16T05:38:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.802+0000 7f785a1f5740 1 -- 172.21.15.32:0/3445011115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 msgr2=0x556e9943c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.802+0000 7f785a1f5740 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9943c750 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7840014900 tx=0x7f784000b420 comp rx=0 tx=0).stop 2024-04-16T05:38:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 -- 172.21.15.32:0/3445011115 shutdown_connections 2024-04-16T05:38:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9943c750 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 0x556e99439e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 --2- 172.21.15.32:0/3445011115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e99431240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 -- 172.21.15.32:0/3445011115 >> 172.21.15.32:0/3445011115 conn(0x556e9942b590 msgr2=0x556e9942b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 -- 172.21.15.32:0/3445011115 shutdown_connections 2024-04-16T05:38:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 -- 172.21.15.32:0/3445011115 wait complete. 2024-04-16T05:38:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 -- start start 2024-04-16T05:38:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 Processor -- start 2024-04-16T05:38:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e99437490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 0x556e994379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e99437f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556e9943d000 con 0x556e9943a360 2024-04-16T05:38:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556e9934f4d0 con 0x556e99430e60 2024-04-16T05:38:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e9943d180 con 0x556e99431780 2024-04-16T05:38:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f785a1f5740 10 client.0 _ll_register_callbacks cb 0x556e993342c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.803+0000 7f7854ad3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e99437f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e99437f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36688/0 (socket says 172.21.15.32:36688) 2024-04-16T05:38:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 -- 172.21.15.32:0/1057902798 learned_addr learned my addr 172.21.15.32:0/1057902798 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f78552d4640 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 0x556e994379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e99437490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 -- 172.21.15.32:0/1057902798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 msgr2=0x556e994379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 0x556e994379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 -- 172.21.15.32:0/1057902798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 msgr2=0x556e99437490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e99437490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 -- 172.21.15.32:0/1057902798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e990a83f0 con 0x556e9943a360 2024-04-16T05:38:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7854ad3640 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e99437f10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f784000be10 tx=0x7f7840004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a453b4f2d5cd1b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:14.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7838ff9640 1 -- 172.21.15.32:0/1057902798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840023020 con 0x556e9943a360 2024-04-16T05:38:14.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7838ff9640 1 -- 172.21.15.32:0/1057902798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7840014110 con 0x556e9943a360 2024-04-16T05:38:14.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f785a1f5740 1 -- 172.21.15.32:0/1057902798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7840005500 con 0x556e9943a360 2024-04-16T05:38:14.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f7838ff9640 1 -- 172.21.15.32:0/1057902798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f784000f760 con 0x556e9943a360 2024-04-16T05:38:14.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f785a1f5740 10 client.16419 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:14.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.804+0000 7f785a1f5740 1 -- 172.21.15.32:0/1057902798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556e991727d0 con 0x556e9943a360 2024-04-16T05:38:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f7838ff9640 1 -- 172.21.15.32:0/1057902798 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7840005500 con 0x556e9943a360 2024-04-16T05:38:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f7838ff9640 1 -- 172.21.15.32:0/1057902798 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f784001f810 con 0x556e9943a360 2024-04-16T05:38:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f785a1f5740 10 client.16419 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f785a1f5740 10 client.16419 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:38:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f785a1f5740 20 client.16419 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x556e9944a3c0 0x556e9944c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f785a1f5740 4 client.16419 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f785a1f5740 1 -- 172.21.15.32:0/1057902798 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e993a3b00 con 0x556e9944a3c0 2024-04-16T05:38:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.805+0000 7f78552d4640 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x556e9944a3c0 0x556e9944c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f7838ff9640 10 client.16419 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:38:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f78552d4640 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x556e9944a3c0 0x556e9944c7b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55cc7e1df1a72d47 server_cookie=618103cd343eac84 in_seq=0 out_seq=0 2024-04-16T05:38:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f7838ff9640 1 -- 172.21.15.32:0/1057902798 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e993a3b00 con 0x556e9944a3c0 2024-04-16T05:38:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f7838ff9640 10 client.16419 handle_command_reply: tid=0 2024-04-16T05:38:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 client.16419 shutdown 2024-04-16T05:38:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 -- 172.21.15.32:0/1057902798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 msgr2=0x556e99437f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e99437f10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f784000be10 tx=0x7f7840004480 comp rx=0 tx=0).stop 2024-04-16T05:38:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 -- 172.21.15.32:0/1057902798 shutdown_connections 2024-04-16T05:38:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x556e9944a3c0 0x556e9944c7b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e99437f10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f7838ff9640 0 client.16419 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:38:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99431780 0x556e994379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1057902798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e99437490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.806+0000 7f785a1f5740 1 -- 172.21.15.32:0/1057902798 >> 172.21.15.32:0/1057902798 conn(0x556e9942b590 msgr2=0x556e9942d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.807+0000 7f785a1f5740 1 -- 172.21.15.32:0/1057902798 shutdown_connections 2024-04-16T05:38:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.807+0000 7f785a1f5740 1 -- 172.21.15.32:0/1057902798 wait complete. 2024-04-16T05:38:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.807+0000 7f785a1f5740 20 client.16419 trim_cache size 0 max 16384 2024-04-16T05:38:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f785a1f5740 1 Processor -- start 2024-04-16T05:38:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f785a1f5740 1 -- start start 2024-04-16T05:38:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99430e60 0x556e99439bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99431780 0x556e99446010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9942a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f785a1f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78501b3db0 con 0x556e9943a360 2024-04-16T05:38:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f785a1f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f78501b1680 con 0x556e99431780 2024-04-16T05:38:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f785a1f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78501af4d0 con 0x556e99430e60 2024-04-16T05:38:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f7854ad3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9942a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f7854ad3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9942a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36700/0 (socket says 172.21.15.32:36700) 2024-04-16T05:38:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.820+0000 7f7854ad3640 1 -- 172.21.15.32:0/1024839144 learned_addr learned my addr 172.21.15.32:0/1024839144 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f7854ad3640 1 -- 172.21.15.32:0/1024839144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99430e60 msgr2=0x556e99439bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f7854ad3640 1 --2- 172.21.15.32:0/1024839144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99430e60 0x556e99439bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f7854ad3640 1 -- 172.21.15.32:0/1024839144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99431780 msgr2=0x556e99446010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f7854ad3640 1 --2- 172.21.15.32:0/1024839144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99431780 0x556e99446010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f7854ad3640 1 -- 172.21.15.32:0/1024839144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e9942b0c0 con 0x556e9943a360 2024-04-16T05:38:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f7854ad3640 1 --2- 172.21.15.32:0/1024839144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9942a5b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7840004580 tx=0x7f7840005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db78ff385b972172 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f783affd640 1 -- 172.21.15.32:0/1024839144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840023040 con 0x556e9943a360 2024-04-16T05:38:14.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f783affd640 1 -- 172.21.15.32:0/1024839144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784001fe50 con 0x556e9943a360 2024-04-16T05:38:14.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f783affd640 1 -- 172.21.15.32:0/1024839144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784001f470 con 0x556e9943a360 2024-04-16T05:38:14.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f785a1f5740 1 -- 172.21.15.32:0/1024839144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 msgr2=0x556e9942a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1024839144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9942a5b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7840004580 tx=0x7f7840005780 comp rx=0 tx=0).stop 2024-04-16T05:38:14.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f785a1f5740 1 -- 172.21.15.32:0/1024839144 shutdown_connections 2024-04-16T05:38:14.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1024839144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e9943a360 0x556e9942a5b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1024839144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99431780 0x556e99446010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f785a1f5740 1 --2- 172.21.15.32:0/1024839144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e99430e60 0x556e99439bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.821+0000 7f785a1f5740 1 -- 172.21.15.32:0/1024839144 >> 172.21.15.32:0/1024839144 conn(0x556e9942b590 msgr2=0x556e99429ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 -- 172.21.15.32:0/1024839144 shutdown_connections 2024-04-16T05:38:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 -- 172.21.15.32:0/1024839144 wait complete. 2024-04-16T05:38:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 -- start start 2024-04-16T05:38:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 Processor -- start 2024-04-16T05:38:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e9943d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e99431780 0x556e9943dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e9943a360 0x556e994ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78501afb40 con 0x556e99431780 2024-04-16T05:38:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f78501b0fb0 con 0x556e99430e60 2024-04-16T05:38:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78501b0210 con 0x556e9943a360 2024-04-16T05:38:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f785a1f5740 10 client.0 _ll_register_callbacks cb 0x7f784000be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f78552d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e99431780 0x556e9943dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f78552d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e99431780 0x556e9943dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36716/0 (socket says 172.21.15.32:36716) 2024-04-16T05:38:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.822+0000 7f78552d4640 1 -- 172.21.15.32:0/2242292555 learned_addr learned my addr 172.21.15.32:0/2242292555 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7854ad3640 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e9943d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7854ad3640 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e9943a360 0x556e994ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f78552d4640 1 -- 172.21.15.32:0/2242292555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e9943a360 msgr2=0x556e994ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f78552d4640 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e9943a360 0x556e994ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f78552d4640 1 -- 172.21.15.32:0/2242292555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 msgr2=0x556e9943d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f78552d4640 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e9943d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f78552d4640 1 -- 172.21.15.32:0/2242292555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e994aec90 con 0x556e99431780 2024-04-16T05:38:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7854ad3640 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e9943a360 0x556e994ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f78552d4640 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e99431780 0x556e9943dc50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x556e9944cde0 tx=0x7f784c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b31ccfcf868206 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7838ff9640 1 -- 172.21.15.32:0/2242292555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c01e250 con 0x556e99431780 2024-04-16T05:38:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7838ff9640 1 -- 172.21.15.32:0/2242292555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f784c01e3f0 con 0x556e99431780 2024-04-16T05:38:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f785a1f5740 1 -- 172.21.15.32:0/2242292555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f784001fe50 con 0x556e99431780 2024-04-16T05:38:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7838ff9640 1 -- 172.21.15.32:0/2242292555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f784c02e050 con 0x556e99431780 2024-04-16T05:38:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f785a1f5740 10 client.16425 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f785a1f5740 1 -- 172.21.15.32:0/2242292555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556e994b0000 con 0x556e99431780 2024-04-16T05:38:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7838ff9640 1 -- 172.21.15.32:0/2242292555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784001fe50 con 0x556e99431780 2024-04-16T05:38:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f7838ff9640 1 -- 172.21.15.32:0/2242292555 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f784c00f040 con 0x556e99431780 2024-04-16T05:38:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f785a1f5740 10 client.16425 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f785a1f5740 10 client.16425 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:38:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.823+0000 7f785a1f5740 20 client.16425 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.824+0000 7f785a1f5740 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x556e994b0760 0x556e994b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.824+0000 7f785a1f5740 4 client.16425 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.824+0000 7f785a1f5740 1 -- 172.21.15.32:0/2242292555 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e9939d970 con 0x556e994b0760 2024-04-16T05:38:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.824+0000 7f7854ad3640 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x556e994b0760 0x556e994b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.824+0000 7f7854ad3640 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x556e994b0760 0x556e994b2b80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fd9dbe8f08397bd server_cookie=c9e72d0c28e3890f in_seq=0 out_seq=0 2024-04-16T05:38:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.824+0000 7f7838ff9640 10 client.16425 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:38:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7838ff9640 1 -- 172.21.15.32:0/2242292555 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e9939d970 con 0x556e994b0760 2024-04-16T05:38:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7838ff9640 10 client.16425 handle_command_reply: tid=0 2024-04-16T05:38:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 client.16425 shutdown 2024-04-16T05:38:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 -- 172.21.15.32:0/2242292555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e99431780 msgr2=0x556e9943dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e99431780 0x556e9943dc50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x556e9944cde0 tx=0x7f784c014840 comp rx=0 tx=0).stop 2024-04-16T05:38:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 -- 172.21.15.32:0/2242292555 shutdown_connections 2024-04-16T05:38:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x556e994b0760 0x556e994b2b80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e9943a360 0x556e994ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e99431780 0x556e9943dc50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 --2- 172.21.15.32:0/2242292555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e99430e60 0x556e9943d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7838ff9640 0 client.16425 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:38:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 -- 172.21.15.32:0/2242292555 >> 172.21.15.32:0/2242292555 conn(0x556e9942b590 msgr2=0x556e9942a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 -- 172.21.15.32:0/2242292555 shutdown_connections 2024-04-16T05:38:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 1 -- 172.21.15.32:0/2242292555 wait complete. 2024-04-16T05:38:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f785a1f5740 20 client.16425 trim_cache size 0 max 16384 2024-04-16T05:38:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:38:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 -- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f783c0a3dd0 msgr2=0x7f783c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f783c0a3dd0 0x7f783c0a6290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78480f03d0 tx=0x7f78480df040 comp rx=0 tx=0).stop 2024-04-16T05:38:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 -- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850102160 msgr2=0x7f78501a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850102160 0x7f78501a6a20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f784802c700 tx=0x7f7848032f20 comp rx=0 tx=0).stop 2024-04-16T05:38:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 -- 172.21.15.32:0/2074781160 shutdown_connections 2024-04-16T05:38:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f783c0a3dd0 0x7f783c0a6290 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7850102160 0x7f78501a6a20 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7850107380 0x7f7850101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 --2- 172.21.15.32:0/2074781160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7850100770 0x7f78501016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.825+0000 7f7857538640 1 -- 172.21.15.32:0/2074781160 >> 172.21.15.32:0/2074781160 conn(0x7f7850077430 msgr2=0x7f78500fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.826+0000 7f7857538640 1 -- 172.21.15.32:0/2074781160 shutdown_connections 2024-04-16T05:38:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.826+0000 7f7857538640 1 -- 172.21.15.32:0/2074781160 wait complete. 2024-04-16T05:38:14.845 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:38:14.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 -- 172.21.15.32:0/1531667685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138102250 msgr2=0x7f6138102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 --2- 172.21.15.32:0/1531667685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138102250 0x7f6138102630 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6128009900 tx=0x7f612802bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:14.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 -- 172.21.15.32:0/1531667685 shutdown_connections 2024-04-16T05:38:14.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 --2- 172.21.15.32:0/1531667685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6138107320 0x7f613810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 --2- 172.21.15.32:0/1531667685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6138102c00 0x7f6138106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 --2- 172.21.15.32:0/1531667685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138102250 0x7f6138102630 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 -- 172.21.15.32:0/1531667685 >> 172.21.15.32:0/1531667685 conn(0x7f6138079490 msgr2=0x7f61380798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 -- 172.21.15.32:0/1531667685 shutdown_connections 2024-04-16T05:38:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.955+0000 7f613c904640 1 -- 172.21.15.32:0/1531667685 wait complete. 2024-04-16T05:38:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613c904640 1 Processor -- start 2024-04-16T05:38:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613c904640 1 -- start start 2024-04-16T05:38:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613c904640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6138102250 0x7f6138074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613c904640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6138107320 0x7f6138074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613c904640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138074d80 0x7f61381a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613c904640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61381123e0 con 0x7f6138074d80 2024-04-16T05:38:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613c904640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6138112260 con 0x7f6138102250 2024-04-16T05:38:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613c904640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6138112560 con 0x7f6138107320 2024-04-16T05:38:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f6136d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138074d80 0x7f61381a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f6136d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138074d80 0x7f61381a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36736/0 (socket says 172.21.15.32:36736) 2024-04-16T05:38:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f6136d9d640 1 -- 172.21.15.32:0/758769756 learned_addr learned my addr 172.21.15.32:0/758769756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f612ffff640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6138107320 0x7f6138074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f6136d9d640 1 -- 172.21.15.32:0/758769756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6138107320 msgr2=0x7f6138074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f6136d9d640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6138107320 0x7f6138074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f6136d9d640 1 -- 172.21.15.32:0/758769756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6138102250 msgr2=0x7f6138074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f613659c640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6138102250 0x7f6138074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.956+0000 7f6136d9d640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6138102250 0x7f6138074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f6136d9d640 1 -- 172.21.15.32:0/758769756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61381a7150 con 0x7f6138074d80 2024-04-16T05:38:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f612ffff640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6138107320 0x7f6138074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f613659c640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6138102250 0x7f6138074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f6136d9d640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138074d80 0x7f61381a6ab0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f613000eaa0 tx=0x7f613000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f612f7fe640 1 -- 172.21.15.32:0/758769756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613000cdb0 con 0x7f6138074d80 2024-04-16T05:38:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f612f7fe640 1 -- 172.21.15.32:0/758769756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6130005020 con 0x7f6138074d80 2024-04-16T05:38:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f613c904640 1 -- 172.21.15.32:0/758769756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61381a7440 con 0x7f6138074d80 2024-04-16T05:38:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f612f7fe640 1 -- 172.21.15.32:0/758769756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61300116b0 con 0x7f6138074d80 2024-04-16T05:38:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.957+0000 7f613c904640 1 -- 172.21.15.32:0/758769756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61381a7990 con 0x7f6138074d80 2024-04-16T05:38:14.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.959+0000 7f612f7fe640 1 -- 172.21.15.32:0/758769756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6130005560 con 0x7f6138074d80 2024-04-16T05:38:14.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.959+0000 7f613f5c8740 1 Processor -- start 2024-04-16T05:38:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613f5c8740 1 -- start start 2024-04-16T05:38:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790ba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790c5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613f5c8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55db7901f0e0 con 0x55db790b9e60 2024-04-16T05:38:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613f5c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55db78fbc520 con 0x55db790ba780 2024-04-16T05:38:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613f5c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55db7901b0a0 con 0x55db790c3360 2024-04-16T05:38:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612f7fe640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f61240a3d20 0x7f61240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790ba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790ba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36742/0 (socket says 172.21.15.32:36742) 2024-04-16T05:38:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 -- 172.21.15.32:0/3246150936 learned_addr learned my addr 172.21.15.32:0/3246150936 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f6136d9d640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f61240a3d20 0x7f61240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613659c640 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612f7fe640 1 -- 172.21.15.32:0/758769756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f61300d0da0 con 0x7f6138074d80 2024-04-16T05:38:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 -- 172.21.15.32:0/3246150936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 msgr2=0x55db790c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 -- 172.21.15.32:0/3246150936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 msgr2=0x55db790c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f612ffff640 1 -- 172.21.15.32:0/3246150936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db78fe48d0 con 0x55db790b9e60 2024-04-16T05:38:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.960+0000 7f613659c640 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.961+0000 7f612ffff640 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790ba240 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6120014490 tx=0x7f61200096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ea7d27e7d3ffbf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.961+0000 7f6136d9d640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f61240a3d20 0x7f61240a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6130098e60 tx=0x7f6130087040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.961+0000 7f612cff9640 1 -- 172.21.15.32:0/3246150936 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612001a030 con 0x55db790b9e60 2024-04-16T05:38:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.961+0000 7f612cff9640 1 -- 172.21.15.32:0/3246150936 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6120003610 con 0x55db790b9e60 2024-04-16T05:38:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.961+0000 7f612cff9640 1 -- 172.21.15.32:0/3246150936 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120012650 con 0x55db790b9e60 2024-04-16T05:38:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.961+0000 7f613f5c8740 1 -- 172.21.15.32:0/3246150936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 msgr2=0x55db790ba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.961+0000 7f613f5c8740 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790ba240 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6120014490 tx=0x7f61200096a0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 -- 172.21.15.32:0/3246150936 shutdown_connections 2024-04-16T05:38:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 --2- 172.21.15.32:0/3246150936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790ba240 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 -- 172.21.15.32:0/3246150936 >> 172.21.15.32:0/3246150936 conn(0x55db790b4590 msgr2=0x55db790b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 -- 172.21.15.32:0/3246150936 shutdown_connections 2024-04-16T05:38:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 -- 172.21.15.32:0/3246150936 wait complete. 2024-04-16T05:38:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 -- start start 2024-04-16T05:38:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 Processor -- start 2024-04-16T05:38:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55db790c6000 con 0x55db790b9e60 2024-04-16T05:38:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55db78fd84d0 con 0x55db790ba780 2024-04-16T05:38:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55db790c6180 con 0x55db790c3360 2024-04-16T05:38:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f613f5c8740 10 client.0 _ll_register_callbacks cb 0x55db78fbd2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f612ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f612ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790c0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36756/0 (socket says 172.21.15.32:36756) 2024-04-16T05:38:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.962+0000 7f612ffff640 1 -- 172.21.15.32:0/2864604600 learned_addr learned my addr 172.21.15.32:0/2864604600 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f612ffff640 1 -- 172.21.15.32:0/2864604600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 msgr2=0x55db790c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f613659c640 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f612ffff640 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f612ffff640 1 -- 172.21.15.32:0/2864604600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 msgr2=0x55db790c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f612ffff640 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f612ffff640 1 -- 172.21.15.32:0/2864604600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db78d313f0 con 0x55db790b9e60 2024-04-16T05:38:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f612ffff640 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790c0320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6120009880 tx=0x7f6120005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff5d7e1ca89d23c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f611dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f611dffb640 1 -- 172.21.15.32:0/2864604600 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612001a070 con 0x55db790b9e60 2024-04-16T05:38:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f611dffb640 1 -- 172.21.15.32:0/2864604600 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f612001f660 con 0x55db790b9e60 2024-04-16T05:38:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f613f5c8740 1 -- 172.21.15.32:0/2864604600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6120003610 con 0x55db790b9e60 2024-04-16T05:38:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f611dffb640 1 -- 172.21.15.32:0/2864604600 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f612001f860 con 0x55db790b9e60 2024-04-16T05:38:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f613f5c8740 10 client.16437 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f613f5c8740 1 -- 172.21.15.32:0/2864604600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db78dfb7d0 con 0x55db790b9e60 2024-04-16T05:38:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.963+0000 7f611dffb640 1 -- 172.21.15.32:0/2864604600 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6120003610 con 0x55db790b9e60 2024-04-16T05:38:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f611dffb640 1 -- 172.21.15.32:0/2864604600 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6120027070 con 0x55db790b9e60 2024-04-16T05:38:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f613f5c8740 10 client.16437 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f613f5c8740 10 client.16437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f613f5c8740 20 client.16437 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f613f5c8740 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55db790d3230 0x55db790d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f613f5c8740 4 client.16437 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f613f5c8740 1 -- 172.21.15.32:0/2864604600 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db7902cb00 con 0x55db790d3230 2024-04-16T05:38:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f613659c640 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55db790d3230 0x55db790d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f611dffb640 10 client.16437 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.964+0000 7f613659c640 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55db790d3230 0x55db790d5620 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf11d309da27f012 server_cookie=6ced50c6ecf0e680 in_seq=0 out_seq=0 2024-04-16T05:38:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f611dffb640 1 -- 172.21.15.32:0/2864604600 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db7902cb00 con 0x55db790d3230 2024-04-16T05:38:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f611dffb640 10 client.16437 handle_command_reply: tid=0 2024-04-16T05:38:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 client.16437 shutdown 2024-04-16T05:38:14.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 -- 172.21.15.32:0/2864604600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 msgr2=0x55db790c0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790c0320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6120009880 tx=0x7f6120005740 comp rx=0 tx=0).stop 2024-04-16T05:38:14.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 -- 172.21.15.32:0/2864604600 shutdown_connections 2024-04-16T05:38:14.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55db790d3230 0x55db790d5620 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f611dffb640 0 client.16437 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:14.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 --2- 172.21.15.32:0/2864604600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790c0320 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 -- 172.21.15.32:0/2864604600 >> 172.21.15.32:0/2864604600 conn(0x55db790b4590 msgr2=0x55db790b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 -- 172.21.15.32:0/2864604600 shutdown_connections 2024-04-16T05:38:14.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 1 -- 172.21.15.32:0/2864604600 wait complete. 2024-04-16T05:38:14.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.965+0000 7f613f5c8740 20 client.16437 trim_cache size 0 max 16384 2024-04-16T05:38:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613f5c8740 1 Processor -- start 2024-04-16T05:38:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613f5c8740 1 -- start start 2024-04-16T05:38:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613f5c8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61381b3ee0 con 0x55db790b9e60 2024-04-16T05:38:14.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613f5c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f61381b17b0 con 0x55db790ba780 2024-04-16T05:38:14.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613f5c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61381af600 con 0x55db790c3360 2024-04-16T05:38:14.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36772/0 (socket says 172.21.15.32:36772) 2024-04-16T05:38:14.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 -- 172.21.15.32:0/3124882967 learned_addr learned my addr 172.21.15.32:0/3124882967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 -- 172.21.15.32:0/3124882967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 msgr2=0x55db790bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 --2- 172.21.15.32:0/3124882967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 -- 172.21.15.32:0/3124882967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 msgr2=0x55db790cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f613659c640 1 --2- 172.21.15.32:0/3124882967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 --2- 172.21.15.32:0/3124882967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 -- 172.21.15.32:0/3124882967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db790d5be0 con 0x55db790b9e60 2024-04-16T05:38:14.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f612ffff640 1 --2- 172.21.15.32:0/3124882967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790b3970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6120005e00 tx=0x7f6120003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fdce145bd4853c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.979+0000 7f611ffff640 1 -- 172.21.15.32:0/3124882967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612001a070 con 0x55db790b9e60 2024-04-16T05:38:14.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f611ffff640 1 -- 172.21.15.32:0/3124882967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612001b040 con 0x55db790b9e60 2024-04-16T05:38:14.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f611ffff640 1 -- 172.21.15.32:0/3124882967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120009530 con 0x55db790b9e60 2024-04-16T05:38:14.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 -- 172.21.15.32:0/3124882967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 msgr2=0x55db790b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:14.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 --2- 172.21.15.32:0/3124882967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790b3970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6120005e00 tx=0x7f6120003b80 comp rx=0 tx=0).stop 2024-04-16T05:38:14.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 -- 172.21.15.32:0/3124882967 shutdown_connections 2024-04-16T05:38:14.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 --2- 172.21.15.32:0/3124882967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790c3360 0x55db790bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 --2- 172.21.15.32:0/3124882967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 --2- 172.21.15.32:0/3124882967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790b9e60 0x55db790b3970 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 -- 172.21.15.32:0/3124882967 >> 172.21.15.32:0/3124882967 conn(0x55db790b4590 msgr2=0x55db790c0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:14.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 -- 172.21.15.32:0/3124882967 shutdown_connections 2024-04-16T05:38:14.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 -- 172.21.15.32:0/3124882967 wait complete. 2024-04-16T05:38:14.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 -- start start 2024-04-16T05:38:14.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.980+0000 7f613f5c8740 1 Processor -- start 2024-04-16T05:38:14.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790b9e60 0x55db790c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613f5c8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790c3360 0x55db791316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:14.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613f5c8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61381afc70 con 0x55db790c3360 2024-04-16T05:38:14.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613f5c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f61381b10e0 con 0x55db790ba780 2024-04-16T05:38:14.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613f5c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61381b0340 con 0x55db790b9e60 2024-04-16T05:38:14.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613f5c8740 10 client.0 _ll_register_callbacks cb 0x7f6120009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:14.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790c3360 0x55db791316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790c3360 0x55db791316b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36776/0 (socket says 172.21.15.32:36776) 2024-04-16T05:38:14.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 -- 172.21.15.32:0/1793128276 learned_addr learned my addr 172.21.15.32:0/1793128276 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:14.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613659c640 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790b9e60 0x55db790c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:14.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 -- 172.21.15.32:0/1793128276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790b9e60 msgr2=0x55db790c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790b9e60 0x55db790c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 -- 172.21.15.32:0/1793128276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 msgr2=0x55db790c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:14.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:14.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 -- 172.21.15.32:0/1793128276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db79131cf0 con 0x55db790c3360 2024-04-16T05:38:14.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f612ffff640 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790c3360 0x55db791316b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6120012d80 tx=0x7f6120022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec914a12bb1be23 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:14.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f611dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:14.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f611dffb640 1 -- 172.21.15.32:0/1793128276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612001a070 con 0x55db790c3360 2024-04-16T05:38:14.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f611dffb640 1 -- 172.21.15.32:0/1793128276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f61200095f0 con 0x55db790c3360 2024-04-16T05:38:14.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.981+0000 7f613f5c8740 1 -- 172.21.15.32:0/1793128276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f612001b040 con 0x55db790c3360 2024-04-16T05:38:14.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f611dffb640 1 -- 172.21.15.32:0/1793128276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f612001f460 con 0x55db790c3360 2024-04-16T05:38:14.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613f5c8740 10 client.16443 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:14.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613f5c8740 1 -- 172.21.15.32:0/1793128276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db791390a0 con 0x55db790c3360 2024-04-16T05:38:15.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f611dffb640 1 -- 172.21.15.32:0/1793128276 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612001b040 con 0x55db790c3360 2024-04-16T05:38:15.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f611dffb640 1 -- 172.21.15.32:0/1793128276 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6120027040 con 0x55db790c3360 2024-04-16T05:38:15.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613f5c8740 10 client.16443 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:15.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613f5c8740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:15.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613f5c8740 20 client.16443 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:15.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613f5c8740 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55db79139700 0x55db7913baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:15.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613f5c8740 4 client.16443 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:15.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613f5c8740 1 -- 172.21.15.32:0/1793128276 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db79026970 con 0x55db79139700 2024-04-16T05:38:15.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.982+0000 7f613659c640 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55db79139700 0x55db7913baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:15.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613659c640 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55db79139700 0x55db7913baf0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d0f0854d7a312c server_cookie=fe34083c1b452de0 in_seq=0 out_seq=0 2024-04-16T05:38:15.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f611dffb640 10 client.16443 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:15.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f611dffb640 1 -- 172.21.15.32:0/1793128276 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db79026970 con 0x55db79139700 2024-04-16T05:38:15.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f611dffb640 10 client.16443 handle_command_reply: tid=0 2024-04-16T05:38:15.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 client.16443 shutdown 2024-04-16T05:38:15.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 -- 172.21.15.32:0/1793128276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790c3360 msgr2=0x55db791316b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:15.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790c3360 0x55db791316b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6120012d80 tx=0x7f6120022040 comp rx=0 tx=0).stop 2024-04-16T05:38:15.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 -- 172.21.15.32:0/1793128276 shutdown_connections 2024-04-16T05:38:15.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55db790c3360 0x55db791316b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:15.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db790ba780 0x55db790c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:15.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55db79139700 0x55db7913baf0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:15.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 --2- 172.21.15.32:0/1793128276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db790b9e60 0x55db790c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f611dffb640 0 client.16443 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 -- 172.21.15.32:0/1793128276 >> 172.21.15.32:0/1793128276 conn(0x55db790b4590 msgr2=0x55db790c0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 -- 172.21.15.32:0/1793128276 shutdown_connections 2024-04-16T05:38:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 1 -- 172.21.15.32:0/1793128276 wait complete. 2024-04-16T05:38:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.983+0000 7f613f5c8740 20 client.16443 trim_cache size 0 max 16384 2024-04-16T05:38:15.004 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:38:15.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 -- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f61240a3d20 msgr2=0x7f61240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:15.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f61240a3d20 0x7f61240a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6130098e60 tx=0x7f6130087040 comp rx=0 tx=0).stop 2024-04-16T05:38:15.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 -- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138074d80 msgr2=0x7f61381a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:15.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138074d80 0x7f61381a6ab0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f613000eaa0 tx=0x7f613000ef60 comp rx=0 tx=0).stop 2024-04-16T05:38:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 -- 172.21.15.32:0/758769756 shutdown_connections 2024-04-16T05:38:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f61240a3d20 0x7f61240a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6138074d80 0x7f61381a6ab0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6138107320 0x7f6138074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 --2- 172.21.15.32:0/758769756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6138102250 0x7f6138074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 -- 172.21.15.32:0/758769756 >> 172.21.15.32:0/758769756 conn(0x7f6138079490 msgr2=0x7f61380ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 -- 172.21.15.32:0/758769756 shutdown_connections 2024-04-16T05:38:15.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:14.984+0000 7f613c904640 1 -- 172.21.15.32:0/758769756 wait complete. 2024-04-16T05:38:15.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:15 smithi032 ceph-mon[32351]: pgmap v102: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:38:15.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:15 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2448753974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:15.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:15 smithi134 ceph-mon[36153]: pgmap v102: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:38:15.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:15 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2448753974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:15.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:15 smithi082 ceph-mon[36195]: pgmap v102: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:38:15.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:15 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2448753974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:16.008 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:38:16.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:38:16.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:38:16.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 -- 172.21.15.32:0/3345705135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c102bf0 msgr2=0x7ffa8c106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3345705135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c102bf0 0x7ffa8c106c30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009900 tx=0x7ffa7c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 -- 172.21.15.32:0/3345705135 shutdown_connections 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3345705135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa8c107310 0x7ffa8c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3345705135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c102bf0 0x7ffa8c106c30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3345705135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c102240 0x7ffa8c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 -- 172.21.15.32:0/3345705135 >> 172.21.15.32:0/3345705135 conn(0x7ffa8c0fda40 msgr2=0x7ffa8c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 -- 172.21.15.32:0/3345705135 shutdown_connections 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.133+0000 7ffa942d9640 1 -- 172.21.15.32:0/3345705135 wait complete. 2024-04-16T05:38:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa942d9640 1 Processor -- start 2024-04-16T05:38:16.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa942d9640 1 -- start start 2024-04-16T05:38:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa942d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa8c102240 0x7ffa8c074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa942d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c107310 0x7ffa8c074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa942d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c074d80 0x7ffa8c1a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa942d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1123a0 con 0x7ffa8c074d80 2024-04-16T05:38:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa942d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c112220 con 0x7ffa8c107310 2024-04-16T05:38:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa942d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c112520 con 0x7ffa8c102240 2024-04-16T05:38:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa92876640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c074d80 0x7ffa8c1a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa92876640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c074d80 0x7ffa8c1a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36790/0 (socket says 172.21.15.32:36790) 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.134+0000 7ffa92876640 1 -- 172.21.15.32:0/3807607158 learned_addr learned my addr 172.21.15.32:0/3807607158 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa91874640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c107310 0x7ffa8c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa92075640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa8c102240 0x7ffa8c074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa92876640 1 -- 172.21.15.32:0/3807607158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa8c102240 msgr2=0x7ffa8c074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa92876640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa8c102240 0x7ffa8c074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa92876640 1 -- 172.21.15.32:0/3807607158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c107310 msgr2=0x7ffa8c074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa92876640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c107310 0x7ffa8c074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa92876640 1 -- 172.21.15.32:0/3807607158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c1a71f0 con 0x7ffa8c074d80 2024-04-16T05:38:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa91874640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c107310 0x7ffa8c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa92075640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa8c102240 0x7ffa8c074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa92876640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c074d80 0x7ffa8c1a6af0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400eb10 tx=0x7ffa8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa837fe640 1 -- 172.21.15.32:0/3807607158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84004b20 con 0x7ffa8c074d80 2024-04-16T05:38:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa837fe640 1 -- 172.21.15.32:0/3807607158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa84004cc0 con 0x7ffa8c074d80 2024-04-16T05:38:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa942d9640 1 -- 172.21.15.32:0/3807607158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c1a74e0 con 0x7ffa8c074d80 2024-04-16T05:38:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa837fe640 1 -- 172.21.15.32:0/3807607158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84010760 con 0x7ffa8c074d80 2024-04-16T05:38:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.135+0000 7ffa942d9640 1 -- 172.21.15.32:0/3807607158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c1a7a80 con 0x7ffa8c074d80 2024-04-16T05:38:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa837fe640 1 -- 172.21.15.32:0/3807607158 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa84005020 con 0x7ffa8c074d80 2024-04-16T05:38:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa96fa4740 1 Processor -- start 2024-04-16T05:38:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa96fa4740 1 -- start start 2024-04-16T05:38:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6cab0 0x56367df6ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6d3d0 0x56367df75a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56367ded1e90 con 0x56367df75fb0 2024-04-16T05:38:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56367de6f4c0 con 0x56367df6d3d0 2024-04-16T05:38:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.138+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56367decde50 con 0x56367df6cab0 2024-04-16T05:38:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa837fe640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ffa780a3dd0 0x7ffa780a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df783a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36804/0 (socket says 172.21.15.32:36804) 2024-04-16T05:38:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 -- 172.21.15.32:0/121942418 learned_addr learned my addr 172.21.15.32:0/121942418 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6cab0 0x56367df6ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa92075640 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6d3d0 0x56367df75a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa92876640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ffa780a3dd0 0x7ffa780a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa837fe640 1 -- 172.21.15.32:0/3807607158 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ffa840d1100 con 0x7ffa8c074d80 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 -- 172.21.15.32:0/121942418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6cab0 msgr2=0x56367df6ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6cab0 0x56367df6ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 -- 172.21.15.32:0/121942418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6d3d0 msgr2=0x56367df75a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6d3d0 0x56367df75a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 -- 172.21.15.32:0/121942418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56367dcac7e0 con 0x56367df75fb0 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa91874640 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df783a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c005c20 tx=0x7ffa7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670c85a032013232 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa92876640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ffa780a3dd0 0x7ffa780a6290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffa84099160 tx=0x7ffa8400efd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.139+0000 7ffa80bfe640 1 -- 172.21.15.32:0/121942418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c02cad0 con 0x56367df75fb0 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa80bfe640 1 -- 172.21.15.32:0/121942418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c02cc70 con 0x56367df75fb0 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa80bfe640 1 -- 172.21.15.32:0/121942418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0048c0 con 0x56367df75fb0 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 -- 172.21.15.32:0/121942418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 msgr2=0x56367df783a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df783a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c005c20 tx=0x7ffa7c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 -- 172.21.15.32:0/121942418 shutdown_connections 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df783a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6d3d0 0x56367df75a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/121942418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6cab0 0x56367df6ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 -- 172.21.15.32:0/121942418 >> 172.21.15.32:0/121942418 conn(0x56367df671e0 msgr2=0x56367df675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 -- 172.21.15.32:0/121942418 shutdown_connections 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.140+0000 7ffa96fa4740 1 -- 172.21.15.32:0/121942418 wait complete. 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 1 -- start start 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 1 Processor -- start 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df6cab0 0x56367df73080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df75fb0 0x56367df73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56367df78ce0 con 0x56367df6cab0 2024-04-16T05:38:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56367de5a410 con 0x56367df75fb0 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56367df78e60 con 0x56367df6d3d0 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa96fa4740 10 client.0 _ll_register_callbacks cb 0x56367de55b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df6cab0 0x56367df73080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df6cab0 0x56367df73080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36820/0 (socket says 172.21.15.32:36820) 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 -- 172.21.15.32:0/3143179291 learned_addr learned my addr 172.21.15.32:0/3143179291 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa92075640 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df75fb0 0x56367df73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 -- 172.21.15.32:0/3143179291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 msgr2=0x56367df735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 -- 172.21.15.32:0/3143179291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df75fb0 msgr2=0x56367df73b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df75fb0 0x56367df73b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.141+0000 7ffa91874640 1 -- 172.21.15.32:0/3143179291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56367dd0cf70 con 0x56367df6cab0 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa91874640 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df6cab0 0x56367df73080 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02cc40 tx=0x7ffa7c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56709bf49c5c9170 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa75ffb640 1 -- 172.21.15.32:0/3143179291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c004490 con 0x56367df6cab0 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa75ffb640 1 -- 172.21.15.32:0/3143179291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ffa7c0035f0 con 0x56367df6cab0 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3143179291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c02cc70 con 0x56367df6cab0 2024-04-16T05:38:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa75ffb640 1 -- 172.21.15.32:0/3143179291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c05d050 con 0x56367df6cab0 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa96fa4740 10 client.16455 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3143179291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56367dba4170 con 0x56367df6cab0 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa75ffb640 1 -- 172.21.15.32:0/3143179291 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c02cc70 con 0x56367df6cab0 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa75ffb640 1 -- 172.21.15.32:0/3143179291 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa7c043820 con 0x56367df6cab0 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa96fa4740 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa96fa4740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa96fa4740 20 client.16455 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.142+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56367df861c0 0x56367df885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.143+0000 7ffa96fa4740 4 client.16455 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.143+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3143179291 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56367dedf8b0 con 0x56367df861c0 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.143+0000 7ffa92075640 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56367df861c0 0x56367df885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.143+0000 7ffa92075640 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56367df861c0 0x56367df885b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c72125ba85e6b428 server_cookie=16e923ccecb69969 in_seq=0 out_seq=0 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.143+0000 7ffa75ffb640 10 client.16455 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa75ffb640 1 -- 172.21.15.32:0/3143179291 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56367dedf8b0 con 0x56367df861c0 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa75ffb640 10 client.16455 handle_command_reply: tid=0 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 client.16455 shutdown 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3143179291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df6cab0 msgr2=0x56367df73080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df6cab0 0x56367df73080 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02cc40 tx=0x7ffa7c001e50 comp rx=0 tx=0).stop 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3143179291 shutdown_connections 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df75fb0 0x56367df73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56367df861c0 0x56367df885b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3143179291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df6cab0 0x56367df73080 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3143179291 >> 172.21.15.32:0/3143179291 conn(0x56367df671e0 msgr2=0x56367df68f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa75ffb640 0 client.16455 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3143179291 shutdown_connections 2024-04-16T05:38:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.144+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3143179291 wait complete. 2024-04-16T05:38:16.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.145+0000 7ffa96fa4740 20 client.16455 trim_cache size 0 max 16384 2024-04-16T05:38:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.162+0000 7ffa96fa4740 1 Processor -- start 2024-04-16T05:38:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.162+0000 7ffa96fa4740 1 -- start start 2024-04-16T05:38:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 0x56367df81bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df73920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df660d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b3ee0 con 0x56367df75fb0 2024-04-16T05:38:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b17b0 con 0x56367df6cab0 2024-04-16T05:38:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1af600 con 0x56367df6d3d0 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df660d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36832/0 (socket says 172.21.15.32:36832) 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 -- 172.21.15.32:0/3347349914 learned_addr learned my addr 172.21.15.32:0/3347349914 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 -- 172.21.15.32:0/3347349914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 msgr2=0x56367df73920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa92075640 1 --2- 172.21.15.32:0/3347349914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df73920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 --2- 172.21.15.32:0/3347349914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df73920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 -- 172.21.15.32:0/3347349914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 msgr2=0x56367df81bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 --2- 172.21.15.32:0/3347349914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 0x56367df81bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 -- 172.21.15.32:0/3347349914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56367df65660 con 0x56367df75fb0 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa91874640 1 --2- 172.21.15.32:0/3347349914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df660d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02bc70 tx=0x7ffa7c0097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9284374f0287cbab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa77fff640 1 -- 172.21.15.32:0/3347349914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c02c2b0 con 0x56367df75fb0 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa77fff640 1 -- 172.21.15.32:0/3347349914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c054070 con 0x56367df75fb0 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.163+0000 7ffa77fff640 1 -- 172.21.15.32:0/3347349914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c042680 con 0x56367df75fb0 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3347349914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 msgr2=0x56367df660d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3347349914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df660d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02bc70 tx=0x7ffa7c0097c0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3347349914 shutdown_connections 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3347349914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df660d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3347349914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df73920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/3347349914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 0x56367df81bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3347349914 >> 172.21.15.32:0/3347349914 conn(0x56367df671e0 msgr2=0x56367df659c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3347349914 shutdown_connections 2024-04-16T05:38:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 -- 172.21.15.32:0/3347349914 wait complete. 2024-04-16T05:38:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 -- start start 2024-04-16T05:38:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.164+0000 7ffa96fa4740 1 Processor -- start 2024-04-16T05:38:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 0x56367df6fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df70190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa96fa4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1afc70 con 0x56367df75fb0 2024-04-16T05:38:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b10e0 con 0x56367df6cab0 2024-04-16T05:38:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa96fa4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b0340 con 0x56367df6d3d0 2024-04-16T05:38:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa96fa4740 10 client.0 _ll_register_callbacks cb 0x7ffa88009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36844/0 (socket says 172.21.15.32:36844) 2024-04-16T05:38:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 -- 172.21.15.32:0/575757016 learned_addr learned my addr 172.21.15.32:0/575757016 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 0x56367df6fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa92075640 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df70190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 -- 172.21.15.32:0/575757016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 msgr2=0x56367df70190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df70190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 -- 172.21.15.32:0/575757016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 msgr2=0x56367df6fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 0x56367df6fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 -- 172.21.15.32:0/575757016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56367dfeb3c0 con 0x56367df75fb0 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa92075640 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df70190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa91874640 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df706d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02c7e0 tx=0x7ffa7c02cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c644b8187a176b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa75ffb640 1 -- 172.21.15.32:0/575757016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c066040 con 0x56367df75fb0 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa75ffb640 1 -- 172.21.15.32:0/575757016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ffa7c043700 con 0x56367df75fb0 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.165+0000 7ffa96fa4740 1 -- 172.21.15.32:0/575757016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c054070 con 0x56367df75fb0 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa75ffb640 1 -- 172.21.15.32:0/575757016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c075050 con 0x56367df75fb0 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa96fa4740 10 client.16461 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa96fa4740 1 -- 172.21.15.32:0/575757016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56367dfec5b0 con 0x56367df75fb0 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa75ffb640 1 -- 172.21.15.32:0/575757016 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c054070 con 0x56367df75fb0 2024-04-16T05:38:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa75ffb640 1 -- 172.21.15.32:0/575757016 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa7c06c070 con 0x56367df75fb0 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa96fa4740 10 client.16461 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa96fa4740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa96fa4740 20 client.16461 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56367dfecc10 0x56367dfef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa96fa4740 4 client.16461 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa96fa4740 1 -- 172.21.15.32:0/575757016 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56367ded9720 con 0x56367dfecc10 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.166+0000 7ffa92075640 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56367dfecc10 0x56367dfef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.167+0000 7ffa75ffb640 10 client.16461 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.167+0000 7ffa92075640 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56367dfecc10 0x56367dfef030 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d0c3eb049ba0f2 server_cookie=c8e3a6468fe7826e in_seq=0 out_seq=0 2024-04-16T05:38:16.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa75ffb640 1 -- 172.21.15.32:0/575757016 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56367ded9720 con 0x56367dfecc10 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa75ffb640 10 client.16461 handle_command_reply: tid=0 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 client.16461 shutdown 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 -- 172.21.15.32:0/575757016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 msgr2=0x56367df706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df706d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02c7e0 tx=0x7ffa7c02cd80 comp rx=0 tx=0).stop 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 -- 172.21.15.32:0/575757016 shutdown_connections 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56367df75fb0 0x56367df706d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56367dfecc10 0x56367dfef030 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56367df6d3d0 0x56367df70190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 --2- 172.21.15.32:0/575757016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56367df6cab0 0x56367df6fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 -- 172.21.15.32:0/575757016 >> 172.21.15.32:0/575757016 conn(0x56367df671e0 msgr2=0x56367df68ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa75ffb640 0 client.16461 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 -- 172.21.15.32:0/575757016 shutdown_connections 2024-04-16T05:38:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 1 -- 172.21.15.32:0/575757016 wait complete. 2024-04-16T05:38:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.168+0000 7ffa96fa4740 20 client.16461 trim_cache size 0 max 16384 2024-04-16T05:38:16.176 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:38:16.176 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:38:16.177 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "4b912f9c-ba05-459e-82b8-b1d0a79677cc", 2024-04-16T05:38:16.177 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:38:16.177 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:38:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 -- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ffa780a3dd0 msgr2=0x7ffa780a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ffa780a3dd0 0x7ffa780a6290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffa84099160 tx=0x7ffa8400efd0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 -- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c074d80 msgr2=0x7ffa8c1a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c074d80 0x7ffa8c1a6af0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400eb10 tx=0x7ffa8400c2d0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 -- 172.21.15.32:0/3807607158 shutdown_connections 2024-04-16T05:38:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ffa780a3dd0 0x7ffa780a6290 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c074d80 0x7ffa8c1a6af0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c107310 0x7ffa8c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 --2- 172.21.15.32:0/3807607158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa8c102240 0x7ffa8c074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 -- 172.21.15.32:0/3807607158 >> 172.21.15.32:0/3807607158 conn(0x7ffa8c0fda40 msgr2=0x7ffa8c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 -- 172.21.15.32:0/3807607158 shutdown_connections 2024-04-16T05:38:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.169+0000 7ffa942d9640 1 -- 172.21.15.32:0/3807607158 wait complete. 2024-04-16T05:38:16.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:38:16.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:38:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 -- 172.21.15.32:0/796564633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c102400 msgr2=0x7f9d0c10ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 --2- 172.21.15.32:0/796564633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c102400 0x7f9d0c10ddb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009cd0 tx=0x7f9d0402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 -- 172.21.15.32:0/796564633 shutdown_connections 2024-04-16T05:38:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 --2- 172.21.15.32:0/796564633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c102400 0x7f9d0c10ddb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 --2- 172.21.15.32:0/796564633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c101a40 0x7f9d0c101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 --2- 172.21.15.32:0/796564633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c105ed0 0x7f9d0c1062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 -- 172.21.15.32:0/796564633 >> 172.21.15.32:0/796564633 conn(0x7f9d0c076190 msgr2=0x7f9d0c0765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 -- 172.21.15.32:0/796564633 shutdown_connections 2024-04-16T05:38:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.290+0000 7f9d11349640 1 -- 172.21.15.32:0/796564633 wait complete. 2024-04-16T05:38:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d11349640 1 Processor -- start 2024-04-16T05:38:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d11349640 1 -- start start 2024-04-16T05:38:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d11349640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c101a40 0x7f9d0c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d11349640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c102400 0x7f9d0c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d11349640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c105ed0 0x7f9d0c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d11349640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c111280 con 0x7f9d0c105ed0 2024-04-16T05:38:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d11349640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c111100 con 0x7f9d0c101a40 2024-04-16T05:38:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d11349640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c111400 con 0x7f9d0c102400 2024-04-16T05:38:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c105ed0 0x7f9d0c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c105ed0 0x7f9d0c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36856/0 (socket says 172.21.15.32:36856) 2024-04-16T05:38:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0b7fe640 1 -- 172.21.15.32:0/1604266972 learned_addr learned my addr 172.21.15.32:0/1604266972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d03fff640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c102400 0x7f9d0c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0affd640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c101a40 0x7f9d0c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0b7fe640 1 -- 172.21.15.32:0/1604266972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c102400 msgr2=0x7f9d0c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c102400 0x7f9d0c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0b7fe640 1 -- 172.21.15.32:0/1604266972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c101a40 msgr2=0x7f9d0c19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c101a40 0x7f9d0c19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.291+0000 7f9d0b7fe640 1 -- 172.21.15.32:0/1604266972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0c198d70 con 0x7f9d0c105ed0 2024-04-16T05:38:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.292+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c105ed0 0x7f9d0c1985f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9d04004af0 tx=0x7f9d04032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.292+0000 7f9d08ff9640 1 -- 172.21.15.32:0/1604266972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04042020 con 0x7f9d0c105ed0 2024-04-16T05:38:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.292+0000 7f9d08ff9640 1 -- 172.21.15.32:0/1604266972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d04034640 con 0x7f9d0c105ed0 2024-04-16T05:38:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.292+0000 7f9d08ff9640 1 -- 172.21.15.32:0/1604266972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04039040 con 0x7f9d0c105ed0 2024-04-16T05:38:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.292+0000 7f9d11349640 1 -- 172.21.15.32:0/1604266972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0c199000 con 0x7f9d0c105ed0 2024-04-16T05:38:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.292+0000 7f9d11349640 1 -- 172.21.15.32:0/1604266972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0c1a5300 con 0x7f9d0c105ed0 2024-04-16T05:38:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.294+0000 7f9d08ff9640 1 -- 172.21.15.32:0/1604266972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d04045020 con 0x7f9d0c105ed0 2024-04-16T05:38:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.294+0000 7f9d1400b740 1 Processor -- start 2024-04-16T05:38:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.294+0000 7f9d1400b740 1 -- start start 2024-04-16T05:38:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e0e50 0x55d8601e1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601ea350 0x55d8601ec740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d1400b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d860146180 con 0x55d8601e0e50 2024-04-16T05:38:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d1400b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d8600e3360 con 0x55d8601ea350 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d1400b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d860142140 con 0x55d8601e1770 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e0e50 0x55d8601e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e0e50 0x55d8601e1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36866/0 (socket says 172.21.15.32:36866) 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 -- 172.21.15.32:0/819352463 learned_addr learned my addr 172.21.15.32:0/819352463 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d0affd640 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601ea350 0x55d8601ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 -- 172.21.15.32:0/819352463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 msgr2=0x55d8601e9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 -- 172.21.15.32:0/819352463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601ea350 msgr2=0x55d8601ec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601ea350 0x55d8601ec740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 -- 172.21.15.32:0/819352463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85ff227d0 con 0x55d8601e0e50 2024-04-16T05:38:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d0affd640 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d08ff9640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9cf80a3d20 0x7f9cf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d03fff640 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e0e50 0x55d8601e1230 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40058a0 tx=0x7f9cf400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d82f501ab3e5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.295+0000 7f9d0affd640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9cf80a3d20 0x7f9cf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d08ff9640 1 -- 172.21.15.32:0/1604266972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f9d040f2bc0 con 0x7f9d0c105ed0 2024-04-16T05:38:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d017fa640 1 -- 172.21.15.32:0/819352463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004030 con 0x55d8601e0e50 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d017fa640 1 -- 172.21.15.32:0/819352463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4005500 con 0x55d8601e0e50 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d017fa640 1 -- 172.21.15.32:0/819352463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004c40 con 0x55d8601e0e50 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 -- 172.21.15.32:0/819352463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e0e50 msgr2=0x55d8601e1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d0affd640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9cf80a3d20 0x7f9cf80a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55d8601e0920 tx=0x7f9cfc00d3c0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e0e50 0x55d8601e1230 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40058a0 tx=0x7f9cf400b820 comp rx=0 tx=0).stop 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 -- 172.21.15.32:0/819352463 shutdown_connections 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601ea350 0x55d8601ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 --2- 172.21.15.32:0/819352463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e0e50 0x55d8601e1230 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 -- 172.21.15.32:0/819352463 >> 172.21.15.32:0/819352463 conn(0x55d8601db580 msgr2=0x55d8601db960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 -- 172.21.15.32:0/819352463 shutdown_connections 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.296+0000 7f9d1400b740 1 -- 172.21.15.32:0/819352463 wait complete. 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 1 -- start start 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 1 Processor -- start 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601ea350 0x55d8601e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d8601ed140 con 0x55d8601ea350 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d8600ff2c0 con 0x55d8601e0e50 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d8601ed2c0 con 0x55d8601e1770 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d1400b740 10 client.0 _ll_register_callbacks cb 0x55d8600e6090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601ea350 0x55d8601e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601ea350 0x55d8601e7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36874/0 (socket says 172.21.15.32:36874) 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d0b7fe640 1 -- 172.21.15.32:0/564309543 learned_addr learned my addr 172.21.15.32:0/564309543 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d0affd640 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d03fff640 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.297+0000 7f9d0b7fe640 1 -- 172.21.15.32:0/564309543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 msgr2=0x55d8601e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d0b7fe640 1 -- 172.21.15.32:0/564309543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 msgr2=0x55d8601e7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601e7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d0b7fe640 1 -- 172.21.15.32:0/564309543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85fe583f0 con 0x55d8601ea350 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d03fff640 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601ea350 0x55d8601e7e90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9d040bad90 tx=0x7f9d040be290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c164d1227e43451 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9cf27fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9cf27fc640 1 -- 172.21.15.32:0/564309543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040c2ce0 con 0x55d8601ea350 2024-04-16T05:38:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9cf27fc640 1 -- 172.21.15.32:0/564309543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f9d040a90d0 con 0x55d8601ea350 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 1 -- 172.21.15.32:0/564309543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cf4005500 con 0x55d8601ea350 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9cf27fc640 1 -- 172.21.15.32:0/564309543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d040a9310 con 0x55d8601ea350 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 10 client.16473 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 1 -- 172.21.15.32:0/564309543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d85fe180f0 con 0x55d8601ea350 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9cf27fc640 1 -- 172.21.15.32:0/564309543 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4005500 con 0x55d8601ea350 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9cf27fc640 1 -- 172.21.15.32:0/564309543 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d04004b40 con 0x55d8601ea350 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 10 client.16473 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 20 client.16473 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d8601fa6c0 0x55d8601fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 4 client.16473 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.298+0000 7f9d1400b740 1 -- 172.21.15.32:0/564309543 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d860153ba0 con 0x55d8601fa6c0 2024-04-16T05:38:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.299+0000 7f9d0affd640 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d8601fa6c0 0x55d8601fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.299+0000 7f9d0affd640 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d8601fa6c0 0x55d8601fcae0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4944ab04d15e0e8d server_cookie=4763177457a77a0 in_seq=0 out_seq=0 2024-04-16T05:38:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.299+0000 7f9cf27fc640 10 client.16473 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9cf27fc640 1 -- 172.21.15.32:0/564309543 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d860153ba0 con 0x55d8601fa6c0 2024-04-16T05:38:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9cf27fc640 10 client.16473 handle_command_reply: tid=0 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 client.16473 shutdown 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 -- 172.21.15.32:0/564309543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601ea350 msgr2=0x55d8601e7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601ea350 0x55d8601e7e90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9d040bad90 tx=0x7f9d040be290 comp rx=0 tx=0).stop 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 -- 172.21.15.32:0/564309543 shutdown_connections 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601ea350 0x55d8601e7e90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d8601fa6c0 0x55d8601fcae0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601e1770 0x55d8601e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 --2- 172.21.15.32:0/564309543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601e7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9cf27fc640 0 client.16473 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 -- 172.21.15.32:0/564309543 >> 172.21.15.32:0/564309543 conn(0x55d8601db580 msgr2=0x55d8601dd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 -- 172.21.15.32:0/564309543 shutdown_connections 2024-04-16T05:38:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 1 -- 172.21.15.32:0/564309543 wait complete. 2024-04-16T05:38:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.300+0000 7f9d1400b740 20 client.16473 trim_cache size 0 max 16384 2024-04-16T05:38:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d1400b740 1 Processor -- start 2024-04-16T05:38:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d1400b740 1 -- start start 2024-04-16T05:38:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601f6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601e8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d1400b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b1ae0 con 0x55d8601e1770 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d1400b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1af3b0 con 0x55d8601e0e50 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d1400b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1ad200 con 0x55d8601ea350 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.318+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601e7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36884/0 (socket says 172.21.15.32:36884) 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d03fff640 1 -- 172.21.15.32:0/2793106646 learned_addr learned my addr 172.21.15.32:0/2793106646 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d0affd640 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d03fff640 1 -- 172.21.15.32:0/2793106646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 msgr2=0x55d8601e8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d03fff640 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601e8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d03fff640 1 -- 172.21.15.32:0/2793106646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 msgr2=0x55d8601f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d03fff640 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d03fff640 1 -- 172.21.15.32:0/2793106646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8601e15a0 con 0x55d8601e1770 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d03fff640 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601e7ae0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400bfd0 tx=0x7f9cf4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0c5764d64bec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d00ff9640 1 -- 172.21.15.32:0/2793106646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400bc60 con 0x55d8601e1770 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d00ff9640 1 -- 172.21.15.32:0/2793106646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4005510 con 0x55d8601e1770 2024-04-16T05:38:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.319+0000 7f9d00ff9640 1 -- 172.21.15.32:0/2793106646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4003c60 con 0x55d8601e1770 2024-04-16T05:38:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 -- 172.21.15.32:0/2793106646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 msgr2=0x55d8601e7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601e7ae0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400bfd0 tx=0x7f9cf4005c50 comp rx=0 tx=0).stop 2024-04-16T05:38:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 -- 172.21.15.32:0/2793106646 shutdown_connections 2024-04-16T05:38:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601e8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601e7ae0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 --2- 172.21.15.32:0/2793106646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 -- 172.21.15.32:0/2793106646 >> 172.21.15.32:0/2793106646 conn(0x55d8601db580 msgr2=0x55d8601e73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 -- 172.21.15.32:0/2793106646 shutdown_connections 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 -- 172.21.15.32:0/2793106646 wait complete. 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 -- start start 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.320+0000 7f9d1400b740 1 Processor -- start 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d1400b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d1400b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1ad870 con 0x55d8601e1770 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d1400b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1aece0 con 0x55d8601e0e50 2024-04-16T05:38:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d1400b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1adf40 con 0x55d8601ea350 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d1400b740 10 client.0 _ll_register_callbacks cb 0x7f9d040be700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36894/0 (socket says 172.21.15.32:36894) 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 -- 172.21.15.32:0/329633008 learned_addr learned my addr 172.21.15.32:0/329633008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d0affd640 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 -- 172.21.15.32:0/329633008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 msgr2=0x55d8601f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 -- 172.21.15.32:0/329633008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 msgr2=0x55d8601f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 -- 172.21.15.32:0/329633008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8601f1340 con 0x55d8601e1770 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d0b7fe640 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d03fff640 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601f2ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400bfa0 tx=0x7f9cf40056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b60cc17d2a1b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9cf27fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9cf27fc640 1 -- 172.21.15.32:0/329633008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4005070 con 0x55d8601e1770 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9d1400b740 1 -- 172.21.15.32:0/329633008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cf4005510 con 0x55d8601e1770 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.321+0000 7f9cf27fc640 1 -- 172.21.15.32:0/329633008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f9cf4025070 con 0x55d8601e1770 2024-04-16T05:38:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9cf27fc640 1 -- 172.21.15.32:0/329633008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9cf4037050 con 0x55d8601e1770 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d1400b740 10 client.16479 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d1400b740 1 -- 172.21.15.32:0/329633008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d860261960 con 0x55d8601e1770 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9cf27fc640 1 -- 172.21.15.32:0/329633008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4005510 con 0x55d8601e1770 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9cf27fc640 1 -- 172.21.15.32:0/329633008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9cf401c020 con 0x55d8601e1770 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d1400b740 10 client.16479 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d1400b740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d1400b740 20 client.16479 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d1400b740 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d860261e90 0x55d8602642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d1400b740 4 client.16479 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d1400b740 1 -- 172.21.15.32:0/329633008 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d86014da10 con 0x55d860261e90 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.322+0000 7f9d0affd640 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d860261e90 0x55d8602642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9cf27fc640 10 client.16479 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d0affd640 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d860261e90 0x55d8602642b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7b17e956d53e31 server_cookie=6137c463239f1995 in_seq=0 out_seq=0 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9cf27fc640 1 -- 172.21.15.32:0/329633008 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d86014da10 con 0x55d860261e90 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9cf27fc640 10 client.16479 handle_command_reply: tid=0 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 client.16479 shutdown 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 -- 172.21.15.32:0/329633008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 msgr2=0x55d8601f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601f2ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400bfa0 tx=0x7f9cf40056c0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 -- 172.21.15.32:0/329633008 shutdown_connections 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d860261e90 0x55d8602642b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8601ea350 0x55d8601f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d8601e1770 0x55d8601f2ab0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9cf27fc640 0 client.16479 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 --2- 172.21.15.32:0/329633008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d8601e0e50 0x55d8601f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 -- 172.21.15.32:0/329633008 >> 172.21.15.32:0/329633008 conn(0x55d8601db580 msgr2=0x55d8601e7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 -- 172.21.15.32:0/329633008 shutdown_connections 2024-04-16T05:38:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 1 -- 172.21.15.32:0/329633008 wait complete. 2024-04-16T05:38:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.323+0000 7f9d1400b740 20 client.16479 trim_cache size 0 max 16384 2024-04-16T05:38:16.332 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:38:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:38:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:38:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "4b912f9c-ba05-459e-82b8-b1d0a79677cc": { 2024-04-16T05:38:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:38:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "4b912f9c-ba05-459e-82b8-b1d0a79677cc", 2024-04-16T05:38:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:38:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:38:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:38:16.333 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:38:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 -- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9cf80a3d20 msgr2=0x7f9cf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9cf80a3d20 0x7f9cf80a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55d8601e0920 tx=0x7f9cfc00d3c0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 -- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c105ed0 msgr2=0x7f9d0c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c105ed0 0x7f9d0c1985f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9d04004af0 tx=0x7f9d04032ec0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 -- 172.21.15.32:0/1604266972 shutdown_connections 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9cf80a3d20 0x7f9cf80a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c105ed0 0x7f9d0c1985f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c102400 0x7f9d0c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 --2- 172.21.15.32:0/1604266972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c101a40 0x7f9d0c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 -- 172.21.15.32:0/1604266972 >> 172.21.15.32:0/1604266972 conn(0x7f9d0c076190 msgr2=0x7f9d0c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 -- 172.21.15.32:0/1604266972 shutdown_connections 2024-04-16T05:38:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:16.324+0000 7f9d11349640 1 -- 172.21.15.32:0/1604266972 wait complete. 2024-04-16T05:38:16.336 INFO:tasks.cephfs.filesystem:scrub status for tag:4b912f9c-ba05-459e-82b8-b1d0a79677cc - {'path': '/', 'tag': '4b912f9c-ba05-459e-82b8-b1d0a79677cc', 'options': 'recursive,force'} 2024-04-16T05:38:17.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:17 smithi032 ceph-mon[32351]: pgmap v103: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:38:17.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:17 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:38:17.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:17 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:17.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:17 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:38:17.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:17 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:17 smithi134 ceph-mon[36153]: pgmap v103: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:38:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:17 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:38:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:17 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:17 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:38:17.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:17 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:17 smithi082 ceph-mon[36195]: pgmap v103: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:38:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:17 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:38:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:17 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:17 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:38:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:17 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:19.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:19 smithi032 ceph-mon[32351]: pgmap v104: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:38:19.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:19 smithi134 ceph-mon[36153]: pgmap v104: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:38:19.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:19 smithi082 ceph-mon[36195]: pgmap v104: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-16T05:38:21.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:21 smithi032 ceph-mon[32351]: pgmap v105: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-16T05:38:21.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:21 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:38:21.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:21 smithi134 ceph-mon[36153]: pgmap v105: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-16T05:38:21.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:21 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:38:21.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:21 smithi082 ceph-mon[36195]: pgmap v105: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-16T05:38:21.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:21 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:38:23.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:23 smithi032 ceph-mon[32351]: pgmap v106: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:38:23.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:23 smithi134 ceph-mon[36153]: pgmap v106: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:38:23.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:23 smithi082 ceph-mon[36195]: pgmap v106: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:38:25.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:25 smithi032 ceph-mon[32351]: pgmap v107: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:25.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:25 smithi134 ceph-mon[36153]: pgmap v107: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:25.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:25 smithi082 ceph-mon[36195]: pgmap v107: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:27.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:27 smithi032 ceph-mon[32351]: pgmap v108: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:27.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:27 smithi134 ceph-mon[36153]: pgmap v108: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:27.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:27 smithi082 ceph-mon[36195]: pgmap v108: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:29.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:29 smithi032 ceph-mon[32351]: pgmap v109: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:29.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:29 smithi134 ceph-mon[36153]: pgmap v109: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:29.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:29 smithi082 ceph-mon[36195]: pgmap v109: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:31.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:31 smithi032 ceph-mon[32351]: pgmap v110: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:31.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:31 smithi134 ceph-mon[36153]: pgmap v110: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:31.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:31 smithi082 ceph-mon[36195]: pgmap v110: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:33.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:33 smithi134 ceph-mon[36153]: pgmap v111: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:33.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:33 smithi082 ceph-mon[36195]: pgmap v111: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:33.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:33 smithi032 ceph-mon[32351]: pgmap v111: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:35.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:35 smithi134 ceph-mon[36153]: pgmap v112: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:35.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:35 smithi082 ceph-mon[36195]: pgmap v112: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:35.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:35 smithi032 ceph-mon[32351]: pgmap v112: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:37.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:37 smithi134 ceph-mon[36153]: pgmap v113: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:37.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:37 smithi082 ceph-mon[36195]: pgmap v113: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:37.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:37 smithi032 ceph-mon[32351]: pgmap v113: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:39.535 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:39 smithi032 ceph-mon[32351]: pgmap v114: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:39.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:39 smithi134 ceph-mon[36153]: pgmap v114: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:39.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:39 smithi082 ceph-mon[36195]: pgmap v114: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:41.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:41 smithi134 ceph-mon[36153]: pgmap v115: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:41.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:41 smithi082 ceph-mon[36195]: pgmap v115: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:41.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:41 smithi032 ceph-mon[32351]: pgmap v115: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:43.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:43 smithi134 ceph-mon[36153]: pgmap v116: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:43.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:43 smithi082 ceph-mon[36195]: pgmap v116: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:43.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:43 smithi032 ceph-mon[32351]: pgmap v116: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:45.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:45 smithi134 ceph-mon[36153]: pgmap v117: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:45.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:45 smithi082 ceph-mon[36195]: pgmap v117: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:45.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:45 smithi032 ceph-mon[32351]: pgmap v117: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:46.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:38:46.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:38:46.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/1171415449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 msgr2=0x7fbdf4110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/1171415449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 0x7fbdf4110c80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009ca0 tx=0x7fbdec02c180 comp rx=0 tx=0).stop 2024-04-16T05:38:46.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/1171415449 shutdown_connections 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/1171415449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 0x7fbdf4110c80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/1171415449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdf4074cf0 0x7fbdf410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/1171415449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf40743d0 0x7fbdf40747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/1171415449 >> 172.21.15.32:0/1171415449 conn(0x7fbdf40fda40 msgr2=0x7fbdf40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/1171415449 shutdown_connections 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.459+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/1171415449 wait complete. 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf9c1b640 1 Processor -- start 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf9c1b640 1 -- start start 2024-04-16T05:38:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf9c1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdf40743d0 0x7fbdf41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf9c1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4074cf0 0x7fbdf41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf9c1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 0x7fbdf419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf9c1b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41145b0 con 0x7fbdf410e870 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf9c1b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4114430 con 0x7fbdf40743d0 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf9c1b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4114730 con 0x7fbdf4074cf0 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.460+0000 7fbdf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 0x7fbdf419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 0x7fbdf419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56868/0 (socket says 172.21.15.32:56868) 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf3fff640 1 -- 172.21.15.32:0/2549943578 learned_addr learned my addr 172.21.15.32:0/2549943578 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4074cf0 0x7fbdf41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf37fe640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdf40743d0 0x7fbdf41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf3fff640 1 -- 172.21.15.32:0/2549943578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4074cf0 msgr2=0x7fbdf41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf3fff640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4074cf0 0x7fbdf41a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf3fff640 1 -- 172.21.15.32:0/2549943578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdf40743d0 msgr2=0x7fbdf41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf3fff640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdf40743d0 0x7fbdf41a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf3fff640 1 -- 172.21.15.32:0/2549943578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf419b090 con 0x7fbdf410e870 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf3fff640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 0x7fbdf419a910 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005cc0 tx=0x7fbdec035ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf0ff9640 1 -- 172.21.15.32:0/2549943578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec039040 con 0x7fbdf410e870 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf0ff9640 1 -- 172.21.15.32:0/2549943578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdec044070 con 0x7fbdf410e870 2024-04-16T05:38:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/2549943578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf419b320 con 0x7fbdf410e870 2024-04-16T05:38:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf0ff9640 1 -- 172.21.15.32:0/2549943578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0329f0 con 0x7fbdf410e870 2024-04-16T05:38:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.461+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/2549943578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf41a7620 con 0x7fbdf410e870 2024-04-16T05:38:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.464+0000 7fbdf0ff9640 1 -- 172.21.15.32:0/2549943578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbdec032b90 con 0x7fbdf410e870 2024-04-16T05:38:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.464+0000 7fbdfc806740 1 Processor -- start 2024-04-16T05:38:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.464+0000 7fbdfc806740 1 -- start start 2024-04-16T05:38:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.464+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac68230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac70e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac73740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdfc806740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e27abcd180 con 0x55e27ac67e50 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdfc806740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e27ab6a360 con 0x55e27ac68770 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdfc806740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e27abc9140 con 0x55e27ac71350 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac68230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac68230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56872/0 (socket says 172.21.15.32:56872) 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/3456526307 learned_addr learned my addr 172.21.15.32:0/3456526307 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf0ff9640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf37fe640 1 --2- 172.21.15.32:0/3456526307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac70e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf3fff640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/3456526307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 msgr2=0x55e27ac73740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/3456526307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac73740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/3456526307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 msgr2=0x55e27ac70e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/3456526307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac70e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/3456526307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27a9a97d0 con 0x55e27ac67e50 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf0ff9640 1 -- 172.21.15.32:0/2549943578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbdec0f3030 con 0x7fbdf410e870 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/3456526307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac68230 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbde800dc80 tx=0x7fbde8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48fb21e99b1ce4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdd9ffb640 1 -- 172.21.15.32:0/3456526307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde801a400 con 0x55e27ac67e50 2024-04-16T05:38:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdf3fff640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0bed30 tx=0x7fbdec02c180 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdd9ffb640 1 -- 172.21.15.32:0/3456526307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbde80128d0 con 0x55e27ac67e50 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.465+0000 7fbdd9ffb640 1 -- 172.21.15.32:0/3456526307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde800f910 con 0x55e27ac67e50 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 -- 172.21.15.32:0/3456526307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 msgr2=0x55e27ac68230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 --2- 172.21.15.32:0/3456526307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac68230 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbde800dc80 tx=0x7fbde8011750 comp rx=0 tx=0).stop 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 -- 172.21.15.32:0/3456526307 shutdown_connections 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 --2- 172.21.15.32:0/3456526307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac73740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 --2- 172.21.15.32:0/3456526307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac70e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 --2- 172.21.15.32:0/3456526307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac68230 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 -- 172.21.15.32:0/3456526307 >> 172.21.15.32:0/3456526307 conn(0x55e27ac62580 msgr2=0x55e27ac62960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 -- 172.21.15.32:0/3456526307 shutdown_connections 2024-04-16T05:38:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.466+0000 7fbdfc806740 1 -- 172.21.15.32:0/3456526307 wait complete. 2024-04-16T05:38:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 1 -- start start 2024-04-16T05:38:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 1 Processor -- start 2024-04-16T05:38:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac6e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac6e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac6ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e27ac74140 con 0x55e27ac67e50 2024-04-16T05:38:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e27ab862c0 con 0x55e27ac68770 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e27ac742c0 con 0x55e27ac71350 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac6e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdfc806740 10 client.0 _ll_register_callbacks cb 0x55e27ab6d090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac6e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56884/0 (socket says 172.21.15.32:56884) 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/784116491 learned_addr learned my addr 172.21.15.32:0/784116491 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdf37fe640 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac6e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.467+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac6ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/784116491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 msgr2=0x55e27ac6ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac6ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/784116491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 msgr2=0x55e27ac6e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac6e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/784116491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27a8df3f0 con 0x55e27ac67e50 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac6e3b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbde800d5a0 tx=0x7fbde800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cbcae9abc077c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/784116491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8004030 con 0x55e27ac67e50 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/784116491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbde80041d0 con 0x55e27ac67e50 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdfc806740 1 -- 172.21.15.32:0/784116491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde80128d0 con 0x55e27ac67e50 2024-04-16T05:38:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/784116491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbde802f050 con 0x55e27ac67e50 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdfc806740 10 client.16491 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdfc806740 1 -- 172.21.15.32:0/784116491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e27a89f0f0 con 0x55e27ac67e50 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/784116491 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbde80128d0 con 0x55e27ac67e50 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.468+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/784116491 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbde8004920 con 0x55e27ac67e50 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdfc806740 10 client.16491 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdfc806740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdfc806740 20 client.16491 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdfc806740 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e27ac81620 0x55e27ac83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdfc806740 4 client.16491 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdfc806740 1 -- 172.21.15.32:0/784116491 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e27abdaba0 con 0x55e27ac81620 2024-04-16T05:38:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdf37fe640 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e27ac81620 0x55e27ac83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdd2ffd640 10 client.16491 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.469+0000 7fbdf37fe640 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e27ac81620 0x55e27ac83a10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18b1161dc1a9d93 server_cookie=7cef2d178137d7b0 in_seq=0 out_seq=0 2024-04-16T05:38:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/784116491 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e27abdaba0 con 0x55e27ac81620 2024-04-16T05:38:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdd2ffd640 10 client.16491 handle_command_reply: tid=0 2024-04-16T05:38:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 client.16491 shutdown 2024-04-16T05:38:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 -- 172.21.15.32:0/784116491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 msgr2=0x55e27ac6e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac6e3b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbde800d5a0 tx=0x7fbde800f3c0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 -- 172.21.15.32:0/784116491 shutdown_connections 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e27ac81620 0x55e27ac83a10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac6ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac6e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdd2ffd640 0 client.16491 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 --2- 172.21.15.32:0/784116491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac6e3b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 -- 172.21.15.32:0/784116491 >> 172.21.15.32:0/784116491 conn(0x55e27ac62580 msgr2=0x55e27ac642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 -- 172.21.15.32:0/784116491 shutdown_connections 2024-04-16T05:38:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 1 -- 172.21.15.32:0/784116491 wait complete. 2024-04-16T05:38:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.470+0000 7fbdfc806740 20 client.16491 trim_cache size 0 max 16384 2024-04-16T05:38:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdfc806740 1 Processor -- start 2024-04-16T05:38:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdfc806740 1 -- start start 2024-04-16T05:38:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac7d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac6ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdfc806740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41b3f00 con 0x55e27ac67e50 2024-04-16T05:38:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdfc806740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41b17d0 con 0x55e27ac68770 2024-04-16T05:38:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdfc806740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41af620 con 0x55e27ac71350 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac7d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac7d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56892/0 (socket says 172.21.15.32:56892) 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.488+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/567881976 learned_addr learned my addr 172.21.15.32:0/567881976 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/567881976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 msgr2=0x55e27ac88fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/567881976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/567881976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 msgr2=0x55e27ac6ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/567881976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac6ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/567881976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27ac691f0 con 0x55e27ac67e50 2024-04-16T05:38:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/567881976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac7d030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbde80058c0 tx=0x7fbde800d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b5d8415e3dd6b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdd97fa640 1 -- 172.21.15.32:0/567881976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8011c50 con 0x55e27ac67e50 2024-04-16T05:38:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdd97fa640 1 -- 172.21.15.32:0/567881976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbde801f070 con 0x55e27ac67e50 2024-04-16T05:38:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.489+0000 7fbdd97fa640 1 -- 172.21.15.32:0/567881976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8005170 con 0x55e27ac67e50 2024-04-16T05:38:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- 172.21.15.32:0/567881976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 msgr2=0x55e27ac7d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 --2- 172.21.15.32:0/567881976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac7d030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbde80058c0 tx=0x7fbde800d3e0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- 172.21.15.32:0/567881976 shutdown_connections 2024-04-16T05:38:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 --2- 172.21.15.32:0/567881976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac71350 0x55e27ac88fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 --2- 172.21.15.32:0/567881976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac68770 0x55e27ac6ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 --2- 172.21.15.32:0/567881976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac67e50 0x55e27ac7d030 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- 172.21.15.32:0/567881976 >> 172.21.15.32:0/567881976 conn(0x55e27ac62580 msgr2=0x55e27ac64240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- 172.21.15.32:0/567881976 shutdown_connections 2024-04-16T05:38:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- 172.21.15.32:0/567881976 wait complete. 2024-04-16T05:38:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- start start 2024-04-16T05:38:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 Processor -- start 2024-04-16T05:38:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac67e50 0x55e27ac770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac68770 0x55e27ac77600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac71350 0x55e27ac77b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41afc90 con 0x55e27ac71350 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41b1100 con 0x55e27ac67e50 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41b0360 con 0x55e27ac68770 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.490+0000 7fbdfc806740 10 client.0 _ll_register_callbacks cb 0x7fbde800db50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac71350 0x55e27ac77b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac71350 0x55e27ac77b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56898/0 (socket says 172.21.15.32:56898) 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/1430665453 learned_addr learned my addr 172.21.15.32:0/1430665453 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac67e50 0x55e27ac770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/1430665453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac68770 msgr2=0x55e27ac77600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf37fe640 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac68770 0x55e27ac77600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac68770 0x55e27ac77600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/1430665453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac67e50 msgr2=0x55e27ac770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac67e50 0x55e27ac770c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 -- 172.21.15.32:0/1430665453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27ace6960 con 0x55e27ac71350 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdf2ffd640 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac71350 0x55e27ac77b40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbde800d5b0 tx=0x7fbde8005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3ec6aad5d1689a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/1430665453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde80127e0 con 0x55e27ac71350 2024-04-16T05:38:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdfc806740 1 -- 172.21.15.32:0/1430665453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde801f070 con 0x55e27ac71350 2024-04-16T05:38:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/1430665453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbde8004890 con 0x55e27ac71350 2024-04-16T05:38:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/1430665453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbde8004a90 con 0x55e27ac71350 2024-04-16T05:38:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdfc806740 10 client.16497 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.491+0000 7fbdfc806740 1 -- 172.21.15.32:0/1430665453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e27ace7c50 con 0x55e27ac71350 2024-04-16T05:38:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/1430665453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbde801f070 con 0x55e27ac71350 2024-04-16T05:38:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/1430665453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbde8028800 con 0x55e27ac71350 2024-04-16T05:38:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdfc806740 10 client.16497 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdfc806740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdfc806740 20 client.16497 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdfc806740 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e27ace8300 0x55e27acea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdfc806740 4 client.16497 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdfc806740 1 -- 172.21.15.32:0/1430665453 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e27abd4a10 con 0x55e27ace8300 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdf37fe640 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e27ace8300 0x55e27acea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdd2ffd640 10 client.16497 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.492+0000 7fbdf37fe640 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e27ace8300 0x55e27acea720 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58308d0b563708e2 server_cookie=f5075b7e97f5fb8a in_seq=0 out_seq=0 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdd2ffd640 1 -- 172.21.15.32:0/1430665453 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e27abd4a10 con 0x55e27ace8300 2024-04-16T05:38:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdd2ffd640 10 client.16497 handle_command_reply: tid=0 2024-04-16T05:38:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 client.16497 shutdown 2024-04-16T05:38:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 -- 172.21.15.32:0/1430665453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac71350 msgr2=0x55e27ac77b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac71350 0x55e27ac77b40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbde800d5b0 tx=0x7fbde8005fb0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 -- 172.21.15.32:0/1430665453 shutdown_connections 2024-04-16T05:38:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e27ac71350 0x55e27ac77b40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e27ace8300 0x55e27acea720 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e27ac68770 0x55e27ac77600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 --2- 172.21.15.32:0/1430665453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e27ac67e50 0x55e27ac770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdd2ffd640 0 client.16497 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 -- 172.21.15.32:0/1430665453 >> 172.21.15.32:0/1430665453 conn(0x55e27ac62580 msgr2=0x55e27ac638e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 -- 172.21.15.32:0/1430665453 shutdown_connections 2024-04-16T05:38:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 1 -- 172.21.15.32:0/1430665453 wait complete. 2024-04-16T05:38:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.493+0000 7fbdfc806740 20 client.16497 trim_cache size 0 max 16384 2024-04-16T05:38:46.501 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:38:46.501 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:38:46.502 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:38:46.502 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:38:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbddc0a3d20 msgr2=0x7fbddc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0bed30 tx=0x7fbdec02c180 comp rx=0 tx=0).stop 2024-04-16T05:38:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 msgr2=0x7fbdf419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 0x7fbdf419a910 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005cc0 tx=0x7fbdec035ed0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/2549943578 shutdown_connections 2024-04-16T05:38:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf410e870 0x7fbdf419a910 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4074cf0 0x7fbdf41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 --2- 172.21.15.32:0/2549943578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdf40743d0 0x7fbdf41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/2549943578 >> 172.21.15.32:0/2549943578 conn(0x7fbdf40fda40 msgr2=0x7fbdf40fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/2549943578 shutdown_connections 2024-04-16T05:38:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.494+0000 7fbdf9c1b640 1 -- 172.21.15.32:0/2549943578 wait complete. 2024-04-16T05:38:46.505 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:38:46.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:38:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 -- 172.21.15.32:0/2463457999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e8103c00 msgr2=0x7f05e810c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/2463457999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e8103c00 0x7f05e810c0e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009900 tx=0x7f05d002bcd0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 -- 172.21.15.32:0/2463457999 shutdown_connections 2024-04-16T05:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/2463457999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e810c620 0x7f05e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/2463457999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e8103c00 0x7f05e810c0e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/2463457999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e81032e0 0x7f05e81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 -- 172.21.15.32:0/2463457999 >> 172.21.15.32:0/2463457999 conn(0x7f05e8078060 msgr2=0x7f05e8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 -- 172.21.15.32:0/2463457999 shutdown_connections 2024-04-16T05:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.612+0000 7f05ed4eb640 1 -- 172.21.15.32:0/2463457999 wait complete. 2024-04-16T05:38:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05ed4eb640 1 Processor -- start 2024-04-16T05:38:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05ed4eb640 1 -- start start 2024-04-16T05:38:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05ed4eb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e81032e0 0x7f05e819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05ed4eb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e8103c00 0x7f05e819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05ed4eb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e810c620 0x7f05e81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05ed4eb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05e81123d0 con 0x7f05e810c620 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05ed4eb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8112250 con 0x7f05e8103c00 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05ed4eb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8112550 con 0x7f05e81032e0 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e810c620 0x7f05e81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e810c620 0x7f05e81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56916/0 (socket says 172.21.15.32:56916) 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.613+0000 7f05e77fe640 1 -- 172.21.15.32:0/1581983136 learned_addr learned my addr 172.21.15.32:0/1581983136 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e6ffd640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e81032e0 0x7f05e819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e77fe640 1 -- 172.21.15.32:0/1581983136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e81032e0 msgr2=0x7f05e819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e77fe640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e81032e0 0x7f05e819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05de7fc640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e8103c00 0x7f05e819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e77fe640 1 -- 172.21.15.32:0/1581983136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e8103c00 msgr2=0x7f05e819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e77fe640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e8103c00 0x7f05e819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e77fe640 1 -- 172.21.15.32:0/1581983136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e8198f90 con 0x7f05e810c620 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e77fe640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e810c620 0x7f05e81986d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f05e000cab0 tx=0x7f05e000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e4ff9640 1 -- 172.21.15.32:0/1581983136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e000ae20 con 0x7f05e810c620 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e4ff9640 1 -- 172.21.15.32:0/1581983136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e000f930 con 0x7f05e810c620 2024-04-16T05:38:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e8199280 con 0x7f05e810c620 2024-04-16T05:38:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05e4ff9640 1 -- 172.21.15.32:0/1581983136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0018b00 con 0x7f05e810c620 2024-04-16T05:38:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.614+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e81a5540 con 0x7f05e810c620 2024-04-16T05:38:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.615+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e806ae30 con 0x7f05e810c620 2024-04-16T05:38:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.616+0000 7f05e4ff9640 1 -- 172.21.15.32:0/1581983136 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05e000fae0 con 0x7f05e810c620 2024-04-16T05:38:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.617+0000 7f05e4ff9640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f05d40a3d20 0x7f05d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.617+0000 7f05e4ff9640 1 -- 172.21.15.32:0/1581983136 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f05e00d12b0 con 0x7f05e810c620 2024-04-16T05:38:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.617+0000 7f05e6ffd640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f05d40a3d20 0x7f05d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.617+0000 7f05e6ffd640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f05d40a3d20 0x7f05d40a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005fd0 tx=0x7f05d8005850 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.621+0000 7f05e4ff9640 1 -- 172.21.15.32:0/1581983136 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05e0014030 con 0x7f05e810c620 2024-04-16T05:38:46.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.852+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05e81994b0 con 0x7f05e810c620 2024-04-16T05:38:46.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.853+0000 7f05e4ff9640 1 -- 172.21.15.32:0/1581983136 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f05e009d5b0 con 0x7f05e810c620 2024-04-16T05:38:46.852 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:38:46.852 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:38:46.852 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:38:46.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f05d40a3d20 msgr2=0x7f05d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f05d40a3d20 0x7f05d40a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005fd0 tx=0x7f05d8005850 comp rx=0 tx=0).stop 2024-04-16T05:38:46.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e810c620 msgr2=0x7f05e81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e810c620 0x7f05e81986d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f05e000cab0 tx=0x7f05e000cf70 comp rx=0 tx=0).stop 2024-04-16T05:38:46.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 shutdown_connections 2024-04-16T05:38:46.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f05d40a3d20 0x7f05d40a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e810c620 0x7f05e81986d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e8103c00 0x7f05e819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 --2- 172.21.15.32:0/1581983136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e81032e0 0x7f05e819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 >> 172.21.15.32:0/1581983136 conn(0x7f05e8078060 msgr2=0x7f05e80fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 shutdown_connections 2024-04-16T05:38:46.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.855+0000 7f05ed4eb640 1 -- 172.21.15.32:0/1581983136 wait complete. 2024-04-16T05:38:46.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:38:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4100748534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c1072f0 msgr2=0x7f5a2c10eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4100748534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c1072f0 0x7f5a2c10eec0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009ca0 tx=0x7f5a2402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4100748534 shutdown_connections 2024-04-16T05:38:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4100748534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c1072f0 0x7f5a2c10eec0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4100748534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a2c102bd0 0x7f5a2c106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4100748534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c102220 0x7f5a2c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4100748534 >> 172.21.15.32:0/4100748534 conn(0x7f5a2c0fdae0 msgr2=0x7f5a2c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4100748534 shutdown_connections 2024-04-16T05:38:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.975+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4100748534 wait complete. 2024-04-16T05:38:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a31b1c640 1 Processor -- start 2024-04-16T05:38:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a31b1c640 1 -- start start 2024-04-16T05:38:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a31b1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c102220 0x7f5a2c074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a31b1c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c1072f0 0x7f5a2c074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a31b1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a2c074d80 0x7f5a2c1a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a31b1c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1123b0 con 0x7f5a2c102220 2024-04-16T05:38:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a31b1c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c112230 con 0x7f5a2c1072f0 2024-04-16T05:38:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a31b1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c112530 con 0x7f5a2c074d80 2024-04-16T05:38:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a2b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c102220 0x7f5a2c074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a2b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c102220 0x7f5a2c074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56930/0 (socket says 172.21.15.32:56930) 2024-04-16T05:38:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/4165689502 learned_addr learned my addr 172.21.15.32:0/4165689502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.976+0000 7f5a2affd640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c1072f0 0x7f5a2c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a2c074d80 0x7f5a2c1a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/4165689502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a2c074d80 msgr2=0x7f5a2c1a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a2c074d80 0x7f5a2c1a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/4165689502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c1072f0 msgr2=0x7f5a2c074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c1072f0 0x7f5a2c074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/4165689502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c1a7180 con 0x7f5a2c102220 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c102220 0x7f5a2c074300 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00d950 tx=0x7f5a1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a28ff9640 1 -- 172.21.15.32:0/4165689502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c017030 con 0x7f5a2c102220 2024-04-16T05:38:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a28ff9640 1 -- 172.21.15.32:0/4165689502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1c004020 con 0x7f5a2c102220 2024-04-16T05:38:46.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4165689502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2c1a7470 con 0x7f5a2c102220 2024-04-16T05:38:46.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a28ff9640 1 -- 172.21.15.32:0/4165689502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0047a0 con 0x7f5a2c102220 2024-04-16T05:38:46.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.977+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4165689502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2c1a79c0 con 0x7f5a2c102220 2024-04-16T05:38:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.979+0000 7f5a28ff9640 1 -- 172.21.15.32:0/4165689502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a1c0041d0 con 0x7f5a2c102220 2024-04-16T05:38:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.979+0000 7f5a347e3740 1 Processor -- start 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.979+0000 7f5a347e3740 1 -- start start 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cabe60 0x558d59cac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 0x558d59cb7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a347e3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558d59c110e0 con 0x558d59cac780 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a347e3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558d59bae520 con 0x558d59cabe60 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a347e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d59c0d0a0 con 0x558d59cb5360 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a28ff9640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a140a3d50 0x7f5a140a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56932/0 (socket says 172.21.15.32:56932) 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/3664171286 learned_addr learned my addr 172.21.15.32:0/3664171286 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a140a3d50 0x7f5a140a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2affd640 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 0x558d59cb7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cabe60 0x558d59cac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a28ff9640 1 -- 172.21.15.32:0/4165689502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f5a1c0d0630 con 0x7f5a2c102220 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/3664171286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 msgr2=0x558d59cb7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 0x558d59cb7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/3664171286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cabe60 msgr2=0x558d59cac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cabe60 0x558d59cac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/3664171286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d59bd68d0 con 0x558d59cac780 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb4e20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c005fd0 tx=0x7f5a1c09c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f45e1cd8cd1a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.980+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a140a3d50 0x7f5a140a6210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009430 tx=0x7f5a240324d0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a11ffb640 1 -- 172.21.15.32:0/3664171286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c09bd70 con 0x558d59cac780 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a11ffb640 1 -- 172.21.15.32:0/3664171286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1c0a05e0 con 0x558d59cac780 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a11ffb640 1 -- 172.21.15.32:0/3664171286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c09bb90 con 0x558d59cac780 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 -- 172.21.15.32:0/3664171286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 msgr2=0x558d59cb4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb4e20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c005fd0 tx=0x7f5a1c09c130 comp rx=0 tx=0).stop 2024-04-16T05:38:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 -- 172.21.15.32:0/3664171286 shutdown_connections 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 0x558d59cb7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb4e20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 --2- 172.21.15.32:0/3664171286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cabe60 0x558d59cac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 -- 172.21.15.32:0/3664171286 >> 172.21.15.32:0/3664171286 conn(0x558d59ca6590 msgr2=0x558d59ca6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 -- 172.21.15.32:0/3664171286 shutdown_connections 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.981+0000 7f5a347e3740 1 -- 172.21.15.32:0/3664171286 wait complete. 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 1 -- start start 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 1 Processor -- start 2024-04-16T05:38:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cb5360 0x558d59cb2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558d59cb7fa0 con 0x558d59cac780 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558d59bca4d0 con 0x558d59cb5360 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d59cb8120 con 0x558d59cabe60 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a347e3740 10 client.0 _ll_register_callbacks cb 0x558d59baf2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a2affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a2affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56946/0 (socket says 172.21.15.32:56946) 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a2affd640 1 -- 172.21.15.32:0/2391003905 learned_addr learned my addr 172.21.15.32:0/2391003905 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.982+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cb5360 0x558d59cb2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a2affd640 1 -- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 msgr2=0x558d59cb2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a2affd640 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a2affd640 1 -- 172.21.15.32:0/2391003905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cb5360 msgr2=0x558d59cb2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a2affd640 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cb5360 0x558d59cb2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a2affd640 1 -- 172.21.15.32:0/2391003905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d599233f0 con 0x558d59cac780 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a2affd640 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb29a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000bdd0 tx=0x7f5a200117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50de4ebda28f14d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a0affd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a0affd640 1 -- 172.21.15.32:0/2391003905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20012650 con 0x558d59cac780 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a0affd640 1 -- 172.21.15.32:0/2391003905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f5a200127f0 con 0x558d59cac780 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a347e3740 1 -- 172.21.15.32:0/2391003905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d59c1eb00 con 0x558d59cac780 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a0affd640 1 -- 172.21.15.32:0/2391003905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a200129f0 con 0x558d59cac780 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a347e3740 10 client.16515 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a347e3740 1 -- 172.21.15.32:0/2391003905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d599ed7d0 con 0x558d59cac780 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a0affd640 1 -- 172.21.15.32:0/2391003905 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558d59c1eb00 con 0x558d59cac780 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a0affd640 1 -- 172.21.15.32:0/2391003905 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5a2001a3d0 con 0x558d59cac780 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.983+0000 7f5a347e3740 10 client.16515 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.984+0000 7f5a347e3740 10 client.16515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.984+0000 7f5a347e3740 20 client.16515 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.984+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558d59cc53a0 0x558d59cc7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.984+0000 7f5a347e3740 4 client.16515 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.984+0000 7f5a347e3740 1 -- 172.21.15.32:0/2391003905 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d59c18970 con 0x558d59cc53a0 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.984+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558d59cc53a0 0x558d59cc7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.984+0000 7f5a0affd640 10 client.16515 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.984+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558d59cc53a0 0x558d59cc7810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d8bddae5ce028f7 server_cookie=5b285e0001c6901a in_seq=0 out_seq=0 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a0affd640 1 -- 172.21.15.32:0/2391003905 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d59c18970 con 0x558d59cc53a0 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a0affd640 10 client.16515 handle_command_reply: tid=0 2024-04-16T05:38:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 client.16515 shutdown 2024-04-16T05:38:46.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 -- 172.21.15.32:0/2391003905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 msgr2=0x558d59cb29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:46.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb29a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000bdd0 tx=0x7f5a200117b0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 -- 172.21.15.32:0/2391003905 shutdown_connections 2024-04-16T05:38:46.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558d59cc53a0 0x558d59cc7810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cb5360 0x558d59cb2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cac780 0x558d59cb29a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a0affd640 0 client.16515 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:38:46.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2391003905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:46.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 -- 172.21.15.32:0/2391003905 >> 172.21.15.32:0/2391003905 conn(0x558d59ca6590 msgr2=0x558d59ca82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:46.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 -- 172.21.15.32:0/2391003905 shutdown_connections 2024-04-16T05:38:46.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 1 -- 172.21.15.32:0/2391003905 wait complete. 2024-04-16T05:38:46.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.985+0000 7f5a347e3740 20 client.16515 trim_cache size 0 max 16384 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a347e3740 1 Processor -- start 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a347e3740 1 -- start start 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 0x558d59ccd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cb5360 0x558d59ccd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a347e3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1b3ec0 con 0x558d59cb5360 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a347e3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1b1790 con 0x558d59cac780 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a347e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1af5e0 con 0x558d59cabe60 2024-04-16T05:38:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cb5360 0x558d59ccd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cb5360 0x558d59ccd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56952/0 (socket says 172.21.15.32:56952) 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2bfff640 1 -- 172.21.15.32:0/1016781300 learned_addr learned my addr 172.21.15.32:0/1016781300 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2affd640 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 0x558d59ccd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2bfff640 1 -- 172.21.15.32:0/1016781300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 msgr2=0x558d59cb4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2bfff640 1 -- 172.21.15.32:0/1016781300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 msgr2=0x558d59ccd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 0x558d59ccd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:46.999+0000 7f5a2bfff640 1 -- 172.21.15.32:0/1016781300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d59cb09b0 con 0x558d59cb5360 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cb5360 0x558d59ccd660 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009ca0 tx=0x7f5a2403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315800684ba64d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a117fa640 1 -- 172.21.15.32:0/1016781300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24004cc0 con 0x558d59cb5360 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a117fa640 1 -- 172.21.15.32:0/1016781300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a24052070 con 0x558d59cb5360 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a117fa640 1 -- 172.21.15.32:0/1016781300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a240042a0 con 0x558d59cb5360 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 -- 172.21.15.32:0/1016781300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cb5360 msgr2=0x558d59ccd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cb5360 0x558d59ccd660 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009ca0 tx=0x7f5a2403a000 comp rx=0 tx=0).stop 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 -- 172.21.15.32:0/1016781300 shutdown_connections 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cb5360 0x558d59ccd660 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 0x558d59ccd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 --2- 172.21.15.32:0/1016781300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cabe60 0x558d59cb4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 -- 172.21.15.32:0/1016781300 >> 172.21.15.32:0/1016781300 conn(0x558d59ca6590 msgr2=0x558d59cad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 -- 172.21.15.32:0/1016781300 shutdown_connections 2024-04-16T05:38:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.000+0000 7f5a347e3740 1 -- 172.21.15.32:0/1016781300 wait complete. 2024-04-16T05:38:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 1 -- start start 2024-04-16T05:38:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 1 Processor -- start 2024-04-16T05:38:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cabe60 0x558d59cb8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 0x558d59cb94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 0x558d59cb99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1afc50 con 0x558d59cabe60 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1b10c0 con 0x558d59cac780 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1b0320 con 0x558d59cb5360 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a347e3740 10 client.0 _ll_register_callbacks cb 0x7f5a20005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a2b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cabe60 0x558d59cb8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a2b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cabe60 0x558d59cb8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56968/0 (socket says 172.21.15.32:56968) 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/2646411781 learned_addr learned my addr 172.21.15.32:0/2646411781 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.001+0000 7f5a2bfff640 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 0x558d59cb99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/2646411781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 msgr2=0x558d59cb99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 0x558d59cb99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a2affd640 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 0x558d59cb94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/2646411781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 msgr2=0x558d59cb94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 0x558d59cb94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a2b7fe640 1 -- 172.21.15.32:0/2646411781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d59d2aa40 con 0x558d59cabe60 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a2b7fe640 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cabe60 0x558d59cb8f70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0045f0 tx=0x7f5a1c09bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439a03265a22a6ec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a0affd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a0affd640 1 -- 172.21.15.32:0/2646411781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c09bd70 con 0x558d59cabe60 2024-04-16T05:38:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a0affd640 1 -- 172.21.15.32:0/2646411781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f5a1c0d0240 con 0x558d59cabe60 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 1 -- 172.21.15.32:0/2646411781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a24052070 con 0x558d59cabe60 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a0affd640 1 -- 172.21.15.32:0/2646411781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c09c930 con 0x558d59cabe60 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 10 client.16521 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 1 -- 172.21.15.32:0/2646411781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d59d2bd80 con 0x558d59cabe60 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a0affd640 1 -- 172.21.15.32:0/2646411781 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a24052070 con 0x558d59cabe60 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a0affd640 1 -- 172.21.15.32:0/2646411781 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5a1c0cdc60 con 0x558d59cabe60 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 10 client.16521 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 10 client.16521 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:38:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 20 client.16521 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558d59d2c070 0x558d59d2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 4 client.16521 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.002+0000 7f5a347e3740 1 -- 172.21.15.32:0/2646411781 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d59b946b0 con 0x558d59d2c070 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.003+0000 7f5a2affd640 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558d59d2c070 0x558d59d2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.003+0000 7f5a0affd640 10 client.16521 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.003+0000 7f5a2affd640 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558d59d2c070 0x558d59d2e460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4158b069c5ea449f server_cookie=dca60512bc99a350 in_seq=0 out_seq=0 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.003+0000 7f5a0affd640 1 -- 172.21.15.32:0/2646411781 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d59b946b0 con 0x558d59d2c070 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.003+0000 7f5a0affd640 10 client.16521 handle_command_reply: tid=0 2024-04-16T05:38:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.003+0000 7f5a347e3740 1 client.16521 shutdown 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 -- 172.21.15.32:0/2646411781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cabe60 msgr2=0x558d59cb8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cabe60 0x558d59cb8f70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0045f0 tx=0x7f5a1c09bc90 comp rx=0 tx=0).stop 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 -- 172.21.15.32:0/2646411781 shutdown_connections 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558d59d2c070 0x558d59d2e460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d59cb5360 0x558d59cb99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d59cac780 0x558d59cb94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 --2- 172.21.15.32:0/2646411781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d59cabe60 0x558d59cb8f70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a0affd640 0 client.16521 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 -- 172.21.15.32:0/2646411781 >> 172.21.15.32:0/2646411781 conn(0x558d59ca6590 msgr2=0x558d59cb2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 -- 172.21.15.32:0/2646411781 shutdown_connections 2024-04-16T05:38:47.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 1 -- 172.21.15.32:0/2646411781 wait complete. 2024-04-16T05:38:47.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a347e3740 20 client.16521 trim_cache size 0 max 16384 2024-04-16T05:38:47.012 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:38:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a140a3d50 msgr2=0x7f5a140a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a140a3d50 0x7f5a140a6210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009430 tx=0x7f5a240324d0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c102220 msgr2=0x7f5a2c074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c102220 0x7f5a2c074300 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00d950 tx=0x7f5a1c00de10 comp rx=0 tx=0).stop 2024-04-16T05:38:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4165689502 shutdown_connections 2024-04-16T05:38:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a2c074d80 0x7f5a2c1a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c1072f0 0x7f5a2c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5a140a3d50 0x7f5a140a6210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 --2- 172.21.15.32:0/4165689502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a2c102220 0x7f5a2c074300 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4165689502 >> 172.21.15.32:0/4165689502 conn(0x7f5a2c0fdae0 msgr2=0x7f5a2c0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4165689502 shutdown_connections 2024-04-16T05:38:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.004+0000 7f5a31b1c640 1 -- 172.21.15.32:0/4165689502 wait complete. 2024-04-16T05:38:47.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:38:47.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 -- 172.21.15.32:0/1487900207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac109500 msgr2=0x7f0cac1110d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 --2- 172.21.15.32:0/1487900207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac109500 0x7f0cac1110d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009900 tx=0x7f0ca402bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 -- 172.21.15.32:0/1487900207 shutdown_connections 2024-04-16T05:38:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 --2- 172.21.15.32:0/1487900207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac109500 0x7f0cac1110d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 --2- 172.21.15.32:0/1487900207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cac104de0 0x7f0cac108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 --2- 172.21.15.32:0/1487900207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cac104430 0x7f0cac104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 -- 172.21.15.32:0/1487900207 >> 172.21.15.32:0/1487900207 conn(0x7f0cac0ffcb0 msgr2=0x7f0cac1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 -- 172.21.15.32:0/1487900207 shutdown_connections 2024-04-16T05:38:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.124+0000 7f0cb4740640 1 -- 172.21.15.32:0/1487900207 wait complete. 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb4740640 1 Processor -- start 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb4740640 1 -- start start 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb4740640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac104430 0x7f0cac1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb4740640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cac109500 0x7f0cac1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb4740640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cac19a980 0x7f0cac19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb4740640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1145c0 con 0x7f0cac104430 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb4740640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac114440 con 0x7f0cac19a980 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb4740640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac114740 con 0x7f0cac109500 2024-04-16T05:38:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb24dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac104430 0x7f0cac1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb24dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac104430 0x7f0cac1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56998/0 (socket says 172.21.15.32:56998) 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.125+0000 7f0cb24dc640 1 -- 172.21.15.32:0/4227421180 learned_addr learned my addr 172.21.15.32:0/4227421180 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb24dc640 1 -- 172.21.15.32:0/4227421180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cac109500 msgr2=0x7f0cac1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cac109500 0x7f0cac1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cac19a980 0x7f0cac19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb24dc640 1 -- 172.21.15.32:0/4227421180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cac19a980 msgr2=0x7f0cac19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cac19a980 0x7f0cac19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb24dc640 1 -- 172.21.15.32:0/4227421180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac19b4a0 con 0x7f0cac104430 2024-04-16T05:38:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac104430 0x7f0cac1a0780 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800eb90 tx=0x7f0ca800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0ca37fe640 1 -- 172.21.15.32:0/4227421180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8004d20 con 0x7f0cac104430 2024-04-16T05:38:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0ca37fe640 1 -- 172.21.15.32:0/4227421180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca800ee90 con 0x7f0cac104430 2024-04-16T05:38:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb4740640 1 -- 172.21.15.32:0/4227421180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cac19b790 con 0x7f0cac104430 2024-04-16T05:38:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0ca37fe640 1 -- 172.21.15.32:0/4227421180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8010830 con 0x7f0cac104430 2024-04-16T05:38:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.126+0000 7f0cb4740640 1 -- 172.21.15.32:0/4227421180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cac1a7940 con 0x7f0cac104430 2024-04-16T05:38:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.128+0000 7f0ca37fe640 1 -- 172.21.15.32:0/4227421180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ca8005020 con 0x7f0cac104430 2024-04-16T05:38:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb7407740 1 Processor -- start 2024-04-16T05:38:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb7407740 1 -- start start 2024-04-16T05:38:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe090240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe090780 0x55ebfe098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe099360 0x55ebfe09b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb7407740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ebfdff50e0 con 0x55ebfe08fe60 2024-04-16T05:38:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb7407740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ebfdf92520 con 0x55ebfe099360 2024-04-16T05:38:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb7407740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ebfdff10a0 con 0x55ebfe090780 2024-04-16T05:38:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb2cdd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe090240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb2cdd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe090240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57006/0 (socket says 172.21.15.32:57006) 2024-04-16T05:38:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/1923307881 learned_addr learned my addr 172.21.15.32:0/1923307881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0ca37fe640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0c980a3d20 0x7f0c980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb1cdb640 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe099360 0x55ebfe09b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0c980a3d20 0x7f0c980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.129+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe090780 0x55ebfe098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0ca37fe640 1 -- 172.21.15.32:0/4227421180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f0ca80d1350 con 0x7f0cac104430 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/1923307881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe090780 msgr2=0x55ebfe098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe090780 0x55ebfe098e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/1923307881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe099360 msgr2=0x55ebfe09b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe099360 0x55ebfe09b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/1923307881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebfdfba8d0 con 0x55ebfe08fe60 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe090240 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4036040 tx=0x7f0ca402ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffa97deba28cefb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0ca0ff9640 1 -- 172.21.15.32:0/1923307881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4005240 con 0x55ebfe08fe60 2024-04-16T05:38:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0ca0ff9640 1 -- 172.21.15.32:0/1923307881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca4004ba0 con 0x55ebfe08fe60 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.130+0000 7f0ca0ff9640 1 -- 172.21.15.32:0/1923307881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4005600 con 0x55ebfe08fe60 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 -- 172.21.15.32:0/1923307881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 msgr2=0x55ebfe090240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0c980a3d20 0x7f0c980a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0ca404b000 tx=0x7f0ca4049000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe090240 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4036040 tx=0x7f0ca402ca90 comp rx=0 tx=0).stop 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 -- 172.21.15.32:0/1923307881 shutdown_connections 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe099360 0x55ebfe09b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe090780 0x55ebfe098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1923307881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe090240 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 -- 172.21.15.32:0/1923307881 >> 172.21.15.32:0/1923307881 conn(0x55ebfe08a590 msgr2=0x55ebfe08a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 -- 172.21.15.32:0/1923307881 shutdown_connections 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 -- 172.21.15.32:0/1923307881 wait complete. 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 -- start start 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.131+0000 7f0cb7407740 1 Processor -- start 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe096460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe099360 0x55ebfe096ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ebfe09bfa0 con 0x55ebfe08fe60 2024-04-16T05:38:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ebfdfae4d0 con 0x55ebfe090780 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ebfe09c120 con 0x55ebfe099360 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 10 client.0 _ll_register_callbacks cb 0x55ebfdf932c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe096460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe096460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57010/0 (socket says 172.21.15.32:57010) 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 -- 172.21.15.32:0/1748082550 learned_addr learned my addr 172.21.15.32:0/1748082550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe099360 0x55ebfe096ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb1cdb640 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 -- 172.21.15.32:0/1748082550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe099360 msgr2=0x55ebfe096ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe099360 0x55ebfe096ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 -- 172.21.15.32:0/1748082550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 msgr2=0x55ebfe0969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 -- 172.21.15.32:0/1748082550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebfdd073f0 con 0x55ebfe08fe60 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe096460 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80049d0 tx=0x7f0ca800a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da92c76c4a7d9b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0c95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0c95ffb640 1 -- 172.21.15.32:0/1748082550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800a250 con 0x55ebfe08fe60 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 1 -- 172.21.15.32:0/1748082550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca4004ba0 con 0x55ebfe08fe60 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0c95ffb640 1 -- 172.21.15.32:0/1748082550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f0ca8016db0 con 0x55ebfe08fe60 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0c95ffb640 1 -- 172.21.15.32:0/1748082550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ca809cf50 con 0x55ebfe08fe60 2024-04-16T05:38:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 10 client.16533 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.132+0000 7f0cb7407740 1 -- 172.21.15.32:0/1748082550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ebfddd17d0 con 0x55ebfe08fe60 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0c95ffb640 1 -- 172.21.15.32:0/1748082550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca4004ba0 con 0x55ebfe08fe60 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0c95ffb640 1 -- 172.21.15.32:0/1748082550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0ca809d1b0 con 0x55ebfe08fe60 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0cb7407740 10 client.16533 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0cb7407740 10 client.16533 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0cb7407740 20 client.16533 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55ebfe0a9340 0x55ebfe0ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0cb7407740 4 client.16533 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0cb7407740 1 -- 172.21.15.32:0/1748082550 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebfe002b00 con 0x55ebfe0a9340 2024-04-16T05:38:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0cb1cdb640 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55ebfe0a9340 0x55ebfe0ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0c95ffb640 10 client.16533 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.133+0000 7f0cb1cdb640 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55ebfe0a9340 0x55ebfe0ab7d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3743c566279b3f1e server_cookie=26f2ba5cf7882122 in_seq=0 out_seq=0 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0c95ffb640 1 -- 172.21.15.32:0/1748082550 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebfe002b00 con 0x55ebfe0a9340 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0c95ffb640 10 client.16533 handle_command_reply: tid=0 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 client.16533 shutdown 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 -- 172.21.15.32:0/1748082550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 msgr2=0x55ebfe096460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe096460 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80049d0 tx=0x7f0ca800a440 comp rx=0 tx=0).stop 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 -- 172.21.15.32:0/1748082550 shutdown_connections 2024-04-16T05:38:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55ebfe0a9340 0x55ebfe0ab7d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe099360 0x55ebfe096ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0c95ffb640 0 client.16533 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 --2- 172.21.15.32:0/1748082550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe08fe60 0x55ebfe096460 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 -- 172.21.15.32:0/1748082550 >> 172.21.15.32:0/1748082550 conn(0x55ebfe08a590 msgr2=0x55ebfe08c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 -- 172.21.15.32:0/1748082550 shutdown_connections 2024-04-16T05:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.134+0000 7f0cb7407740 1 -- 172.21.15.32:0/1748082550 wait complete. 2024-04-16T05:38:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.135+0000 7f0cb7407740 20 client.16533 trim_cache size 0 max 16384 2024-04-16T05:38:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb7407740 1 Processor -- start 2024-04-16T05:38:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb7407740 1 -- start start 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 0x55ebfe098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb7407740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1b3e40 con 0x55ebfe099360 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb7407740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1b1710 con 0x55ebfe090780 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb7407740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1af560 con 0x55ebfe08fe60 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe0b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57014/0 (socket says 172.21.15.32:57014) 2024-04-16T05:38:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/320939961 learned_addr learned my addr 172.21.15.32:0/320939961 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb1cdb640 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/320939961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 msgr2=0x55ebfe098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 0x55ebfe098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/320939961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 msgr2=0x55ebfe0895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/320939961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebfe085c30 con 0x55ebfe099360 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb1cdb640 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.148+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe0b10a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009430 tx=0x7f0ca4009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66b5addaa8aa67b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0c97fff640 1 -- 172.21.15.32:0/320939961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca405b020 con 0x55ebfe099360 2024-04-16T05:38:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0c97fff640 1 -- 172.21.15.32:0/320939961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca40054e0 con 0x55ebfe099360 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0c97fff640 1 -- 172.21.15.32:0/320939961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4004990 con 0x55ebfe099360 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 -- 172.21.15.32:0/320939961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 msgr2=0x55ebfe0b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe0b10a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009430 tx=0x7f0ca4009230 comp rx=0 tx=0).stop 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 -- 172.21.15.32:0/320939961 shutdown_connections 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe0b10a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe0895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 --2- 172.21.15.32:0/320939961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 0x55ebfe098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 -- 172.21.15.32:0/320939961 >> 172.21.15.32:0/320939961 conn(0x55ebfe08a590 msgr2=0x55ebfe088ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 -- 172.21.15.32:0/320939961 shutdown_connections 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 -- 172.21.15.32:0/320939961 wait complete. 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 -- start start 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.149+0000 7f0cb7407740 1 Processor -- start 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 0x55ebfe09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe09d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb7407740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe09d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb7407740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1afbd0 con 0x55ebfe099360 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb7407740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1b1040 con 0x55ebfe090780 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb7407740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1b02a0 con 0x55ebfe08fe60 2024-04-16T05:38:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb7407740 10 client.0 _ll_register_callbacks cb 0x7f0ca80992e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe09d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe09d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57028/0 (socket says 172.21.15.32:57028) 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/2416897834 learned_addr learned my addr 172.21.15.32:0/2416897834 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 0x55ebfe09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb1cdb640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe09d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/2416897834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 msgr2=0x55ebfe09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 0x55ebfe09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 msgr2=0x55ebfe09d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe09d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 -- 172.21.15.32:0/2416897834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebfe10e9c0 con 0x55ebfe099360 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb1cdb640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe09d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 0x55ebfe09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0cb2cdd640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe09d980 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005820 tx=0x7f0ca402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcff6c21562d353d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0c95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.150+0000 7f0c95ffb640 1 -- 172.21.15.32:0/2416897834 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca405b040 con 0x55ebfe099360 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 1 -- 172.21.15.32:0/2416897834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca40054e0 con 0x55ebfe099360 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0c95ffb640 1 -- 172.21.15.32:0/2416897834 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f0ca4004930 con 0x55ebfe099360 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0c95ffb640 1 -- 172.21.15.32:0/2416897834 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ca4004b70 con 0x55ebfe099360 2024-04-16T05:38:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 10 client.16539 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 1 -- 172.21.15.32:0/2416897834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ebfe10fbf0 con 0x55ebfe099360 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0c95ffb640 1 -- 172.21.15.32:0/2416897834 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca40054e0 con 0x55ebfe099360 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0c95ffb640 1 -- 172.21.15.32:0/2416897834 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0ca4060470 con 0x55ebfe099360 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 10 client.16539 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 10 client.16539 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 20 client.16539 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55ebfe10fef0 0x55ebfe1122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 4 client.16539 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb7407740 1 -- 172.21.15.32:0/2416897834 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebfdffc970 con 0x55ebfe10fef0 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.151+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55ebfe10fef0 0x55ebfe1122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb24dc640 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55ebfe10fef0 0x55ebfe1122e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebb58bc589043b81 server_cookie=6f9711100f67d955 in_seq=0 out_seq=0 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0c95ffb640 10 client.16539 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0c95ffb640 1 -- 172.21.15.32:0/2416897834 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebfdffc970 con 0x55ebfe10fef0 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0c95ffb640 10 client.16539 handle_command_reply: tid=0 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 client.16539 shutdown 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 -- 172.21.15.32:0/2416897834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 msgr2=0x55ebfe09d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe09d980 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005820 tx=0x7f0ca402c670 comp rx=0 tx=0).stop 2024-04-16T05:38:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 -- 172.21.15.32:0/2416897834 shutdown_connections 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55ebfe10fef0 0x55ebfe1122e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebfe099360 0x55ebfe09d980 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0c95ffb640 0 client.16539 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebfe090780 0x55ebfe09d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 --2- 172.21.15.32:0/2416897834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebfe08fe60 0x55ebfe09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 -- 172.21.15.32:0/2416897834 >> 172.21.15.32:0/2416897834 conn(0x55ebfe08a590 msgr2=0x55ebfe096cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 -- 172.21.15.32:0/2416897834 shutdown_connections 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 1 -- 172.21.15.32:0/2416897834 wait complete. 2024-04-16T05:38:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.152+0000 7f0cb7407740 20 client.16539 trim_cache size 0 max 16384 2024-04-16T05:38:47.161 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:38:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 -- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0c980a3d20 msgr2=0x7f0c980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0c980a3d20 0x7f0c980a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0ca404b000 tx=0x7f0ca4049000 comp rx=0 tx=0).stop 2024-04-16T05:38:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 -- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac104430 msgr2=0x7f0cac1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac104430 0x7f0cac1a0780 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800eb90 tx=0x7f0ca800c6a0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 -- 172.21.15.32:0/4227421180 shutdown_connections 2024-04-16T05:38:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cac19a980 0x7f0cac19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cac109500 0x7f0cac1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0c980a3d20 0x7f0c980a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 --2- 172.21.15.32:0/4227421180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cac104430 0x7f0cac1a0780 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 -- 172.21.15.32:0/4227421180 >> 172.21.15.32:0/4227421180 conn(0x7f0cac0ffcb0 msgr2=0x7f0cac101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 -- 172.21.15.32:0/4227421180 shutdown_connections 2024-04-16T05:38:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.153+0000 7f0cb4740640 1 -- 172.21.15.32:0/4227421180 wait complete. 2024-04-16T05:38:47.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:38:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 -- 172.21.15.32:0/2496252484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 msgr2=0x7f876410dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 --2- 172.21.15.32:0/2496252484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 0x7f876410dcf0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f875c009ca0 tx=0x7f875c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 -- 172.21.15.32:0/2496252484 shutdown_connections 2024-04-16T05:38:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 --2- 172.21.15.32:0/2496252484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 0x7f876410dcf0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 --2- 172.21.15.32:0/2496252484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8764101250 0x7f87641016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 --2- 172.21.15.32:0/2496252484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8764107040 0x7f8764107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 -- 172.21.15.32:0/2496252484 >> 172.21.15.32:0/2496252484 conn(0x7f8764076d00 msgr2=0x7f8764077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 -- 172.21.15.32:0/2496252484 shutdown_connections 2024-04-16T05:38:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 -- 172.21.15.32:0/2496252484 wait complete. 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.273+0000 7f8769738640 1 Processor -- start 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8769738640 1 -- start start 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8769738640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8764101250 0x7f876419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8769738640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 0x7f876419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8769738640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8764107040 0x7f87641986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8769738640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87641112d0 con 0x7f8764101bf0 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8769738640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8764111150 con 0x7f8764107040 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8769738640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8764111450 con 0x7f8764101250 2024-04-16T05:38:47.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 0x7f876419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 0x7f876419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57046/0 (socket says 172.21.15.32:57046) 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 -- 172.21.15.32:0/1974267821 learned_addr learned my addr 172.21.15.32:0/1974267821 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8762ffd640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8764101250 0x7f876419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87637fe640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8764107040 0x7f87641986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 -- 172.21.15.32:0/1974267821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8764101250 msgr2=0x7f876419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8764101250 0x7f876419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 -- 172.21.15.32:0/1974267821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8764107040 msgr2=0x7f87641986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8764107040 0x7f87641986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 -- 172.21.15.32:0/1974267821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8764198f70 con 0x7f8764101bf0 2024-04-16T05:38:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f8762ffd640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8764101250 0x7f876419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.274+0000 7f87627fc640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 0x7f876419eb30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f875800e9f0 tx=0x7f875800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.275+0000 7f874bfff640 1 -- 172.21.15.32:0/1974267821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875800cd80 con 0x7f8764101bf0 2024-04-16T05:38:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.275+0000 7f874bfff640 1 -- 172.21.15.32:0/1974267821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8758010910 con 0x7f8764101bf0 2024-04-16T05:38:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.275+0000 7f8769738640 1 -- 172.21.15.32:0/1974267821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8764199260 con 0x7f8764101bf0 2024-04-16T05:38:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.275+0000 7f874bfff640 1 -- 172.21.15.32:0/1974267821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758018990 con 0x7f8764101bf0 2024-04-16T05:38:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.275+0000 7f8769738640 1 -- 172.21.15.32:0/1974267821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87641a5570 con 0x7f8764101bf0 2024-04-16T05:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.277+0000 7f874bfff640 1 -- 172.21.15.32:0/1974267821 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8758010ac0 con 0x7f8764101bf0 2024-04-16T05:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.277+0000 7f876c3f5740 1 Processor -- start 2024-04-16T05:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.277+0000 7f876c3f5740 1 -- start start 2024-04-16T05:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 0x55e55ba28240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba30e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba33750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f876c3f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e55b98d0e0 con 0x55e55ba28780 2024-04-16T05:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f876c3f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e55b92a520 con 0x55e55ba31360 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f876c3f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e55b9890a0 con 0x55e55ba27e60 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f874bfff640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f874c0a3d20 0x7f874c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba30e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba30e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57048/0 (socket says 172.21.15.32:57048) 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 -- 172.21.15.32:0/2109443854 learned_addr learned my addr 172.21.15.32:0/2109443854 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f87637fe640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f874c0a3d20 0x7f874c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f87627fc640 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba33750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f87637fe640 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 0x55e55ba28240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f874bfff640 1 -- 172.21.15.32:0/1974267821 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f87580d0fc0 con 0x7f8764101bf0 2024-04-16T05:38:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 -- 172.21.15.32:0/2109443854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 msgr2=0x55e55ba28240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 0x55e55ba28240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 -- 172.21.15.32:0/2109443854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 msgr2=0x55e55ba33750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba33750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 -- 172.21.15.32:0/2109443854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55b9528d0 con 0x55e55ba28780 2024-04-16T05:38:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f87627fc640 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba33750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f87637fe640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f874c0a3d20 0x7f874c0a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f875c004af0 tx=0x7f875c004a40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f8762ffd640 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba30e20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f875400bcc0 tx=0x7f875400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72f21c691eb1ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.278+0000 7f87497fa640 1 -- 172.21.15.32:0/2109443854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754015070 con 0x55e55ba28780 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f87497fa640 1 -- 172.21.15.32:0/2109443854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875401a940 con 0x55e55ba28780 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f87497fa640 1 -- 172.21.15.32:0/2109443854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754011a30 con 0x55e55ba28780 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 -- 172.21.15.32:0/2109443854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 msgr2=0x55e55ba30e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba30e20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f875400bcc0 tx=0x7f875400f730 comp rx=0 tx=0).stop 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 -- 172.21.15.32:0/2109443854 shutdown_connections 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba33750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba30e20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 --2- 172.21.15.32:0/2109443854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 0x55e55ba28240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 -- 172.21.15.32:0/2109443854 >> 172.21.15.32:0/2109443854 conn(0x55e55ba22590 msgr2=0x55e55ba22970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 -- 172.21.15.32:0/2109443854 shutdown_connections 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.279+0000 7f876c3f5740 1 -- 172.21.15.32:0/2109443854 wait complete. 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 1 -- start start 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 1 Processor -- start 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba28780 0x55e55ba2e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba31360 0x55e55ba2ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e55ba340c0 con 0x55e55ba27e60 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e55b9464d0 con 0x55e55ba28780 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e55ba34240 con 0x55e55ba31360 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f876c3f5740 10 client.0 _ll_register_callbacks cb 0x55e55b92b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57062/0 (socket says 172.21.15.32:57062) 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 -- 172.21.15.32:0/3956284574 learned_addr learned my addr 172.21.15.32:0/3956284574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f87627fc640 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba28780 0x55e55ba2e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f87637fe640 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba31360 0x55e55ba2ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 -- 172.21.15.32:0/3956284574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba31360 msgr2=0x55e55ba2ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba31360 0x55e55ba2ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 -- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba28780 msgr2=0x55e55ba2e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba28780 0x55e55ba2e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 -- 172.21.15.32:0/3956284574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55b69f3f0 con 0x55e55ba27e60 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f87627fc640 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba28780 0x55e55ba2e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8762ffd640 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2e240 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8754005850 tx=0x7f875400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9873f5e3e85b27d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.280+0000 7f8746ff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f8746ff5640 1 -- 172.21.15.32:0/3956284574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754015040 con 0x55e55ba27e60 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f8746ff5640 1 -- 172.21.15.32:0/3956284574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f875401a660 con 0x55e55ba27e60 2024-04-16T05:38:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 1 -- 172.21.15.32:0/3956284574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f875401a940 con 0x55e55ba27e60 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f8746ff5640 1 -- 172.21.15.32:0/3956284574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8754030050 con 0x55e55ba27e60 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 10 client.16551 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 1 -- 172.21.15.32:0/3956284574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e55b7697d0 con 0x55e55ba27e60 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f8746ff5640 1 -- 172.21.15.32:0/3956284574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875401a940 con 0x55e55ba27e60 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f8746ff5640 1 -- 172.21.15.32:0/3956284574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f875402b030 con 0x55e55ba27e60 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 10 client.16551 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 20 client.16551 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e55ba41280 0x55e55ba43670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 4 client.16551 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.281+0000 7f876c3f5740 1 -- 172.21.15.32:0/3956284574 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e55b99ab00 con 0x55e55ba41280 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.282+0000 7f87627fc640 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e55ba41280 0x55e55ba43670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.282+0000 7f87627fc640 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e55ba41280 0x55e55ba43670 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f1b6af065c0f7a server_cookie=46f6e9bf5cfd9d99 in_seq=0 out_seq=0 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.282+0000 7f8746ff5640 10 client.16551 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f8746ff5640 1 -- 172.21.15.32:0/3956284574 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e55b99ab00 con 0x55e55ba41280 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f8746ff5640 10 client.16551 handle_command_reply: tid=0 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 client.16551 shutdown 2024-04-16T05:38:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 -- 172.21.15.32:0/3956284574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 msgr2=0x55e55ba2e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2e240 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8754005850 tx=0x7f875400ba10 comp rx=0 tx=0).stop 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 -- 172.21.15.32:0/3956284574 shutdown_connections 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e55ba41280 0x55e55ba43670 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba31360 0x55e55ba2ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f8746ff5640 0 client.16551 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba28780 0x55e55ba2e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3956284574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2e240 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 -- 172.21.15.32:0/3956284574 >> 172.21.15.32:0/3956284574 conn(0x55e55ba22590 msgr2=0x55e55ba242e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 -- 172.21.15.32:0/3956284574 shutdown_connections 2024-04-16T05:38:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 1 -- 172.21.15.32:0/3956284574 wait complete. 2024-04-16T05:38:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.283+0000 7f876c3f5740 20 client.16551 trim_cache size 0 max 16384 2024-04-16T05:38:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f876c3f5740 1 Processor -- start 2024-04-16T05:38:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f876c3f5740 1 -- start start 2024-04-16T05:38:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 0x55e55ba30970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba48d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f876c3f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87641b1bc0 con 0x55e55ba28780 2024-04-16T05:38:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f876c3f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87641af490 con 0x55e55ba31360 2024-04-16T05:38:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f876c3f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87641ad2e0 con 0x55e55ba27e60 2024-04-16T05:38:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.296+0000 7f87627fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87627fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba3d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57068/0 (socket says 172.21.15.32:57068) 2024-04-16T05:38:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87627fc640 1 -- 172.21.15.32:0/4173145971 learned_addr learned my addr 172.21.15.32:0/4173145971 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f8762ffd640 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 0x55e55ba30970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87637fe640 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba48d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87627fc640 1 -- 172.21.15.32:0/4173145971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 msgr2=0x55e55ba30970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87627fc640 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 0x55e55ba30970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87627fc640 1 -- 172.21.15.32:0/4173145971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 msgr2=0x55e55ba48d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87627fc640 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba48d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87627fc640 1 -- 172.21.15.32:0/4173145971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55ba223c0 con 0x55e55ba28780 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87637fe640 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba48d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f87627fc640 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba3d0f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8758018460 tx=0x7f8758013700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8d60504f2a7a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f8748ff9640 1 -- 172.21.15.32:0/4173145971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580a0fd0 con 0x55e55ba28780 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f8748ff9640 1 -- 172.21.15.32:0/4173145971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87580a5890 con 0x55e55ba28780 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.297+0000 7f8748ff9640 1 -- 172.21.15.32:0/4173145971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758014d20 con 0x55e55ba28780 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- 172.21.15.32:0/4173145971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 msgr2=0x55e55ba3d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba3d0f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8758018460 tx=0x7f8758013700 comp rx=0 tx=0).stop 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- 172.21.15.32:0/4173145971 shutdown_connections 2024-04-16T05:38:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55ba48d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba28780 0x55e55ba3d0f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 --2- 172.21.15.32:0/4173145971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba27e60 0x55e55ba30970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- 172.21.15.32:0/4173145971 >> 172.21.15.32:0/4173145971 conn(0x55e55ba22590 msgr2=0x55e55ba21730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- 172.21.15.32:0/4173145971 shutdown_connections 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- 172.21.15.32:0/4173145971 wait complete. 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- start start 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 Processor -- start 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba28780 0x55e55ba2b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55baaa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87641ad950 con 0x55e55ba27e60 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87641aedc0 con 0x55e55ba31360 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87641ae020 con 0x55e55ba28780 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f876c3f5740 10 client.0 _ll_register_callbacks cb 0x7f875400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f8762ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.298+0000 7f8762ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57072/0 (socket says 172.21.15.32:57072) 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8762ffd640 1 -- 172.21.15.32:0/3064362186 learned_addr learned my addr 172.21.15.32:0/3064362186 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f87637fe640 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55baaa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f87627fc640 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba28780 0x55e55ba2b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8762ffd640 1 -- 172.21.15.32:0/3064362186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba28780 msgr2=0x55e55ba2b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8762ffd640 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba28780 0x55e55ba2b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8762ffd640 1 -- 172.21.15.32:0/3064362186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 msgr2=0x55e55baaa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8762ffd640 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55baaa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8762ffd640 1 -- 172.21.15.32:0/3064362186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55ba33fa0 con 0x55e55ba27e60 2024-04-16T05:38:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8762ffd640 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2b2a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f875401aec0 tx=0x7f875401a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cdc7a3a8a7c66e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8746ff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8746ff5640 1 -- 172.21.15.32:0/3064362186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754039060 con 0x55e55ba27e60 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8746ff5640 1 -- 172.21.15.32:0/3064362186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8754005230 con 0x55e55ba27e60 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f876c3f5740 1 -- 172.21.15.32:0/3064362186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87580a5890 con 0x55e55ba27e60 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8746ff5640 1 -- 172.21.15.32:0/3064362186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8754040050 con 0x55e55ba27e60 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f876c3f5740 10 client.16557 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f876c3f5740 1 -- 172.21.15.32:0/3064362186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e55baabd60 con 0x55e55ba27e60 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.299+0000 7f8746ff5640 1 -- 172.21.15.32:0/3064362186 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87580a5890 con 0x55e55ba27e60 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f8746ff5640 1 -- 172.21.15.32:0/3064362186 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f875402b040 con 0x55e55ba27e60 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f876c3f5740 10 client.16557 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f876c3f5740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f876c3f5740 20 client.16557 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e55baac4b0 0x55e55baae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f876c3f5740 4 client.16557 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f876c3f5740 1 -- 172.21.15.32:0/3064362186 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e55ba1dc60 con 0x55e55baac4b0 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f87627fc640 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e55baac4b0 0x55e55baae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f8746ff5640 10 client.16557 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f87627fc640 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e55baac4b0 0x55e55baae8d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28fdebfa4927d19c server_cookie=3aefd4bec331fbf7 in_seq=0 out_seq=0 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f8746ff5640 1 -- 172.21.15.32:0/3064362186 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e55ba1dc60 con 0x55e55baac4b0 2024-04-16T05:38:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f8746ff5640 10 client.16557 handle_command_reply: tid=0 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.300+0000 7f876c3f5740 1 client.16557 shutdown 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 -- 172.21.15.32:0/3064362186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 msgr2=0x55e55ba2b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2b2a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f875401aec0 tx=0x7f875401a710 comp rx=0 tx=0).stop 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 -- 172.21.15.32:0/3064362186 shutdown_connections 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e55ba31360 0x55e55baaa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e55baac4b0 0x55e55baae8d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e55ba28780 0x55e55ba2b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 --2- 172.21.15.32:0/3064362186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e55ba27e60 0x55e55ba2b2a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8746ff5640 0 client.16557 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 -- 172.21.15.32:0/3064362186 >> 172.21.15.32:0/3064362186 conn(0x55e55ba22590 msgr2=0x55e55ba2e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 -- 172.21.15.32:0/3064362186 shutdown_connections 2024-04-16T05:38:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 1 -- 172.21.15.32:0/3064362186 wait complete. 2024-04-16T05:38:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f876c3f5740 20 client.16557 trim_cache size 0 max 16384 2024-04-16T05:38:47.310 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:38:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 -- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f874c0a3d20 msgr2=0x7f874c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f874c0a3d20 0x7f874c0a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f875c004af0 tx=0x7f875c004a40 comp rx=0 tx=0).stop 2024-04-16T05:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 -- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 msgr2=0x7f876419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 0x7f876419eb30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f875800e9f0 tx=0x7f875800eeb0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 -- 172.21.15.32:0/1974267821 shutdown_connections 2024-04-16T05:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8764107040 0x7f87641986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f874c0a3d20 0x7f874c0a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8764101bf0 0x7f876419eb30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 --2- 172.21.15.32:0/1974267821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8764101250 0x7f876419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 -- 172.21.15.32:0/1974267821 >> 172.21.15.32:0/1974267821 conn(0x7f8764076d00 msgr2=0x7f876406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 -- 172.21.15.32:0/1974267821 shutdown_connections 2024-04-16T05:38:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:47.301+0000 7f8769738640 1 -- 172.21.15.32:0/1974267821 wait complete. 2024-04-16T05:38:47.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:47 smithi032 ceph-mon[32351]: pgmap v118: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:47.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:47 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1581983136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:47.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:47 smithi134 ceph-mon[36153]: pgmap v118: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:47.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:47 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1581983136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:47 smithi082 ceph-mon[36195]: pgmap v118: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:47.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:47 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1581983136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:38:48.313 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:38:48.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:38:48.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:38:48.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 -- 172.21.15.32:0/4001416405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c102240 msgr2=0x7f8e7c102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4001416405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c102240 0x7f8e7c102620 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009900 tx=0x7f8e7802bc30 comp rx=0 tx=0).stop 2024-04-16T05:38:48.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 -- 172.21.15.32:0/4001416405 shutdown_connections 2024-04-16T05:38:48.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4001416405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7c107310 0x7f8e7c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4001416405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c102bf0 0x7f8e7c106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4001416405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c102240 0x7f8e7c102620 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 -- 172.21.15.32:0/4001416405 >> 172.21.15.32:0/4001416405 conn(0x7f8e7c079450 msgr2=0x7f8e7c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 -- 172.21.15.32:0/4001416405 shutdown_connections 2024-04-16T05:38:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.438+0000 7f8e8411c640 1 -- 172.21.15.32:0/4001416405 wait complete. 2024-04-16T05:38:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e8411c640 1 Processor -- start 2024-04-16T05:38:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e8411c640 1 -- start start 2024-04-16T05:38:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e8411c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7c102240 0x7f8e7c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e8411c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c107310 0x7f8e7c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e8411c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c198600 0x7f8e7c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e8411c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1123a0 con 0x7f8e7c198600 2024-04-16T05:38:48.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e8411c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c112220 con 0x7f8e7c107310 2024-04-16T05:38:48.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e8411c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c112520 con 0x7f8e7c102240 2024-04-16T05:38:48.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e826b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c198600 0x7f8e7c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e826b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c198600 0x7f8e7c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57086/0 (socket says 172.21.15.32:57086) 2024-04-16T05:38:48.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.439+0000 7f8e826b9640 1 -- 172.21.15.32:0/4014189605 learned_addr learned my addr 172.21.15.32:0/4014189605 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7c102240 0x7f8e7c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e826b9640 1 -- 172.21.15.32:0/4014189605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7c102240 msgr2=0x7f8e7c19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e816b7640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c107310 0x7f8e7c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e826b9640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7c102240 0x7f8e7c19e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e826b9640 1 -- 172.21.15.32:0/4014189605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c107310 msgr2=0x7f8e7c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e826b9640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c107310 0x7f8e7c19e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e826b9640 1 -- 172.21.15.32:0/4014189605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c199120 con 0x7f8e7c198600 2024-04-16T05:38:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e826b9640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c198600 0x7f8e7c198ab0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400cce0 tx=0x7f8e7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e72ffd640 1 -- 172.21.15.32:0/4014189605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400f450 con 0x7f8e7c198600 2024-04-16T05:38:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e72ffd640 1 -- 172.21.15.32:0/4014189605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e7400fad0 con 0x7f8e7c198600 2024-04-16T05:38:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e72ffd640 1 -- 172.21.15.32:0/4014189605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74018c10 con 0x7f8e7c198600 2024-04-16T05:38:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e8411c640 1 -- 172.21.15.32:0/4014189605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c199410 con 0x7f8e7c198600 2024-04-16T05:38:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.440+0000 7f8e8411c640 1 -- 172.21.15.32:0/4014189605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c1a5610 con 0x7f8e7c198600 2024-04-16T05:38:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e72ffd640 1 -- 172.21.15.32:0/4014189605 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e7400f5f0 con 0x7f8e7c198600 2024-04-16T05:38:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e86de6740 1 Processor -- start 2024-04-16T05:38:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e86de6740 1 -- start start 2024-04-16T05:38:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c46ab0 0x55dee6c46e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 0x55dee6c4fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c4ffb0 0x55dee6c523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e86de6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dee6babe90 con 0x55dee6c4ffb0 2024-04-16T05:38:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e86de6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dee6b494c0 con 0x55dee6c473d0 2024-04-16T05:38:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e86de6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dee6ba7e50 con 0x55dee6c46ab0 2024-04-16T05:38:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e816b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c4ffb0 0x55dee6c523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e816b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c4ffb0 0x55dee6c523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57090/0 (socket says 172.21.15.32:57090) 2024-04-16T05:38:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e816b7640 1 -- 172.21.15.32:0/716279689 learned_addr learned my addr 172.21.15.32:0/716279689 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.443+0000 7f8e72ffd640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e680a3d20 0x7f8e680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 0x55dee6c4fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e816b7640 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c46ab0 0x55dee6c46e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e826b9640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e680a3d20 0x7f8e680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e816b7640 1 -- 172.21.15.32:0/716279689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c46ab0 msgr2=0x55dee6c46e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e816b7640 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c46ab0 0x55dee6c46e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e816b7640 1 -- 172.21.15.32:0/716279689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 msgr2=0x55dee6c4fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e816b7640 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 0x55dee6c4fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e816b7640 1 -- 172.21.15.32:0/716279689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee69867e0 con 0x55dee6c4ffb0 2024-04-16T05:38:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e72ffd640 1 -- 172.21.15.32:0/4014189605 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8e740d1210 con 0x7f8e7c198600 2024-04-16T05:38:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e816b7640 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c4ffb0 0x55dee6c523a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00f360 tx=0x7f8e6c00d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7471eebbf8a610 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e826b9640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e680a3d20 0x7f8e680a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8e740ce6d0 tx=0x7f8e7406e8b0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e67fff640 1 -- 172.21.15.32:0/716279689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c005220 con 0x55dee6c4ffb0 2024-04-16T05:38:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e67fff640 1 -- 172.21.15.32:0/716279689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c004c80 con 0x55dee6c4ffb0 2024-04-16T05:38:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.444+0000 7f8e67fff640 1 -- 172.21.15.32:0/716279689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c012a40 con 0x55dee6c4ffb0 2024-04-16T05:38:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 -- 172.21.15.32:0/716279689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c4ffb0 msgr2=0x55dee6c523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c4ffb0 0x55dee6c523a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00f360 tx=0x7f8e6c00d510 comp rx=0 tx=0).stop 2024-04-16T05:38:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 -- 172.21.15.32:0/716279689 shutdown_connections 2024-04-16T05:38:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c4ffb0 0x55dee6c523a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 0x55dee6c4fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 --2- 172.21.15.32:0/716279689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c46ab0 0x55dee6c46e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 -- 172.21.15.32:0/716279689 >> 172.21.15.32:0/716279689 conn(0x55dee6c411e0 msgr2=0x55dee6c415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 -- 172.21.15.32:0/716279689 shutdown_connections 2024-04-16T05:38:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 -- 172.21.15.32:0/716279689 wait complete. 2024-04-16T05:38:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 -- start start 2024-04-16T05:38:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.445+0000 7f8e86de6740 1 Processor -- start 2024-04-16T05:38:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c4d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c4ffb0 0x55dee6c4dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e86de6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dee6c52da0 con 0x55dee6c473d0 2024-04-16T05:38:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e86de6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dee6b34410 con 0x55dee6c46ab0 2024-04-16T05:38:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e86de6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dee6c52f20 con 0x55dee6c4ffb0 2024-04-16T05:38:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e86de6740 10 client.0 _ll_register_callbacks cb 0x55dee6b2fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e81eb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c4d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e81eb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c4d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57104/0 (socket says 172.21.15.32:57104) 2024-04-16T05:38:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e81eb8640 1 -- 172.21.15.32:0/785407818 learned_addr learned my addr 172.21.15.32:0/785407818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e816b7640 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e816b7640 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c4ffb0 0x55dee6c4dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e81eb8640 1 -- 172.21.15.32:0/785407818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c4ffb0 msgr2=0x55dee6c4dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c4ffb0 0x55dee6c4dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e81eb8640 1 -- 172.21.15.32:0/785407818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 msgr2=0x55dee6c4d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e81eb8640 1 -- 172.21.15.32:0/785407818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee69e6f70 con 0x55dee6c473d0 2024-04-16T05:38:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.446+0000 7f8e816b7640 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c4ffb0 0x55dee6c4dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c4d5a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009230 tx=0x7f8e78005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b430f6c624716d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e657fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e657fa640 1 -- 172.21.15.32:0/785407818 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78005030 con 0x55dee6c473d0 2024-04-16T05:38:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e657fa640 1 -- 172.21.15.32:0/785407818 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8e780051d0 con 0x55dee6c473d0 2024-04-16T05:38:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e86de6740 1 -- 172.21.15.32:0/785407818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e6c004c80 con 0x55dee6c473d0 2024-04-16T05:38:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e657fa640 1 -- 172.21.15.32:0/785407818 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8e7804d050 con 0x55dee6c473d0 2024-04-16T05:38:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e86de6740 10 client.16569 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e86de6740 1 -- 172.21.15.32:0/785407818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dee687e170 con 0x55dee6c473d0 2024-04-16T05:38:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e657fa640 1 -- 172.21.15.32:0/785407818 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c004c80 con 0x55dee6c473d0 2024-04-16T05:38:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.447+0000 7f8e657fa640 1 -- 172.21.15.32:0/785407818 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8e78003680 con 0x55dee6c473d0 2024-04-16T05:38:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e86de6740 10 client.16569 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e86de6740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e86de6740 20 client.16569 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e86de6740 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dee6c603c0 0x55dee6c62830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e86de6740 4 client.16569 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e86de6740 1 -- 172.21.15.32:0/785407818 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dee6bb98b0 con 0x55dee6c603c0 2024-04-16T05:38:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e816b7640 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dee6c603c0 0x55dee6c62830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e657fa640 10 client.16569 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.448+0000 7f8e816b7640 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dee6c603c0 0x55dee6c62830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6551b20a17ba4510 server_cookie=da04c8c1f820be33 in_seq=0 out_seq=0 2024-04-16T05:38:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e657fa640 1 -- 172.21.15.32:0/785407818 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dee6bb98b0 con 0x55dee6c603c0 2024-04-16T05:38:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e657fa640 10 client.16569 handle_command_reply: tid=0 2024-04-16T05:38:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 client.16569 shutdown 2024-04-16T05:38:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 -- 172.21.15.32:0/785407818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 msgr2=0x55dee6c4d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c4d5a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009230 tx=0x7f8e78005e40 comp rx=0 tx=0).stop 2024-04-16T05:38:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 -- 172.21.15.32:0/785407818 shutdown_connections 2024-04-16T05:38:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dee6c603c0 0x55dee6c62830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c4ffb0 0x55dee6c4dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c4d5a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 --2- 172.21.15.32:0/785407818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e657fa640 0 client.16569 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 -- 172.21.15.32:0/785407818 >> 172.21.15.32:0/785407818 conn(0x55dee6c411e0 msgr2=0x55dee6c42f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 -- 172.21.15.32:0/785407818 shutdown_connections 2024-04-16T05:38:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 1 -- 172.21.15.32:0/785407818 wait complete. 2024-04-16T05:38:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.449+0000 7f8e86de6740 20 client.16569 trim_cache size 0 max 16384 2024-04-16T05:38:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e86de6740 1 Processor -- start 2024-04-16T05:38:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e86de6740 1 -- start start 2024-04-16T05:38:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 0x55dee6c5bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c68100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e86de6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1b1aa0 con 0x55dee6c473d0 2024-04-16T05:38:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e86de6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1af370 con 0x55dee6c46ab0 2024-04-16T05:38:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e86de6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1ad1c0 con 0x55dee6c411e0 2024-04-16T05:38:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e816b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e816b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c68100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38084/0 (socket says 172.21.15.32:38084) 2024-04-16T05:38:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.467+0000 7f8e816b7640 1 -- 172.21.15.32:0/2234044816 learned_addr learned my addr 172.21.15.32:0/2234044816 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e816b7640 1 -- 172.21.15.32:0/2234044816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 msgr2=0x55dee6c5bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e816b7640 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 0x55dee6c5bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e816b7640 1 -- 172.21.15.32:0/2234044816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 msgr2=0x55dee6c4d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e816b7640 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e816b7640 1 -- 172.21.15.32:0/2234044816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee6c3f660 con 0x55dee6c473d0 2024-04-16T05:38:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e816b7640 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c68100 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00dd40 tx=0x7f8e6c00d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6347c8966e666b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e677fe640 1 -- 172.21.15.32:0/2234044816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c0050e0 con 0x55dee6c473d0 2024-04-16T05:38:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e677fe640 1 -- 172.21.15.32:0/2234044816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c00da40 con 0x55dee6c473d0 2024-04-16T05:38:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e677fe640 1 -- 172.21.15.32:0/2234044816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c012b10 con 0x55dee6c473d0 2024-04-16T05:38:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e86de6740 1 -- 172.21.15.32:0/2234044816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 msgr2=0x55dee6c68100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.468+0000 7f8e86de6740 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c68100 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00dd40 tx=0x7f8e6c00d370 comp rx=0 tx=0).stop 2024-04-16T05:38:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 -- 172.21.15.32:0/2234044816 shutdown_connections 2024-04-16T05:38:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c473d0 0x55dee6c68100 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c46ab0 0x55dee6c4d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 --2- 172.21.15.32:0/2234044816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 0x55dee6c5bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 -- 172.21.15.32:0/2234044816 >> 172.21.15.32:0/2234044816 conn(0x55dee6c4bca0 msgr2=0x55dee6c4d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 -- 172.21.15.32:0/2234044816 shutdown_connections 2024-04-16T05:38:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 -- 172.21.15.32:0/2234044816 wait complete. 2024-04-16T05:38:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 -- start start 2024-04-16T05:38:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 Processor -- start 2024-04-16T05:38:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 0x55dee6c55140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c46ab0 0x55dee6c55680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 0x55dee6c55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1ad830 con 0x55dee6c46ab0 2024-04-16T05:38:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1aeca0 con 0x55dee6c473d0 2024-04-16T05:38:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1adf00 con 0x55dee6c411e0 2024-04-16T05:38:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e86de6740 10 client.0 _ll_register_callbacks cb 0x7f8e78009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.469+0000 7f8e81eb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c46ab0 0x55dee6c55680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e81eb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c46ab0 0x55dee6c55680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38090/0 (socket says 172.21.15.32:38090) 2024-04-16T05:38:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e81eb8640 1 -- 172.21.15.32:0/3179592375 learned_addr learned my addr 172.21.15.32:0/3179592375 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e816b7640 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 0x55dee6c55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e816b7640 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 0x55dee6c55140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e81eb8640 1 -- 172.21.15.32:0/3179592375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 msgr2=0x55dee6c55140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 0x55dee6c55140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e81eb8640 1 -- 172.21.15.32:0/3179592375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 msgr2=0x55dee6c55bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 0x55dee6c55bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e81eb8640 1 -- 172.21.15.32:0/3179592375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee6cc6980 con 0x55dee6c46ab0 2024-04-16T05:38:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e81eb8640 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c46ab0 0x55dee6c55680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8e78003c10 tx=0x7f8e78000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4749e6936cacbc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e657fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e657fa640 1 -- 172.21.15.32:0/3179592375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e780037f0 con 0x55dee6c46ab0 2024-04-16T05:38:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e657fa640 1 -- 172.21.15.32:0/3179592375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8e78003c90 con 0x55dee6c46ab0 2024-04-16T05:38:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e86de6740 1 -- 172.21.15.32:0/3179592375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dee6bb3720 con 0x55dee6c46ab0 2024-04-16T05:38:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e657fa640 1 -- 172.21.15.32:0/3179592375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8e78003eb0 con 0x55dee6c46ab0 2024-04-16T05:38:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e86de6740 10 client.16575 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.470+0000 7f8e86de6740 1 -- 172.21.15.32:0/3179592375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dee6cc7bb0 con 0x55dee6c46ab0 2024-04-16T05:38:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e657fa640 1 -- 172.21.15.32:0/3179592375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dee6bb3720 con 0x55dee6c46ab0 2024-04-16T05:38:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e657fa640 1 -- 172.21.15.32:0/3179592375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8e78051430 con 0x55dee6c46ab0 2024-04-16T05:38:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e86de6740 10 client.16575 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e86de6740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e86de6740 20 client.16575 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e86de6740 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dee6c4ffb0 0x55dee6cca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e86de6740 4 client.16575 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:38:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e86de6740 1 -- 172.21.15.32:0/3179592375 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dee6b2fbe0 con 0x55dee6c4ffb0 2024-04-16T05:38:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e816b7640 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dee6c4ffb0 0x55dee6cca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e657fa640 10 client.16575 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.471+0000 7f8e816b7640 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dee6c4ffb0 0x55dee6cca3c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db1d19233a602f7 server_cookie=e0326f169d70581c in_seq=0 out_seq=0 2024-04-16T05:38:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e657fa640 1 -- 172.21.15.32:0/3179592375 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dee6b2fbe0 con 0x55dee6c4ffb0 2024-04-16T05:38:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e657fa640 10 client.16575 handle_command_reply: tid=0 2024-04-16T05:38:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 client.16575 shutdown 2024-04-16T05:38:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 -- 172.21.15.32:0/3179592375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c46ab0 msgr2=0x55dee6c55680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c46ab0 0x55dee6c55680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8e78003c10 tx=0x7f8e78000bc0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 -- 172.21.15.32:0/3179592375 shutdown_connections 2024-04-16T05:38:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dee6c473d0 0x55dee6c55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dee6c46ab0 0x55dee6c55680 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55dee6c4ffb0 0x55dee6cca3c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 --2- 172.21.15.32:0/3179592375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dee6c411e0 0x55dee6c55140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e657fa640 0 client.16575 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 -- 172.21.15.32:0/3179592375 >> 172.21.15.32:0/3179592375 conn(0x55dee6c4bca0 msgr2=0x55dee6c4d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 -- 172.21.15.32:0/3179592375 shutdown_connections 2024-04-16T05:38:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 1 -- 172.21.15.32:0/3179592375 wait complete. 2024-04-16T05:38:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.473+0000 7f8e86de6740 20 client.16575 trim_cache size 0 max 16384 2024-04-16T05:38:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:38:48.487 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:38:48.487 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "b4c906f4-6458-4472-8f9c-99f3e9af030a", 2024-04-16T05:38:48.488 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:38:48.488 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:38:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 -- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e680a3d20 msgr2=0x7f8e680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e680a3d20 0x7f8e680a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8e740ce6d0 tx=0x7f8e7406e8b0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 -- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c198600 msgr2=0x7f8e7c198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c198600 0x7f8e7c198ab0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400cce0 tx=0x7f8e7400a620 comp rx=0 tx=0).stop 2024-04-16T05:38:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 -- 172.21.15.32:0/4014189605 shutdown_connections 2024-04-16T05:38:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e680a3d20 0x7f8e680a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e7c198600 0x7f8e7c198ab0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c107310 0x7f8e7c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 --2- 172.21.15.32:0/4014189605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7c102240 0x7f8e7c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 -- 172.21.15.32:0/4014189605 >> 172.21.15.32:0/4014189605 conn(0x7f8e7c079450 msgr2=0x7f8e7c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 -- 172.21.15.32:0/4014189605 shutdown_connections 2024-04-16T05:38:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.474+0000 7f8e8411c640 1 -- 172.21.15.32:0/4014189605 wait complete. 2024-04-16T05:38:48.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:38:48.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:38:48.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.600+0000 7f68da277640 1 -- 172.21.15.32:0/597224194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 msgr2=0x7f68d410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.600+0000 7f68da277640 1 --2- 172.21.15.32:0/597224194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 0x7f68d410ddf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009ca0 tx=0x7f68cc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 -- 172.21.15.32:0/597224194 shutdown_connections 2024-04-16T05:38:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 --2- 172.21.15.32:0/597224194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 0x7f68d410ddf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 --2- 172.21.15.32:0/597224194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d4101b00 0x7f68d4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 --2- 172.21.15.32:0/597224194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68d4101150 0x7f68d4101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 -- 172.21.15.32:0/597224194 >> 172.21.15.32:0/597224194 conn(0x7f68d4077f60 msgr2=0x7f68d4078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 -- 172.21.15.32:0/597224194 shutdown_connections 2024-04-16T05:38:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 -- 172.21.15.32:0/597224194 wait complete. 2024-04-16T05:38:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 Processor -- start 2024-04-16T05:38:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 -- start start 2024-04-16T05:38:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.601+0000 7f68da277640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d4101150 0x7f68d419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68da277640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 0x7f68d419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68da277640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68d4198680 0x7f68d4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68da277640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68d41112e0 con 0x7f68d4106220 2024-04-16T05:38:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68da277640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4111160 con 0x7f68d4198680 2024-04-16T05:38:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68da277640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4111460 con 0x7f68d4101150 2024-04-16T05:38:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 0x7f68d419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 0x7f68d419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38106/0 (socket says 172.21.15.32:38106) 2024-04-16T05:38:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 -- 172.21.15.32:0/4239102253 learned_addr learned my addr 172.21.15.32:0/4239102253 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d3fff640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d4101150 0x7f68d419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d8814640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68d4198680 0x7f68d4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 -- 172.21.15.32:0/4239102253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d4101150 msgr2=0x7f68d419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d4101150 0x7f68d419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 -- 172.21.15.32:0/4239102253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68d4198680 msgr2=0x7f68d4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68d4198680 0x7f68d4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 -- 172.21.15.32:0/4239102253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d4199280 con 0x7f68d4106220 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d8814640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68d4198680 0x7f68d4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d37fe640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 0x7f68d419eb00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d740 tx=0x7f68c800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d17fa640 1 -- 172.21.15.32:0/4239102253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8017070 con 0x7f68d4106220 2024-04-16T05:38:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d17fa640 1 -- 172.21.15.32:0/4239102253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c800de50 con 0x7f68d4106220 2024-04-16T05:38:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68da277640 1 -- 172.21.15.32:0/4239102253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d41a5360 con 0x7f68d4106220 2024-04-16T05:38:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68d17fa640 1 -- 172.21.15.32:0/4239102253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8004e00 con 0x7f68d4106220 2024-04-16T05:38:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.602+0000 7f68da277640 1 -- 172.21.15.32:0/4239102253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d4102cf0 con 0x7f68d4106220 2024-04-16T05:38:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68d17fa640 1 -- 172.21.15.32:0/4239102253 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68c8004fa0 con 0x7f68d4106220 2024-04-16T05:38:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68dce59740 1 Processor -- start 2024-04-16T05:38:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68dce59740 1 -- start start 2024-04-16T05:38:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68dce59740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf011e50 0x55a9cf012230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68dce59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 0x55a9cf01ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68dce59740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 0x55a9cf01d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68dce59740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a9cef77180 con 0x55a9cf011e50 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68dce59740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a9cef14360 con 0x55a9cf01b350 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68dce59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a9cef73140 con 0x55a9cf012770 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68d8814640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf011e50 0x55a9cf012230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68d17fa640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68bc0a3d20 0x7f68bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68d8814640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf011e50 0x55a9cf012230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38116/0 (socket says 172.21.15.32:38116) 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68d8814640 1 -- 172.21.15.32:0/1282720409 learned_addr learned my addr 172.21.15.32:0/1282720409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68d3fff640 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 0x55a9cf01ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.605+0000 7f68d8814640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68bc0a3d20 0x7f68bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d8814640 1 -- 172.21.15.32:0/1282720409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 msgr2=0x55a9cf01ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d8814640 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 0x55a9cf01ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d17fa640 1 -- 172.21.15.32:0/4239102253 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f68c80d0510 con 0x7f68d4106220 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d37fe640 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 0x55a9cf01d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d8814640 1 -- 172.21.15.32:0/1282720409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 msgr2=0x55a9cf01d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d8814640 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 0x55a9cf01d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d8814640 1 -- 172.21.15.32:0/1282720409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9ced537d0 con 0x55a9cf011e50 2024-04-16T05:38:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d37fe640 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 0x55a9cf01d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68d8814640 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf011e50 0x55a9cf012230 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f68cc036000 tx=0x7f68cc02cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926daa3471ce726b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68ba7fc640 1 -- 172.21.15.32:0/1282720409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc039040 con 0x55a9cf011e50 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68ba7fc640 1 -- 172.21.15.32:0/1282720409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68cc003b70 con 0x55a9cf011e50 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.606+0000 7f68ba7fc640 1 -- 172.21.15.32:0/1282720409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc034440 con 0x55a9cf011e50 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68d8814640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68bc0a3d20 0x7f68bc0a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68cc04c000 tx=0x7f68cc048000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 -- 172.21.15.32:0/1282720409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf011e50 msgr2=0x55a9cf012230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf011e50 0x55a9cf012230 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f68cc036000 tx=0x7f68cc02cef0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 -- 172.21.15.32:0/1282720409 shutdown_connections 2024-04-16T05:38:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 0x55a9cf01d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 0x55a9cf01ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 --2- 172.21.15.32:0/1282720409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf011e50 0x55a9cf012230 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 -- 172.21.15.32:0/1282720409 >> 172.21.15.32:0/1282720409 conn(0x55a9cf00c580 msgr2=0x55a9cf00c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 -- 172.21.15.32:0/1282720409 shutdown_connections 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 -- 172.21.15.32:0/1282720409 wait complete. 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 -- start start 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.607+0000 7f68dce59740 1 Processor -- start 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68dce59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 0x55a9cf0183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68dce59740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf012770 0x55a9cf018910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68dce59740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf018e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68dce59740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a9cf01e140 con 0x55a9cf01b350 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68dce59740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a9cef302c0 con 0x55a9cf012770 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68dce59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a9cf01e2c0 con 0x55a9cf011e50 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68dce59740 10 client.0 _ll_register_callbacks cb 0x55a9cef17090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf018e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf018e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38130/0 (socket says 172.21.15.32:38130) 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 -- 172.21.15.32:0/800016221 learned_addr learned my addr 172.21.15.32:0/800016221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d3fff640 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 0x55a9cf0183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d37fe640 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf012770 0x55a9cf018910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 -- 172.21.15.32:0/800016221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 msgr2=0x55a9cf0183d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 0x55a9cf0183d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 -- 172.21.15.32:0/800016221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf012770 msgr2=0x55a9cf018910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf012770 0x55a9cf018910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 -- 172.21.15.32:0/800016221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9cec893f0 con 0x55a9cf01b350 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68d8814640 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf018e50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02cfd0 tx=0x7f68cc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60080677dda5c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68b37fe640 1 -- 172.21.15.32:0/800016221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc058020 con 0x55a9cf01b350 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68b37fe640 1 -- 172.21.15.32:0/800016221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f68cc0047b0 con 0x55a9cf01b350 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.608+0000 7f68dce59740 1 -- 172.21.15.32:0/800016221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68cc003b70 con 0x55a9cf01b350 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68b37fe640 1 -- 172.21.15.32:0/800016221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68cc063050 con 0x55a9cf01b350 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68dce59740 10 client.16587 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68dce59740 1 -- 172.21.15.32:0/800016221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9cec490f0 con 0x55a9cf01b350 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68b37fe640 1 -- 172.21.15.32:0/800016221 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68cc003b70 con 0x55a9cf01b350 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68b37fe640 1 -- 172.21.15.32:0/800016221 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68cc034900 con 0x55a9cf01b350 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68dce59740 10 client.16587 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68dce59740 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68dce59740 20 client.16587 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68dce59740 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a9cf02b520 0x55a9cf02d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68dce59740 4 client.16587 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68dce59740 1 -- 172.21.15.32:0/800016221 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9cef84ba0 con 0x55a9cf02b520 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.609+0000 7f68d3fff640 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a9cf02b520 0x55a9cf02d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.610+0000 7f68b37fe640 10 client.16587 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.610+0000 7f68d3fff640 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a9cf02b520 0x55a9cf02d910 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c99d48cfc3e03ef server_cookie=d2d74ff3a02a1fa5 in_seq=0 out_seq=0 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.610+0000 7f68b37fe640 1 -- 172.21.15.32:0/800016221 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9cef84ba0 con 0x55a9cf02b520 2024-04-16T05:38:48.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.610+0000 7f68b37fe640 10 client.16587 handle_command_reply: tid=0 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.610+0000 7f68dce59740 1 client.16587 shutdown 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 -- 172.21.15.32:0/800016221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 msgr2=0x55a9cf018e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf018e50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02cfd0 tx=0x7f68cc0095c0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 -- 172.21.15.32:0/800016221 shutdown_connections 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf018e50 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf012770 0x55a9cf018910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a9cf02b520 0x55a9cf02d910 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 --2- 172.21.15.32:0/800016221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 0x55a9cf0183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68b37fe640 0 client.16587 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 -- 172.21.15.32:0/800016221 >> 172.21.15.32:0/800016221 conn(0x55a9cf00c580 msgr2=0x55a9cf00e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 -- 172.21.15.32:0/800016221 shutdown_connections 2024-04-16T05:38:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 1 -- 172.21.15.32:0/800016221 wait complete. 2024-04-16T05:38:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.611+0000 7f68dce59740 20 client.16587 trim_cache size 0 max 16384 2024-04-16T05:38:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68dce59740 1 Processor -- start 2024-04-16T05:38:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68dce59740 1 -- start start 2024-04-16T05:38:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68dce59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 0x55a9cf01aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68dce59740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf012770 0x55a9cf027000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68dce59740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 0x55a9cf032ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68dce59740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68d41b3e40 con 0x55a9cf012770 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68dce59740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f68d41b1710 con 0x55a9cf01b350 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68dce59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68d41af560 con 0x55a9cf011e50 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68d37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf012770 0x55a9cf027000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68d37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf012770 0x55a9cf027000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38142/0 (socket says 172.21.15.32:38142) 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.628+0000 7f68d37fe640 1 -- 172.21.15.32:0/3976030323 learned_addr learned my addr 172.21.15.32:0/3976030323 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68d37fe640 1 -- 172.21.15.32:0/3976030323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 msgr2=0x55a9cf01aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68d37fe640 1 --2- 172.21.15.32:0/3976030323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 0x55a9cf01aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68d37fe640 1 -- 172.21.15.32:0/3976030323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 msgr2=0x55a9cf032ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68d37fe640 1 --2- 172.21.15.32:0/3976030323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 0x55a9cf032ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68d37fe640 1 -- 172.21.15.32:0/3976030323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9cf02dc80 con 0x55a9cf012770 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68d37fe640 1 --2- 172.21.15.32:0/3976030323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf012770 0x55a9cf027000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f68c80985d0 tx=0x7f68c80a4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70871e22ab7c1924 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68b9ffb640 1 -- 172.21.15.32:0/3976030323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8020040 con 0x55a9cf012770 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68b9ffb640 1 -- 172.21.15.32:0/3976030323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c80a0520 con 0x55a9cf012770 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68b9ffb640 1 -- 172.21.15.32:0/3976030323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c80d0150 con 0x55a9cf012770 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68dce59740 1 -- 172.21.15.32:0/3976030323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf012770 msgr2=0x55a9cf027000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.629+0000 7f68dce59740 1 --2- 172.21.15.32:0/3976030323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf012770 0x55a9cf027000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f68c80985d0 tx=0x7f68c80a4de0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 -- 172.21.15.32:0/3976030323 shutdown_connections 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 --2- 172.21.15.32:0/3976030323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf01b350 0x55a9cf032ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 --2- 172.21.15.32:0/3976030323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf012770 0x55a9cf027000 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 --2- 172.21.15.32:0/3976030323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf011e50 0x55a9cf01aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 -- 172.21.15.32:0/3976030323 >> 172.21.15.32:0/3976030323 conn(0x55a9cf00c580 msgr2=0x55a9cf00b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 -- 172.21.15.32:0/3976030323 shutdown_connections 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 -- 172.21.15.32:0/3976030323 wait complete. 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 -- start start 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 Processor -- start 2024-04-16T05:38:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf011e50 0x55a9cf0238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 0x55a9cf020c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf0211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68d41afbd0 con 0x55a9cf01b350 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f68d41b1040 con 0x55a9cf011e50 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68d41b02a0 con 0x55a9cf012770 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68dce59740 10 client.0 _ll_register_callbacks cb 0x7f68c400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.630+0000 7f68d8814640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf0211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d8814640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf0211d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38150/0 (socket says 172.21.15.32:38150) 2024-04-16T05:38:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d8814640 1 -- 172.21.15.32:0/1576263960 learned_addr learned my addr 172.21.15.32:0/1576263960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d37fe640 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 0x55a9cf020c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d8814640 1 -- 172.21.15.32:0/1576263960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 msgr2=0x55a9cf020c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d3fff640 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf011e50 0x55a9cf0238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d8814640 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 0x55a9cf020c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d8814640 1 -- 172.21.15.32:0/1576263960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf011e50 msgr2=0x55a9cf0238b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d8814640 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf011e50 0x55a9cf0238b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d8814640 1 -- 172.21.15.32:0/1576263960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9cf021a00 con 0x55a9cf01b350 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68d8814640 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf0211d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02cd00 tx=0x7f68cc034950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817d4f9a8df76287 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:38:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68b37fe640 1 -- 172.21.15.32:0/1576263960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc058040 con 0x55a9cf01b350 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68b37fe640 1 -- 172.21.15.32:0/1576263960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f68cc003b80 con 0x55a9cf01b350 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68dce59740 1 -- 172.21.15.32:0/1576263960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c80a0520 con 0x55a9cf01b350 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68b37fe640 1 -- 172.21.15.32:0/1576263960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68cc034440 con 0x55a9cf01b350 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68dce59740 10 client.16593 fetch_fsmap learned FSMap version 25 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68dce59740 1 -- 172.21.15.32:0/1576263960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9cf091760 con 0x55a9cf01b350 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.631+0000 7f68b37fe640 1 -- 172.21.15.32:0/1576263960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c80a0520 con 0x55a9cf01b350 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68b37fe640 1 -- 172.21.15.32:0/1576263960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68cc004520 con 0x55a9cf01b350 2024-04-16T05:38:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68dce59740 10 client.16593 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:38:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68dce59740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:38:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68dce59740 20 client.16593 populate_metadata read hostname 'smithi032' 2024-04-16T05:38:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68dce59740 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a9cf091d10 0x55a9cf094130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:38:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68dce59740 4 client.16593 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:38:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68dce59740 1 -- 172.21.15.32:0/1576263960 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a9cef7ea10 con 0x55a9cf091d10 2024-04-16T05:38:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68d3fff640 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a9cf091d10 0x55a9cf094130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:38:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68b37fe640 10 client.16593 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68d3fff640 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a9cf091d10 0x55a9cf094130 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d5db4ca10b47a6 server_cookie=1c31482a5e2cfa5b in_seq=0 out_seq=0 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68b37fe640 1 -- 172.21.15.32:0/1576263960 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a9cef7ea10 con 0x55a9cf091d10 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68b37fe640 10 client.16593 handle_command_reply: tid=0 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.632+0000 7f68dce59740 1 client.16593 shutdown 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 -- 172.21.15.32:0/1576263960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 msgr2=0x55a9cf0211d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf0211d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02cd00 tx=0x7f68cc034950 comp rx=0 tx=0).stop 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 -- 172.21.15.32:0/1576263960 shutdown_connections 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9cf01b350 0x55a9cf0211d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a9cf091d10 0x55a9cf094130 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9cf012770 0x55a9cf020c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 --2- 172.21.15.32:0/1576263960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9cf011e50 0x55a9cf0238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68b37fe640 0 client.16593 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:38:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 -- 172.21.15.32:0/1576263960 >> 172.21.15.32:0/1576263960 conn(0x55a9cf00c580 msgr2=0x55a9cf00d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 -- 172.21.15.32:0/1576263960 shutdown_connections 2024-04-16T05:38:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 1 -- 172.21.15.32:0/1576263960 wait complete. 2024-04-16T05:38:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68dce59740 20 client.16593 trim_cache size 0 max 16384 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "b4c906f4-6458-4472-8f9c-99f3e9af030a": { 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "b4c906f4-6458-4472-8f9c-99f3e9af030a", 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:38:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:38:48.643 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:38:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 -- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68bc0a3d20 msgr2=0x7f68bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68bc0a3d20 0x7f68bc0a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68cc04c000 tx=0x7f68cc048000 comp rx=0 tx=0).stop 2024-04-16T05:38:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 -- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 msgr2=0x7f68d419eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:38:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 0x7f68d419eb00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d740 tx=0x7f68c800dc00 comp rx=0 tx=0).stop 2024-04-16T05:38:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 -- 172.21.15.32:0/4239102253 shutdown_connections 2024-04-16T05:38:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68d4198680 0x7f68d4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68bc0a3d20 0x7f68bc0a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d4106220 0x7f68d419eb00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 --2- 172.21.15.32:0/4239102253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d4101150 0x7f68d419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:38:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.633+0000 7f68da277640 1 -- 172.21.15.32:0/4239102253 >> 172.21.15.32:0/4239102253 conn(0x7f68d4077f60 msgr2=0x7f68d406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:38:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.634+0000 7f68da277640 1 -- 172.21.15.32:0/4239102253 shutdown_connections 2024-04-16T05:38:48.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:38:48.634+0000 7f68da277640 1 -- 172.21.15.32:0/4239102253 wait complete. 2024-04-16T05:38:48.645 INFO:tasks.cephfs.filesystem:scrub status for tag:b4c906f4-6458-4472-8f9c-99f3e9af030a - {'path': '/', 'tag': 'b4c906f4-6458-4472-8f9c-99f3e9af030a', 'options': 'recursive,force'} 2024-04-16T05:38:49.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:49 smithi134 ceph-mon[36153]: pgmap v119: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:49.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:49 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:38:49.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:49 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:49.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:49 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:38:49.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:49 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:49 smithi082 ceph-mon[36195]: pgmap v119: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:49 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:38:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:49 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:49 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:38:49.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:49 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:49.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:49 smithi032 ceph-mon[32351]: pgmap v119: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:38:49.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:49 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:38:49.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:49 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:49.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:49 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:38:49.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:49 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:38:51.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:51 smithi134 ceph-mon[36153]: pgmap v120: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:51.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:51 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:38:51.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:51 smithi082 ceph-mon[36195]: pgmap v120: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:51.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:51 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:38:51.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:51 smithi032 ceph-mon[32351]: pgmap v120: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:38:51.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:51 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:38:53.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:53 smithi134 ceph-mon[36153]: pgmap v121: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:53.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:53 smithi082 ceph-mon[36195]: pgmap v121: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:53.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:53 smithi032 ceph-mon[32351]: pgmap v121: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:55.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:55 smithi134 ceph-mon[36153]: pgmap v122: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:55.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:55 smithi082 ceph-mon[36195]: pgmap v122: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:55.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:55 smithi032 ceph-mon[32351]: pgmap v122: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:57.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:57 smithi134 ceph-mon[36153]: pgmap v123: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:57.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:57 smithi082 ceph-mon[36195]: pgmap v123: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:57.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:57 smithi032 ceph-mon[32351]: pgmap v123: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:38:59.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:38:59 smithi134 ceph-mon[36153]: pgmap v124: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:59.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:38:59 smithi082 ceph-mon[36195]: pgmap v124: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:38:59.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:38:59 smithi032 ceph-mon[32351]: pgmap v124: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:01.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:01 smithi134 ceph-mon[36153]: pgmap v125: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:01.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:01 smithi082 ceph-mon[36195]: pgmap v125: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:01.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:01 smithi032 ceph-mon[32351]: pgmap v125: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:03 smithi134 ceph-mon[36153]: pgmap v126: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:03.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:03 smithi082 ceph-mon[36195]: pgmap v126: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:03.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:03 smithi032 ceph-mon[32351]: pgmap v126: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:39:05.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:05 smithi134 ceph-mon[36153]: pgmap v127: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:39:05.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:05 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:39:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:05 smithi082 ceph-mon[36195]: pgmap v127: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:39:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:05 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:39:05.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:05 smithi032 ceph-mon[32351]: pgmap v127: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:39:05.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:05 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:39:06.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:06 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:39:06.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:06 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:39:06.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:06 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:39:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:06 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:39:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:06 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:39:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:06 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:39:06.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:06 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:39:06.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:06 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:39:06.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:06 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:39:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:07 smithi134 ceph-mon[36153]: pgmap v128: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:39:07.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:07 smithi082 ceph-mon[36195]: pgmap v128: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:39:07.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:07 smithi032 ceph-mon[32351]: pgmap v128: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-16T05:39:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:09 smithi134 ceph-mon[36153]: pgmap v129: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:39:09.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:09 smithi082 ceph-mon[36195]: pgmap v129: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:39:09.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:09 smithi032 ceph-mon[32351]: pgmap v129: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:39:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:11 smithi134 ceph-mon[36153]: pgmap v130: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:39:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:11 smithi082 ceph-mon[36195]: pgmap v130: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:39:11.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:11 smithi032 ceph-mon[32351]: pgmap v130: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:39:13.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:13 smithi134 ceph-mon[36153]: pgmap v131: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:39:13.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:13 smithi082 ceph-mon[36195]: pgmap v131: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:39:13.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:13 smithi032 ceph-mon[32351]: pgmap v131: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:39:15.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:15 smithi134 ceph-mon[36153]: pgmap v132: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:39:15.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:15 smithi082 ceph-mon[36195]: pgmap v132: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:39:15.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:15 smithi032 ceph-mon[32351]: pgmap v132: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:39:17.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:17 smithi134 ceph-mon[36153]: pgmap v133: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:39:17.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:17 smithi082 ceph-mon[36195]: pgmap v133: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:39:17.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:17 smithi032 ceph-mon[32351]: pgmap v133: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-16T05:39:18.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:39:18.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:39:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 -- 172.21.15.32:0/741937306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102c00 msgr2=0x7f8420106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 --2- 172.21.15.32:0/741937306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102c00 0x7f8420106c40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8410009900 tx=0x7f841002bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 -- 172.21.15.32:0/741937306 shutdown_connections 2024-04-16T05:39:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 --2- 172.21.15.32:0/741937306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8420107320 0x7f842010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 --2- 172.21.15.32:0/741937306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102c00 0x7f8420106c40 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 --2- 172.21.15.32:0/741937306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8420102250 0x7f8420102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 -- 172.21.15.32:0/741937306 >> 172.21.15.32:0/741937306 conn(0x7f8420079400 msgr2=0x7f8420079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 -- 172.21.15.32:0/741937306 shutdown_connections 2024-04-16T05:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.768+0000 7f8427367640 1 -- 172.21.15.32:0/741937306 wait complete. 2024-04-16T05:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8427367640 1 Processor -- start 2024-04-16T05:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8427367640 1 -- start start 2024-04-16T05:39:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8427367640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102250 0x7f842019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8427367640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8420107320 0x7f842019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8427367640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8420198710 0x7f8420198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8427367640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84201123b0 con 0x7f8420102250 2024-04-16T05:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8427367640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8420112230 con 0x7f8420107320 2024-04-16T05:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8427367640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8420112530 con 0x7f8420198710 2024-04-16T05:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425103640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102250 0x7f842019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425103640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102250 0x7f842019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45582/0 (socket says 172.21.15.32:45582) 2024-04-16T05:39:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425103640 1 -- 172.21.15.32:0/795896847 learned_addr learned my addr 172.21.15.32:0/795896847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425904640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8420198710 0x7f8420198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8424902640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8420107320 0x7f842019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425103640 1 -- 172.21.15.32:0/795896847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8420198710 msgr2=0x7f8420198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425103640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8420198710 0x7f8420198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425103640 1 -- 172.21.15.32:0/795896847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8420107320 msgr2=0x7f842019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425103640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8420107320 0x7f842019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.769+0000 7f8425103640 1 -- 172.21.15.32:0/795896847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8420199310 con 0x7f8420102250 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.770+0000 7f8425103640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102250 0x7f842019e650 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f841c00d950 tx=0x7f841c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.770+0000 7f84167fc640 1 -- 172.21.15.32:0/795896847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c017030 con 0x7f8420102250 2024-04-16T05:39:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.770+0000 7f84167fc640 1 -- 172.21.15.32:0/795896847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c004020 con 0x7f8420102250 2024-04-16T05:39:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.770+0000 7f8427367640 1 -- 172.21.15.32:0/795896847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84201a53f0 con 0x7f8420102250 2024-04-16T05:39:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.770+0000 7f84167fc640 1 -- 172.21.15.32:0/795896847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c0047a0 con 0x7f8420102250 2024-04-16T05:39:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.770+0000 7f8427367640 1 -- 172.21.15.32:0/795896847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201a57a0 con 0x7f8420102250 2024-04-16T05:39:18.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.772+0000 7f84167fc640 1 -- 172.21.15.32:0/795896847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f841c0041d0 con 0x7f8420102250 2024-04-16T05:39:18.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f842a02c740 1 Processor -- start 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f842a02c740 1 -- start start 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f842a02c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fc6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f842a02c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f842a02c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fd1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f842a02c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556595f2b180 con 0x556595fc5e50 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f842a02c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556595ec8360 con 0x556595fc6770 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f842a02c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556595f27140 con 0x556595fcf350 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f8425904640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fc6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f84167fc640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f840c0a3d20 0x7f840c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f8425904640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fc6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45596/0 (socket says 172.21.15.32:45596) 2024-04-16T05:39:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f8425904640 1 -- 172.21.15.32:0/3127682093 learned_addr learned my addr 172.21.15.32:0/3127682093 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f8424902640 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fd1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f8425103640 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.773+0000 7f8425904640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f840c0a3d20 0x7f840c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f8425904640 1 -- 172.21.15.32:0/3127682093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 msgr2=0x556595fd1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f8425904640 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fd1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f84167fc640 1 -- 172.21.15.32:0/795896847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f841c0d05b0 con 0x7f8420102250 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f8425904640 1 -- 172.21.15.32:0/3127682093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 msgr2=0x556595fcee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f8425904640 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f8425904640 1 -- 172.21.15.32:0/3127682093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556595d077d0 con 0x556595fc5e50 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f8425103640 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f8425904640 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fc6230 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8418014a60 tx=0x7f8418014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f693fc4680e2cef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f840b7fe640 1 -- 172.21.15.32:0/3127682093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841801ec50 con 0x556595fc5e50 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f840b7fe640 1 -- 172.21.15.32:0/3127682093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8418023070 con 0x556595fc5e50 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.774+0000 7f840b7fe640 1 -- 172.21.15.32:0/3127682093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84180056b0 con 0x556595fc5e50 2024-04-16T05:39:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 -- 172.21.15.32:0/3127682093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 msgr2=0x556595fc6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f8425904640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f840c0a3d20 0x7f840c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8418022000 tx=0x7f841801e000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fc6230 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8418014a60 tx=0x7f8418014f20 comp rx=0 tx=0).stop 2024-04-16T05:39:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 -- 172.21.15.32:0/3127682093 shutdown_connections 2024-04-16T05:39:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fd1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 --2- 172.21.15.32:0/3127682093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fc6230 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 -- 172.21.15.32:0/3127682093 >> 172.21.15.32:0/3127682093 conn(0x556595fc0580 msgr2=0x556595fc0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 -- 172.21.15.32:0/3127682093 shutdown_connections 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 -- 172.21.15.32:0/3127682093 wait complete. 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 -- start start 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.775+0000 7f842a02c740 1 Processor -- start 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f842a02c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fcc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f842a02c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f842a02c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f842a02c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556595fd2140 con 0x556595fc5e50 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f842a02c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556595ee42c0 con 0x556595fc6770 2024-04-16T05:39:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f842a02c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556595fd22c0 con 0x556595fcf350 2024-04-16T05:39:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f842a02c740 10 client.0 _ll_register_callbacks cb 0x556595ecb090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fcc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fcc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45606/0 (socket says 172.21.15.32:45606) 2024-04-16T05:39:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 -- 172.21.15.32:0/1432039884 learned_addr learned my addr 172.21.15.32:0/1432039884 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8424902640 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425904640 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 -- 172.21.15.32:0/1432039884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 msgr2=0x556595fccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 -- 172.21.15.32:0/1432039884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 msgr2=0x556595fcc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 -- 172.21.15.32:0/1432039884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556595c3d3f0 con 0x556595fc5e50 2024-04-16T05:39:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8425103640 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fcc340 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f841c009610 tx=0x7f841c066c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff3ef540ffee223 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.776+0000 7f8408ff9640 1 -- 172.21.15.32:0/1432039884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c098450 con 0x556595fc5e50 2024-04-16T05:39:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f8408ff9640 1 -- 172.21.15.32:0/1432039884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f841c0d01b0 con 0x556595fc5e50 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 1 -- 172.21.15.32:0/1432039884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8418023070 con 0x556595fc5e50 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f8408ff9640 1 -- 172.21.15.32:0/1432039884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f841c066cc0 con 0x556595fc5e50 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 10 client.16605 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 1 -- 172.21.15.32:0/1432039884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556595bfd0f0 con 0x556595fc5e50 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f8408ff9640 1 -- 172.21.15.32:0/1432039884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8418023070 con 0x556595fc5e50 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f8408ff9640 1 -- 172.21.15.32:0/1432039884 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f841c0cdbe0 con 0x556595fc5e50 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 10 client.16605 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 20 client.16605 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556595fdf600 0x556595fe1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 4 client.16605 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.777+0000 7f842a02c740 1 -- 172.21.15.32:0/1432039884 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556595f38ba0 con 0x556595fdf600 2024-04-16T05:39:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.778+0000 7f8424902640 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556595fdf600 0x556595fe1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.778+0000 7f8424902640 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556595fdf600 0x556595fe1a70 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876d90c3b6c23978 server_cookie=16dd8474f6d081ed in_seq=0 out_seq=0 2024-04-16T05:39:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.778+0000 7f8408ff9640 10 client.16605 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f8408ff9640 1 -- 172.21.15.32:0/1432039884 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556595f38ba0 con 0x556595fdf600 2024-04-16T05:39:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f8408ff9640 10 client.16605 handle_command_reply: tid=0 2024-04-16T05:39:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 client.16605 shutdown 2024-04-16T05:39:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 -- 172.21.15.32:0/1432039884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 msgr2=0x556595fcc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fcc340 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f841c009610 tx=0x7f841c066c50 comp rx=0 tx=0).stop 2024-04-16T05:39:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 -- 172.21.15.32:0/1432039884 shutdown_connections 2024-04-16T05:39:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556595fdf600 0x556595fe1a70 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fcc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 --2- 172.21.15.32:0/1432039884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc5e50 0x556595fcc340 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f8408ff9640 0 client.16605 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.779+0000 7f842a02c740 1 -- 172.21.15.32:0/1432039884 >> 172.21.15.32:0/1432039884 conn(0x556595fc0580 msgr2=0x556595fc22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.780+0000 7f842a02c740 1 -- 172.21.15.32:0/1432039884 shutdown_connections 2024-04-16T05:39:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.780+0000 7f842a02c740 1 -- 172.21.15.32:0/1432039884 wait complete. 2024-04-16T05:39:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.780+0000 7f842a02c740 20 client.16605 trim_cache size 0 max 16384 2024-04-16T05:39:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.797+0000 7f842a02c740 1 Processor -- start 2024-04-16T05:39:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f842a02c740 1 -- start start 2024-04-16T05:39:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f842a02c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fc5e50 0x556595fdb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f842a02c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f842a02c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fcf350 0x556595fc20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f842a02c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84201b1d20 con 0x556595fcf350 2024-04-16T05:39:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f842a02c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84201af5f0 con 0x556595fc6770 2024-04-16T05:39:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f842a02c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84201ad440 con 0x556595fc5e50 2024-04-16T05:39:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fcf350 0x556595fc20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fcf350 0x556595fc20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45620/0 (socket says 172.21.15.32:45620) 2024-04-16T05:39:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 -- 172.21.15.32:0/1410297088 learned_addr learned my addr 172.21.15.32:0/1410297088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8424902640 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425103640 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fc5e50 0x556595fdb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 -- 172.21.15.32:0/1410297088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fc5e50 msgr2=0x556595fdb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fc5e50 0x556595fdb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 -- 172.21.15.32:0/1410297088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 msgr2=0x556595fccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 -- 172.21.15.32:0/1410297088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556595fbbe40 con 0x556595fcf350 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.798+0000 7f8425904640 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fcf350 0x556595fc20c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8418014930 tx=0x7f8418014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba13a4e7d7132f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f840affd640 1 -- 172.21.15.32:0/1410297088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418012c70 con 0x556595fcf350 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f840affd640 1 -- 172.21.15.32:0/1410297088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8418023050 con 0x556595fcf350 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f840affd640 1 -- 172.21.15.32:0/1410297088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84180049a0 con 0x556595fcf350 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 -- 172.21.15.32:0/1410297088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fcf350 msgr2=0x556595fc20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fcf350 0x556595fc20c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8418014930 tx=0x7f8418014f90 comp rx=0 tx=0).stop 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 -- 172.21.15.32:0/1410297088 shutdown_connections 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fcf350 0x556595fc20c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc6770 0x556595fccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 --2- 172.21.15.32:0/1410297088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fc5e50 0x556595fdb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 -- 172.21.15.32:0/1410297088 >> 172.21.15.32:0/1410297088 conn(0x556595fc0580 msgr2=0x556595fc19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 -- 172.21.15.32:0/1410297088 shutdown_connections 2024-04-16T05:39:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.799+0000 7f842a02c740 1 -- 172.21.15.32:0/1410297088 wait complete. 2024-04-16T05:39:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 1 -- start start 2024-04-16T05:39:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 1 Processor -- start 2024-04-16T05:39:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc5e50 0x556595fd7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc6770 0x556595fd7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fd4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84201adab0 con 0x556595fc6770 2024-04-16T05:39:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84201aef20 con 0x556595fc5e50 2024-04-16T05:39:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84201ae180 con 0x556595fcf350 2024-04-16T05:39:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f842a02c740 10 client.0 _ll_register_callbacks cb 0x7f841c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f8424902640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc6770 0x556595fd7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f8424902640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc6770 0x556595fd7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45622/0 (socket says 172.21.15.32:45622) 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f8424902640 1 -- 172.21.15.32:0/552544503 learned_addr learned my addr 172.21.15.32:0/552544503 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.800+0000 7f8425103640 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc5e50 0x556595fd7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8425904640 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fd4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8424902640 1 -- 172.21.15.32:0/552544503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 msgr2=0x556595fd4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8424902640 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fd4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8424902640 1 -- 172.21.15.32:0/552544503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc5e50 msgr2=0x556595fd7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8424902640 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc5e50 0x556595fd7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8424902640 1 -- 172.21.15.32:0/552544503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556595fd5310 con 0x556595fc6770 2024-04-16T05:39:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8424902640 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc6770 0x556595fd7ac0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f842006b700 tx=0x7f8410000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f225565619fcc160 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8408ff9640 1 -- 172.21.15.32:0/552544503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410004240 con 0x556595fc6770 2024-04-16T05:39:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8408ff9640 1 -- 172.21.15.32:0/552544503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f84100043e0 con 0x556595fc6770 2024-04-16T05:39:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f842a02c740 1 -- 172.21.15.32:0/552544503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8418023050 con 0x556595fc6770 2024-04-16T05:39:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8408ff9640 1 -- 172.21.15.32:0/552544503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8410057050 con 0x556595fc6770 2024-04-16T05:39:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f842a02c740 10 client.16611 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f842a02c740 1 -- 172.21.15.32:0/552544503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556596045910 con 0x556595fc6770 2024-04-16T05:39:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8408ff9640 1 -- 172.21.15.32:0/552544503 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8418023050 con 0x556595fc6770 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.801+0000 7f8408ff9640 1 -- 172.21.15.32:0/552544503 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8410046070 con 0x556595fc6770 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 10 client.16611 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 20 client.16611 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556596046090 0x5565960484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 4 client.16611 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 -- 172.21.15.32:0/552544503 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556595f32a10 con 0x556596046090 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f8425103640 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556596046090 0x5565960484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f8425103640 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556596046090 0x5565960484b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7878f7a0b9e446c server_cookie=809ac8546143757 in_seq=0 out_seq=0 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f8408ff9640 10 client.16611 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f8408ff9640 1 -- 172.21.15.32:0/552544503 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556595f32a10 con 0x556596046090 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f8408ff9640 10 client.16611 handle_command_reply: tid=0 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 client.16611 shutdown 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 -- 172.21.15.32:0/552544503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc6770 msgr2=0x556595fd7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc6770 0x556595fd7ac0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f842006b700 tx=0x7f8410000bc0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 -- 172.21.15.32:0/552544503 shutdown_connections 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x556596046090 0x5565960484b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556595fcf350 0x556595fd4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556595fc6770 0x556595fd7ac0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 --2- 172.21.15.32:0/552544503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556595fc5e50 0x556595fd7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f8408ff9640 0 client.16611 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.802+0000 7f842a02c740 1 -- 172.21.15.32:0/552544503 >> 172.21.15.32:0/552544503 conn(0x556595fc0580 msgr2=0x556595fbeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f842a02c740 1 -- 172.21.15.32:0/552544503 shutdown_connections 2024-04-16T05:39:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f842a02c740 1 -- 172.21.15.32:0/552544503 wait complete. 2024-04-16T05:39:18.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f842a02c740 20 client.16611 trim_cache size 0 max 16384 2024-04-16T05:39:18.812 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:39:18.812 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:39:18.812 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:39:18.812 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:39:18.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 -- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f840c0a3d20 msgr2=0x7f840c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f840c0a3d20 0x7f840c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8418022000 tx=0x7f841801e000 comp rx=0 tx=0).stop 2024-04-16T05:39:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 -- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102250 msgr2=0x7f842019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102250 0x7f842019e650 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f841c00d950 tx=0x7f841c00de10 comp rx=0 tx=0).stop 2024-04-16T05:39:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 -- 172.21.15.32:0/795896847 shutdown_connections 2024-04-16T05:39:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8420198710 0x7f8420198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8420107320 0x7f842019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f840c0a3d20 0x7f840c0a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 --2- 172.21.15.32:0/795896847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8420102250 0x7f842019e650 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 -- 172.21.15.32:0/795896847 >> 172.21.15.32:0/795896847 conn(0x7f8420079400 msgr2=0x7f84201002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:18.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 -- 172.21.15.32:0/795896847 shutdown_connections 2024-04-16T05:39:18.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.803+0000 7f8427367640 1 -- 172.21.15.32:0/795896847 wait complete. 2024-04-16T05:39:18.815 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:39:18.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:39:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.922+0000 7fe4674d9640 1 -- 172.21.15.32:0/2798512310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 msgr2=0x7fe46010c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.922+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2798512310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 0x7fe46010c060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe454009900 tx=0x7fe45402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.922+0000 7fe4674d9640 1 -- 172.21.15.32:0/2798512310 shutdown_connections 2024-04-16T05:39:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.922+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2798512310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe46010c630 0x7fe46010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.922+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2798512310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 0x7fe46010c060 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.922+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2798512310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe460103260 0x7fe460103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.922+0000 7fe4674d9640 1 -- 172.21.15.32:0/2798512310 >> 172.21.15.32:0/2798512310 conn(0x7fe460077280 msgr2=0x7fe460077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 -- 172.21.15.32:0/2798512310 shutdown_connections 2024-04-16T05:39:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 -- 172.21.15.32:0/2798512310 wait complete. 2024-04-16T05:39:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 Processor -- start 2024-04-16T05:39:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 -- start start 2024-04-16T05:39:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe460103260 0x7fe46019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 0x7fe46019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe46010c630 0x7fe460198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4601123a0 con 0x7fe460103b80 2024-04-16T05:39:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe460112220 con 0x7fe460103260 2024-04-16T05:39:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.923+0000 7fe4674d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe460112520 con 0x7fe46010c630 2024-04-16T05:39:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 0x7fe46019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 0x7fe46019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45638/0 (socket says 172.21.15.32:45638) 2024-04-16T05:39:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 -- 172.21.15.32:0/2011849442 learned_addr learned my addr 172.21.15.32:0/2011849442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe465275640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe460103260 0x7fe46019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe465a76640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe46010c630 0x7fe460198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 -- 172.21.15.32:0/2011849442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe46010c630 msgr2=0x7fe460198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe46010c630 0x7fe460198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 -- 172.21.15.32:0/2011849442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe460103260 msgr2=0x7fe46019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe460103260 0x7fe46019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 -- 172.21.15.32:0/2011849442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe460198fc0 con 0x7fe460103b80 2024-04-16T05:39:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe464a74640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 0x7fe46019eb80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe454005940 tx=0x7fe45402cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:18.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe45e7fc640 1 -- 172.21.15.32:0/2011849442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454041070 con 0x7fe460103b80 2024-04-16T05:39:18.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe460199250 con 0x7fe460103b80 2024-04-16T05:39:18.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe45e7fc640 1 -- 172.21.15.32:0/2011849442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe454004450 con 0x7fe460103b80 2024-04-16T05:39:18.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe45e7fc640 1 -- 172.21.15.32:0/2011849442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454034440 con 0x7fe460103b80 2024-04-16T05:39:18.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.924+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4601a53e0 con 0x7fe460103b80 2024-04-16T05:39:18.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.925+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46006ae80 con 0x7fe460103b80 2024-04-16T05:39:18.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.926+0000 7fe45e7fc640 1 -- 172.21.15.32:0/2011849442 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe454044070 con 0x7fe460103b80 2024-04-16T05:39:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.927+0000 7fe45e7fc640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe4480a3d20 0x7fe4480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.927+0000 7fe465275640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe4480a3d20 0x7fe4480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.927+0000 7fe45e7fc640 1 -- 172.21.15.32:0/2011849442 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fe4540f2850 con 0x7fe460103b80 2024-04-16T05:39:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.927+0000 7fe465275640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe4480a3d20 0x7fe4480a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe45800bbe0 tx=0x7fe45800b390 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:18.931+0000 7fe45e7fc640 1 -- 172.21.15.32:0/2011849442 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4540becc0 con 0x7fe460103b80 2024-04-16T05:39:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.158+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe460069270 con 0x7fe460103b80 2024-04-16T05:39:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.158+0000 7fe45e7fc640 1 -- 172.21.15.32:0/2011849442 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7fe460069270 con 0x7fe460103b80 2024-04-16T05:39:19.157 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:39:19.158 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:39:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:39:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe4480a3d20 msgr2=0x7fe4480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe4480a3d20 0x7fe4480a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe45800bbe0 tx=0x7fe45800b390 comp rx=0 tx=0).stop 2024-04-16T05:39:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 msgr2=0x7fe46019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 0x7fe46019eb80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe454005940 tx=0x7fe45402cd10 comp rx=0 tx=0).stop 2024-04-16T05:39:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 shutdown_connections 2024-04-16T05:39:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe46010c630 0x7fe460198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe4480a3d20 0x7fe4480a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe460103b80 0x7fe46019eb80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 --2- 172.21.15.32:0/2011849442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe460103260 0x7fe46019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 >> 172.21.15.32:0/2011849442 conn(0x7fe460077280 msgr2=0x7fe4600fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 shutdown_connections 2024-04-16T05:39:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.160+0000 7fe4674d9640 1 -- 172.21.15.32:0/2011849442 wait complete. 2024-04-16T05:39:19.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:39:19.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 -- 172.21.15.32:0/2957178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 msgr2=0x7ff65c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 --2- 172.21.15.32:0/2957178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 0x7ff65c10ea80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff654009ca0 tx=0x7ff65402c230 comp rx=0 tx=0).stop 2024-04-16T05:39:19.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 -- 172.21.15.32:0/2957178 shutdown_connections 2024-04-16T05:39:19.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 --2- 172.21.15.32:0/2957178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 0x7ff65c10ea80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 --2- 172.21.15.32:0/2957178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c103bc0 0x7ff65c10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 --2- 172.21.15.32:0/2957178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65c1032a0 0x7ff65c103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 -- 172.21.15.32:0/2957178 >> 172.21.15.32:0/2957178 conn(0x7ff65c077280 msgr2=0x7ff65c077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 -- 172.21.15.32:0/2957178 shutdown_connections 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 -- 172.21.15.32:0/2957178 wait complete. 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.280+0000 7ff66251b640 1 Processor -- start 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff66251b640 1 -- start start 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff66251b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c1032a0 0x7ff65c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff66251b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65c103bc0 0x7ff65c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff66251b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 0x7ff65c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff66251b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c1123a0 con 0x7ff65c10c5e0 2024-04-16T05:39:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff66251b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c112220 con 0x7ff65c103bc0 2024-04-16T05:39:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff66251b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c112520 con 0x7ff65c1032a0 2024-04-16T05:39:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff660ab8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 0x7ff65c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff660ab8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 0x7ff65c198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45656/0 (socket says 172.21.15.32:45656) 2024-04-16T05:39:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff660ab8640 1 -- 172.21.15.32:0/3987884770 learned_addr learned my addr 172.21.15.32:0/3987884770 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff65bfff640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c1032a0 0x7ff65c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65c103bc0 0x7ff65c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff660ab8640 1 -- 172.21.15.32:0/3987884770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c1032a0 msgr2=0x7ff65c19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff660ab8640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c1032a0 0x7ff65c19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff660ab8640 1 -- 172.21.15.32:0/3987884770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65c103bc0 msgr2=0x7ff65c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff660ab8640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65c103bc0 0x7ff65c19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.281+0000 7ff660ab8640 1 -- 172.21.15.32:0/3987884770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c198d80 con 0x7ff65c10c5e0 2024-04-16T05:39:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.282+0000 7ff660ab8640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 0x7ff65c198600 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff65402c840 tx=0x7ff654003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.282+0000 7ff6597fa640 1 -- 172.21.15.32:0/3987884770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654039040 con 0x7ff65c10c5e0 2024-04-16T05:39:19.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.282+0000 7ff6597fa640 1 -- 172.21.15.32:0/3987884770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff654044070 con 0x7ff65c10c5e0 2024-04-16T05:39:19.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.282+0000 7ff66251b640 1 -- 172.21.15.32:0/3987884770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c199010 con 0x7ff65c10c5e0 2024-04-16T05:39:19.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.282+0000 7ff6597fa640 1 -- 172.21.15.32:0/3987884770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540045a0 con 0x7ff65c10c5e0 2024-04-16T05:39:19.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.282+0000 7ff66251b640 1 -- 172.21.15.32:0/3987884770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c1a5310 con 0x7ff65c10c5e0 2024-04-16T05:39:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6597fa640 1 -- 172.21.15.32:0/3987884770 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff654032920 con 0x7ff65c10c5e0 2024-04-16T05:39:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6651d6740 1 Processor -- start 2024-04-16T05:39:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6651d6740 1 -- start start 2024-04-16T05:39:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc4e60 0x562f0fdc5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdce360 0x562f0fdd0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6651d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562f0fd2a0e0 con 0x562f0fdc4e60 2024-04-16T05:39:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6651d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562f0fcc7520 con 0x562f0fdc5780 2024-04-16T05:39:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.284+0000 7ff6651d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562f0fd260a0 con 0x562f0fdce360 2024-04-16T05:39:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc4e60 0x562f0fdc5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc4e60 0x562f0fdc5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45666/0 (socket says 172.21.15.32:45666) 2024-04-16T05:39:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 -- 172.21.15.32:0/3397731690 learned_addr learned my addr 172.21.15.32:0/3397731690 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65bfff640 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdce360 0x562f0fdd0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff6597fa640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff6440a3d20 0x7ff6440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 -- 172.21.15.32:0/3397731690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdce360 msgr2=0x562f0fdd0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdce360 0x562f0fdd0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 -- 172.21.15.32:0/3397731690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 msgr2=0x562f0fdcde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 -- 172.21.15.32:0/3397731690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0fcef8d0 con 0x562f0fdc4e60 2024-04-16T05:39:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65bfff640 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff660ab8640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff6440a3d20 0x7ff6440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc4e60 0x562f0fdc5240 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff65000cf90 tx=0x7ff65000c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cea273f2dc63cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff6597fa640 1 -- 172.21.15.32:0/3987884770 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff6540f3190 con 0x7ff65c10c5e0 2024-04-16T05:39:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff6427fc640 1 -- 172.21.15.32:0/3397731690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650018030 con 0x562f0fdc4e60 2024-04-16T05:39:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff6427fc640 1 -- 172.21.15.32:0/3397731690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff650010e80 con 0x562f0fdc4e60 2024-04-16T05:39:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.285+0000 7ff6427fc640 1 -- 172.21.15.32:0/3397731690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650012660 con 0x562f0fdc4e60 2024-04-16T05:39:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 -- 172.21.15.32:0/3397731690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc4e60 msgr2=0x562f0fdc5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc4e60 0x562f0fdc5240 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff65000cf90 tx=0x7ff65000c4f0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff660ab8640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff6440a3d20 0x7ff6440a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6540bb360 tx=0x7ff6540919f0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 -- 172.21.15.32:0/3397731690 shutdown_connections 2024-04-16T05:39:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdce360 0x562f0fdd0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 --2- 172.21.15.32:0/3397731690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc4e60 0x562f0fdc5240 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 -- 172.21.15.32:0/3397731690 >> 172.21.15.32:0/3397731690 conn(0x562f0fdbf590 msgr2=0x562f0fdbf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 -- 172.21.15.32:0/3397731690 shutdown_connections 2024-04-16T05:39:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 -- 172.21.15.32:0/3397731690 wait complete. 2024-04-16T05:39:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.286+0000 7ff6651d6740 1 -- start start 2024-04-16T05:39:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff6651d6740 1 Processor -- start 2024-04-16T05:39:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdcb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdcb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdcbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff6651d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562f0fdd1060 con 0x562f0fdc5780 2024-04-16T05:39:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff6651d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562f0fce34d0 con 0x562f0fdce360 2024-04-16T05:39:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff6651d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562f0fdd11e0 con 0x562f0fdc4e60 2024-04-16T05:39:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff6651d6740 10 client.0 _ll_register_callbacks cb 0x562f0fcc82c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdcb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdcb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45678/0 (socket says 172.21.15.32:45678) 2024-04-16T05:39:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 -- 172.21.15.32:0/1193730552 learned_addr learned my addr 172.21.15.32:0/1193730552 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdcb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdcbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 -- 172.21.15.32:0/1193730552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 msgr2=0x562f0fdcb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdcb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 -- 172.21.15.32:0/1193730552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 msgr2=0x562f0fdcbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdcbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 -- 172.21.15.32:0/1193730552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f0fa3c3f0 con 0x562f0fdc5780 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdcbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff65bfff640 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdcb920 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0127f0 tx=0x7ff64c012cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc6f3c98045f749 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.287+0000 7ff63b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff63b7fe640 1 -- 172.21.15.32:0/1193730552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c010d60 con 0x562f0fdc5780 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff63b7fe640 1 -- 172.21.15.32:0/1193730552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff64c005020 con 0x562f0fdc5780 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 1 -- 172.21.15.32:0/1193730552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f0fd37b00 con 0x562f0fdc5780 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff63b7fe640 1 -- 172.21.15.32:0/1193730552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff64c025050 con 0x562f0fdc5780 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 10 client.16629 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 1 -- 172.21.15.32:0/1193730552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f0fb067d0 con 0x562f0fdc5780 2024-04-16T05:39:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff63b7fe640 1 -- 172.21.15.32:0/1193730552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562f0fd37b00 con 0x562f0fdc5780 2024-04-16T05:39:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff63b7fe640 1 -- 172.21.15.32:0/1193730552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff64c01a430 con 0x562f0fdc5780 2024-04-16T05:39:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 10 client.16629 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 10 client.16629 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:39:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 20 client.16629 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x562f0fdde440 0x562f0fde0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 4 client.16629 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.288+0000 7ff6651d6740 1 -- 172.21.15.32:0/1193730552 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f0fd31970 con 0x562f0fdde440 2024-04-16T05:39:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.289+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x562f0fdde440 0x562f0fde0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.289+0000 7ff63b7fe640 10 client.16629 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:39:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.289+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x562f0fdde440 0x562f0fde0860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a59945254cc2890 server_cookie=c918a4c2bd72867c in_seq=0 out_seq=0 2024-04-16T05:39:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff63b7fe640 1 -- 172.21.15.32:0/1193730552 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f0fd31970 con 0x562f0fdde440 2024-04-16T05:39:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff63b7fe640 10 client.16629 handle_command_reply: tid=0 2024-04-16T05:39:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 client.16629 shutdown 2024-04-16T05:39:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 -- 172.21.15.32:0/1193730552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 msgr2=0x562f0fdcb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdcb920 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0127f0 tx=0x7ff64c012cb0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 -- 172.21.15.32:0/1193730552 shutdown_connections 2024-04-16T05:39:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x562f0fdde440 0x562f0fde0860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdcbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdcb920 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 --2- 172.21.15.32:0/1193730552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdcb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff63b7fe640 0 client.16629 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:39:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 -- 172.21.15.32:0/1193730552 >> 172.21.15.32:0/1193730552 conn(0x562f0fdbf590 msgr2=0x562f0fdc12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 -- 172.21.15.32:0/1193730552 shutdown_connections 2024-04-16T05:39:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 1 -- 172.21.15.32:0/1193730552 wait complete. 2024-04-16T05:39:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.290+0000 7ff6651d6740 20 client.16629 trim_cache size 0 max 16384 2024-04-16T05:39:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff6651d6740 1 Processor -- start 2024-04-16T05:39:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff6651d6740 1 -- start start 2024-04-16T05:39:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdda030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdce360 0x562f0fde6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff6651d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c1b1ba0 con 0x562f0fdce360 2024-04-16T05:39:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff6651d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c1af470 con 0x562f0fdc5780 2024-04-16T05:39:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff6651d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c1ad2c0 con 0x562f0fdc4e60 2024-04-16T05:39:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdce360 0x562f0fde6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdce360 0x562f0fde6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45680/0 (socket says 172.21.15.32:45680) 2024-04-16T05:39:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 -- 172.21.15.32:0/3392998947 learned_addr learned my addr 172.21.15.32:0/3392998947 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdda030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65bfff640 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 -- 172.21.15.32:0/3392998947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 msgr2=0x562f0fdda030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdda030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 -- 172.21.15.32:0/3392998947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 msgr2=0x562f0fdcbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 -- 172.21.15.32:0/3392998947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0fdd0a80 con 0x562f0fdce360 2024-04-16T05:39:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.304+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdce360 0x562f0fde6130 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff65000c820 tx=0x7ff650004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deaaf273d783842 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff641ffb640 1 -- 172.21.15.32:0/3392998947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650018040 con 0x562f0fdce360 2024-04-16T05:39:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff641ffb640 1 -- 172.21.15.32:0/3392998947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff650028850 con 0x562f0fdce360 2024-04-16T05:39:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff641ffb640 1 -- 172.21.15.32:0/3392998947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650012af0 con 0x562f0fdce360 2024-04-16T05:39:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 -- 172.21.15.32:0/3392998947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdce360 msgr2=0x562f0fde6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdce360 0x562f0fde6130 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff65000c820 tx=0x7ff650004a90 comp rx=0 tx=0).stop 2024-04-16T05:39:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 -- 172.21.15.32:0/3392998947 shutdown_connections 2024-04-16T05:39:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdce360 0x562f0fde6130 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdc5780 0x562f0fdcbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 --2- 172.21.15.32:0/3392998947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdda030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 -- 172.21.15.32:0/3392998947 >> 172.21.15.32:0/3392998947 conn(0x562f0fdbf590 msgr2=0x562f0fdcd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 -- 172.21.15.32:0/3392998947 shutdown_connections 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.305+0000 7ff6651d6740 1 -- 172.21.15.32:0/3392998947 wait complete. 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 1 -- start start 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 1 Processor -- start 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdc8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdc8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdc8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c1ad930 con 0x562f0fdc5780 2024-04-16T05:39:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c1aeda0 con 0x562f0fdce360 2024-04-16T05:39:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c1ae000 con 0x562f0fdc4e60 2024-04-16T05:39:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff6651d6740 10 client.0 _ll_register_callbacks cb 0x7ff64c00a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdc8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdc8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45696/0 (socket says 172.21.15.32:45696) 2024-04-16T05:39:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65bfff640 1 -- 172.21.15.32:0/571458747 learned_addr learned my addr 172.21.15.32:0/571458747 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdc8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdc8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65bfff640 1 -- 172.21.15.32:0/571458747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 msgr2=0x562f0fdc8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65bfff640 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdc8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65bfff640 1 -- 172.21.15.32:0/571458747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 msgr2=0x562f0fdc8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65bfff640 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdc8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.306+0000 7ff65bfff640 1 -- 172.21.15.32:0/571458747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f0fe43a50 con 0x562f0fdc5780 2024-04-16T05:39:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff65bfff640 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdc8690 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x562f0fdcc9b0 tx=0x7ff64c01a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f2f1a2e9e41089 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff63b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff63b7fe640 1 -- 172.21.15.32:0/571458747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c0107a0 con 0x562f0fdc5780 2024-04-16T05:39:19.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff63b7fe640 1 -- 172.21.15.32:0/571458747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff64c01aac0 con 0x562f0fdc5780 2024-04-16T05:39:19.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 1 -- 172.21.15.32:0/571458747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff650028850 con 0x562f0fdc5780 2024-04-16T05:39:19.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff63b7fe640 1 -- 172.21.15.32:0/571458747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff64c03a050 con 0x562f0fdc5780 2024-04-16T05:39:19.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 10 client.16635 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:19.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 1 -- 172.21.15.32:0/571458747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f0fe44d60 con 0x562f0fdc5780 2024-04-16T05:39:19.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff63b7fe640 1 -- 172.21.15.32:0/571458747 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff650028850 con 0x562f0fdc5780 2024-04-16T05:39:19.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff63b7fe640 1 -- 172.21.15.32:0/571458747 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff64c005480 con 0x562f0fdc5780 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 10 client.16635 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 10 client.16635 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 20 client.16635 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x562f0fe45510 0x562f0fe47930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 4 client.16635 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.307+0000 7ff6651d6740 1 -- 172.21.15.32:0/571458747 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f0fcad6b0 con 0x562f0fe45510 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.308+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x562f0fe45510 0x562f0fe47930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.308+0000 7ff65b7fe640 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x562f0fe45510 0x562f0fe47930 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1deb3b5e058d840 server_cookie=c68d1468163c7a37 in_seq=0 out_seq=0 2024-04-16T05:39:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.308+0000 7ff63b7fe640 10 client.16635 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:39:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.308+0000 7ff63b7fe640 1 -- 172.21.15.32:0/571458747 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f0fcad6b0 con 0x562f0fe45510 2024-04-16T05:39:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.308+0000 7ff63b7fe640 10 client.16635 handle_command_reply: tid=0 2024-04-16T05:39:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.308+0000 7ff6651d6740 1 client.16635 shutdown 2024-04-16T05:39:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.308+0000 7ff6651d6740 1 -- 172.21.15.32:0/571458747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 msgr2=0x562f0fdc8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.308+0000 7ff6651d6740 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdc8690 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x562f0fdcc9b0 tx=0x7ff64c01a660 comp rx=0 tx=0).stop 2024-04-16T05:39:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 1 -- 172.21.15.32:0/571458747 shutdown_connections 2024-04-16T05:39:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x562f0fe45510 0x562f0fe47930 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f0fdce360 0x562f0fdc8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f0fdc5780 0x562f0fdc8690 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff63b7fe640 0 client.16635 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:39:19.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 1 --2- 172.21.15.32:0/571458747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f0fdc4e60 0x562f0fdc8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 1 -- 172.21.15.32:0/571458747 >> 172.21.15.32:0/571458747 conn(0x562f0fdbf590 msgr2=0x562f0fdcc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 1 -- 172.21.15.32:0/571458747 shutdown_connections 2024-04-16T05:39:19.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 1 -- 172.21.15.32:0/571458747 wait complete. 2024-04-16T05:39:19.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff6651d6740 20 client.16635 trim_cache size 0 max 16384 2024-04-16T05:39:19.322 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:39:19.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 -- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff6440a3d20 msgr2=0x7ff6440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff6440a3d20 0x7ff6440a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6540bb360 tx=0x7ff6540919f0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 -- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 msgr2=0x7ff65c198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 0x7ff65c198600 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff65402c840 tx=0x7ff654003c20 comp rx=0 tx=0).stop 2024-04-16T05:39:19.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 -- 172.21.15.32:0/3987884770 shutdown_connections 2024-04-16T05:39:19.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff6440a3d20 0x7ff6440a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65c10c5e0 0x7ff65c198600 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65c103bc0 0x7ff65c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 --2- 172.21.15.32:0/3987884770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c1032a0 0x7ff65c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 -- 172.21.15.32:0/3987884770 >> 172.21.15.32:0/3987884770 conn(0x7ff65c077280 msgr2=0x7ff65c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 -- 172.21.15.32:0/3987884770 shutdown_connections 2024-04-16T05:39:19.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.309+0000 7ff66251b640 1 -- 172.21.15.32:0/3987884770 wait complete. 2024-04-16T05:39:19.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:39:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 -- 172.21.15.32:0/850736256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 msgr2=0x7fc4c8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 --2- 172.21.15.32:0/850736256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 0x7fc4c8106a40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400ade0 tx=0x7fc4c402cfd0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 -- 172.21.15.32:0/850736256 shutdown_connections 2024-04-16T05:39:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 --2- 172.21.15.32:0/850736256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4c8101060 0x7fc4c810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 --2- 172.21.15.32:0/850736256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4c81006a0 0x7fc4c8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 --2- 172.21.15.32:0/850736256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 0x7fc4c8106a40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 -- 172.21.15.32:0/850736256 >> 172.21.15.32:0/850736256 conn(0x7fc4c8077410 msgr2=0x7fc4c80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 -- 172.21.15.32:0/850736256 shutdown_connections 2024-04-16T05:39:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.436+0000 7fc4d004a640 1 -- 172.21.15.32:0/850736256 wait complete. 2024-04-16T05:39:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4d004a640 1 Processor -- start 2024-04-16T05:39:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4d004a640 1 -- start start 2024-04-16T05:39:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4d004a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4c81006a0 0x7fc4c819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4d004a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4c8101060 0x7fc4c819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4d004a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 0x7fc4c81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4d004a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81101f0 con 0x7fc4c8106660 2024-04-16T05:39:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4d004a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8110070 con 0x7fc4c8101060 2024-04-16T05:39:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4d004a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8110370 con 0x7fc4c81006a0 2024-04-16T05:39:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4ce5e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 0x7fc4c81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4ce5e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 0x7fc4c81987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45722/0 (socket says 172.21.15.32:45722) 2024-04-16T05:39:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4ce5e7640 1 -- 172.21.15.32:0/2184910789 learned_addr learned my addr 172.21.15.32:0/2184910789 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4c81006a0 0x7fc4c819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.437+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4c8101060 0x7fc4c819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4ce5e7640 1 -- 172.21.15.32:0/2184910789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4c81006a0 msgr2=0x7fc4c819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4ce5e7640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4c81006a0 0x7fc4c819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4ce5e7640 1 -- 172.21.15.32:0/2184910789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4c8101060 msgr2=0x7fc4c819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4ce5e7640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4c8101060 0x7fc4c819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4ce5e7640 1 -- 172.21.15.32:0/2184910789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c8198f70 con 0x7fc4c8106660 2024-04-16T05:39:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4c8101060 0x7fc4c819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4ce5e7640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 0x7fc4c81987a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000c970 tx=0x7fc4c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4beffd640 1 -- 172.21.15.32:0/2184910789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0007bb0 con 0x7fc4c8106660 2024-04-16T05:39:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4beffd640 1 -- 172.21.15.32:0/2184910789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0007d50 con 0x7fc4c8106660 2024-04-16T05:39:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4d004a640 1 -- 172.21.15.32:0/2184910789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c8199260 con 0x7fc4c8106660 2024-04-16T05:39:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4beffd640 1 -- 172.21.15.32:0/2184910789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c000f450 con 0x7fc4c8106660 2024-04-16T05:39:19.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.438+0000 7fc4d004a640 1 -- 172.21.15.32:0/2184910789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c81a5480 con 0x7fc4c8106660 2024-04-16T05:39:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.440+0000 7fc4beffd640 1 -- 172.21.15.32:0/2184910789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc4c0004f60 con 0x7fc4c8106660 2024-04-16T05:39:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.440+0000 7fc4d2c32740 1 Processor -- start 2024-04-16T05:39:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4d2c32740 1 -- start start 2024-04-16T05:39:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa89f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa97230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa97770 0x56538aa99b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56538a9ef0e0 con 0x56538aa8a4b0 2024-04-16T05:39:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56538a98c520 con 0x56538aa97770 2024-04-16T05:39:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56538a9eb0a0 con 0x56538aa89b90 2024-04-16T05:39:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4cdde6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa97230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4beffd640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4b40a3d20 0x7fc4b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4cd5e5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa89f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4cdde6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa97230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45736/0 (socket says 172.21.15.32:45736) 2024-04-16T05:39:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4cdde6640 1 -- 172.21.15.32:0/2841016256 learned_addr learned my addr 172.21.15.32:0/2841016256 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.441+0000 7fc4ce5e7640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4b40a3d20 0x7fc4b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa97770 0x56538aa99b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4cdde6640 1 -- 172.21.15.32:0/2841016256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 msgr2=0x56538aa89f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa89f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4beffd640 1 -- 172.21.15.32:0/2184910789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fc4c00d07f0 con 0x7fc4c8106660 2024-04-16T05:39:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4cdde6640 1 -- 172.21.15.32:0/2841016256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa97770 msgr2=0x56538aa99b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa97770 0x56538aa99b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4cdde6640 1 -- 172.21.15.32:0/2841016256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56538a9b48d0 con 0x56538aa8a4b0 2024-04-16T05:39:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa89f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:39:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa97230 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400adb0 tx=0x7fc4c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05e79d193a8d148 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4ce5e7640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4b40a3d20 0x7fc4b40a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000c940 tx=0x7fc4c0004940 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4b3fff640 1 -- 172.21.15.32:0/2841016256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4043020 con 0x56538aa8a4b0 2024-04-16T05:39:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4b3fff640 1 -- 172.21.15.32:0/2841016256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4033e20 con 0x56538aa8a4b0 2024-04-16T05:39:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.442+0000 7fc4b3fff640 1 -- 172.21.15.32:0/2841016256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c403a040 con 0x56538aa8a4b0 2024-04-16T05:39:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2841016256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 msgr2=0x56538aa97230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa97230 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400adb0 tx=0x7fc4c4005fd0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2841016256 shutdown_connections 2024-04-16T05:39:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa97770 0x56538aa99b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa97230 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/2841016256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa89f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2841016256 >> 172.21.15.32:0/2841016256 conn(0x56538aa84590 msgr2=0x56538aa84970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2841016256 shutdown_connections 2024-04-16T05:39:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2841016256 wait complete. 2024-04-16T05:39:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 -- start start 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.443+0000 7fc4d2c32740 1 Processor -- start 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa89b90 0x56538aa94790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa94cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 0x56538aa95210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56538aa9a4d0 con 0x56538aa8a4b0 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56538a9a84d0 con 0x56538aa89b90 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56538aa9a650 con 0x56538aa97770 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4d2c32740 10 client.0 _ll_register_callbacks cb 0x56538a98d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa94cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa94cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45744/0 (socket says 172.21.15.32:45744) 2024-04-16T05:39:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 -- 172.21.15.32:0/401514476 learned_addr learned my addr 172.21.15.32:0/401514476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 0x56538aa95210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa89b90 0x56538aa94790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 -- 172.21.15.32:0/401514476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 msgr2=0x56538aa95210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 0x56538aa95210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 -- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa89b90 msgr2=0x56538aa94790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa89b90 0x56538aa94790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 -- 172.21.15.32:0/401514476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56538a7013f0 con 0x56538aa8a4b0 2024-04-16T05:39:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa89b90 0x56538aa94790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa94cd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4005e30 tx=0x7fc4c4033480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36abcbe1b8d4a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.444+0000 7fc4b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4b17fa640 1 -- 172.21.15.32:0/401514476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4043030 con 0x56538aa8a4b0 2024-04-16T05:39:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4b17fa640 1 -- 172.21.15.32:0/401514476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fc4c402dd90 con 0x56538aa8a4b0 2024-04-16T05:39:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 1 -- 172.21.15.32:0/401514476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c4033e20 con 0x56538aa8a4b0 2024-04-16T05:39:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4b17fa640 1 -- 172.21.15.32:0/401514476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4c4058050 con 0x56538aa8a4b0 2024-04-16T05:39:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 10 client.16647 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 1 -- 172.21.15.32:0/401514476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56538a7cb7d0 con 0x56538aa8a4b0 2024-04-16T05:39:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4b17fa640 1 -- 172.21.15.32:0/401514476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4033e20 con 0x56538aa8a4b0 2024-04-16T05:39:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4b17fa640 1 -- 172.21.15.32:0/401514476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4c4048550 con 0x56538aa8a4b0 2024-04-16T05:39:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 10 client.16647 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 10 client.16647 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:39:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 20 client.16647 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56538aaa7580 0x56538aaa9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 4 client.16647 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4d2c32740 1 -- 172.21.15.32:0/401514476 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56538a9fcb00 con 0x56538aaa7580 2024-04-16T05:39:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.445+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56538aaa7580 0x56538aaa9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.446+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56538aaa7580 0x56538aaa9970 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cb6055ef15d8990 server_cookie=f10d426f76d9961 in_seq=0 out_seq=0 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.446+0000 7fc4b17fa640 10 client.16647 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4b17fa640 1 -- 172.21.15.32:0/401514476 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56538a9fcb00 con 0x56538aaa7580 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4b17fa640 10 client.16647 handle_command_reply: tid=0 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 client.16647 shutdown 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 -- 172.21.15.32:0/401514476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 msgr2=0x56538aa94cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa94cd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4005e30 tx=0x7fc4c4033480 comp rx=0 tx=0).stop 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 -- 172.21.15.32:0/401514476 shutdown_connections 2024-04-16T05:39:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56538aaa7580 0x56538aaa9970 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 0x56538aa95210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa8a4b0 0x56538aa94cd0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/401514476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa89b90 0x56538aa94790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4b17fa640 0 client.16647 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:39:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 -- 172.21.15.32:0/401514476 >> 172.21.15.32:0/401514476 conn(0x56538aa84590 msgr2=0x56538aa862e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 -- 172.21.15.32:0/401514476 shutdown_connections 2024-04-16T05:39:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 1 -- 172.21.15.32:0/401514476 wait complete. 2024-04-16T05:39:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.447+0000 7fc4d2c32740 20 client.16647 trim_cache size 0 max 16384 2024-04-16T05:39:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.460+0000 7fc4d2c32740 1 Processor -- start 2024-04-16T05:39:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.460+0000 7fc4d2c32740 1 -- start start 2024-04-16T05:39:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa96ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 0x56538aaa33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa97770 0x56538aaaefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81afa20 con 0x56538aa97770 2024-04-16T05:39:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81ad2f0 con 0x56538aa8a4b0 2024-04-16T05:39:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81ab140 con 0x56538aa89b90 2024-04-16T05:39:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa97770 0x56538aaaefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa97770 0x56538aaaefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45750/0 (socket says 172.21.15.32:45750) 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 -- 172.21.15.32:0/2632650308 learned_addr learned my addr 172.21.15.32:0/2632650308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa96ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 0x56538aaa33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 -- 172.21.15.32:0/2632650308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 msgr2=0x56538aa96ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa96ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 -- 172.21.15.32:0/2632650308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 msgr2=0x56538aaa33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 0x56538aaa33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 -- 172.21.15.32:0/2632650308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56538aa7eb40 con 0x56538aa97770 2024-04-16T05:39:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa97770 0x56538aaaefa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8004700 tx=0x7fc4b8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb591edf2292cd01 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4b37fe640 1 -- 172.21.15.32:0/2632650308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8022480 con 0x56538aa97770 2024-04-16T05:39:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4b37fe640 1 -- 172.21.15.32:0/2632650308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b800fdd0 con 0x56538aa97770 2024-04-16T05:39:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.461+0000 7fc4b37fe640 1 -- 172.21.15.32:0/2632650308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80055c0 con 0x56538aa97770 2024-04-16T05:39:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2632650308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa97770 msgr2=0x56538aaaefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa97770 0x56538aaaefa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8004700 tx=0x7fc4b8005850 comp rx=0 tx=0).stop 2024-04-16T05:39:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2632650308 shutdown_connections 2024-04-16T05:39:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa97770 0x56538aaaefa0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 0x56538aaa33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/2632650308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa89b90 0x56538aa96ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2632650308 >> 172.21.15.32:0/2632650308 conn(0x56538aa84590 msgr2=0x56538aa84970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2632650308 shutdown_connections 2024-04-16T05:39:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 -- 172.21.15.32:0/2632650308 wait complete. 2024-04-16T05:39:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 -- start start 2024-04-16T05:39:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.462+0000 7fc4d2c32740 1 Processor -- start 2024-04-16T05:39:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa89b90 0x56538aa9f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 0x56538aa9fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4d2c32740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 0x56538aa90e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81ab7b0 con 0x56538aa89b90 2024-04-16T05:39:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81acc20 con 0x56538aa8a4b0 2024-04-16T05:39:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4d2c32740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81abe80 con 0x56538aa97770 2024-04-16T05:39:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4d2c32740 10 client.0 _ll_register_callbacks cb 0x7fc4b800c590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa89b90 0x56538aa9f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa89b90 0x56538aa9f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45758/0 (socket says 172.21.15.32:45758) 2024-04-16T05:39:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 -- 172.21.15.32:0/3594775664 learned_addr learned my addr 172.21.15.32:0/3594775664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 0x56538aa9fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 0x56538aa90e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 -- 172.21.15.32:0/3594775664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 msgr2=0x56538aa90e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 0x56538aa90e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 -- 172.21.15.32:0/3594775664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 msgr2=0x56538aa9fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 0x56538aa9fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 -- 172.21.15.32:0/3594775664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56538aa91640 con 0x56538aa89b90 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4cd5e5640 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa89b90 0x56538aa9f8e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800d010 tx=0x7fc4b800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ae2fc92ece5927 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4b17fa640 1 -- 172.21.15.32:0/3594775664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8005340 con 0x56538aa89b90 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4b17fa640 1 -- 172.21.15.32:0/3594775664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fc4b80054e0 con 0x56538aa89b90 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.463+0000 7fc4d2c32740 1 -- 172.21.15.32:0/3594775664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b800fdd0 con 0x56538aa89b90 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4b17fa640 1 -- 172.21.15.32:0/3594775664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8040050 con 0x56538aa89b90 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4d2c32740 10 client.16653 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4d2c32740 1 -- 172.21.15.32:0/3594775664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56538ab0d330 con 0x56538aa89b90 2024-04-16T05:39:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4b17fa640 1 -- 172.21.15.32:0/3594775664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b800fdd0 con 0x56538aa89b90 2024-04-16T05:39:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4b17fa640 1 -- 172.21.15.32:0/3594775664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4b8004b00 con 0x56538aa89b90 2024-04-16T05:39:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4d2c32740 10 client.16653 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4d2c32740 10 client.16653 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:39:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4d2c32740 20 client.16653 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56538ab0d990 0x56538ab0fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4d2c32740 4 client.16653 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4d2c32740 1 -- 172.21.15.32:0/3594775664 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56538a9f6970 con 0x56538ab0d990 2024-04-16T05:39:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.464+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56538ab0d990 0x56538ab0fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4cdde6640 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56538ab0d990 0x56538ab0fdb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d10daec1ee653101 server_cookie=576d1cfe1b61bbb2 in_seq=0 out_seq=0 2024-04-16T05:39:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4b17fa640 10 client.16653 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:39:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4b17fa640 1 -- 172.21.15.32:0/3594775664 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56538a9f6970 con 0x56538ab0d990 2024-04-16T05:39:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4b17fa640 10 client.16653 handle_command_reply: tid=0 2024-04-16T05:39:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 client.16653 shutdown 2024-04-16T05:39:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 -- 172.21.15.32:0/3594775664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa89b90 msgr2=0x56538aa9f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa89b90 0x56538aa9f8e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800d010 tx=0x7fc4b800cca0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 -- 172.21.15.32:0/3594775664 shutdown_connections 2024-04-16T05:39:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56538ab0d990 0x56538ab0fdb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56538aa97770 0x56538aa90e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56538aa8a4b0 0x56538aa9fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 --2- 172.21.15.32:0/3594775664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56538aa89b90 0x56538aa9f8e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4b17fa640 0 client.16653 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:39:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 -- 172.21.15.32:0/3594775664 >> 172.21.15.32:0/3594775664 conn(0x56538aa84590 msgr2=0x56538aa94730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 -- 172.21.15.32:0/3594775664 shutdown_connections 2024-04-16T05:39:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 1 -- 172.21.15.32:0/3594775664 wait complete. 2024-04-16T05:39:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.465+0000 7fc4d2c32740 20 client.16653 trim_cache size 0 max 16384 2024-04-16T05:39:19.481 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:39:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 -- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4b40a3d20 msgr2=0x7fc4b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4b40a3d20 0x7fc4b40a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000c940 tx=0x7fc4c0004940 comp rx=0 tx=0).stop 2024-04-16T05:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 -- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 msgr2=0x7fc4c81987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 0x7fc4c81987a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000c970 tx=0x7fc4c000ce30 comp rx=0 tx=0).stop 2024-04-16T05:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 -- 172.21.15.32:0/2184910789 shutdown_connections 2024-04-16T05:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4b40a3d20 0x7fc4b40a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8106660 0x7fc4c81987a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4c8101060 0x7fc4c819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 --2- 172.21.15.32:0/2184910789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4c81006a0 0x7fc4c819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 -- 172.21.15.32:0/2184910789 >> 172.21.15.32:0/2184910789 conn(0x7fc4c8077410 msgr2=0x7fc4c81035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 -- 172.21.15.32:0/2184910789 shutdown_connections 2024-04-16T05:39:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.466+0000 7fc4d004a640 1 -- 172.21.15.32:0/2184910789 wait complete. 2024-04-16T05:39:19.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:19 smithi032 ceph-mon[32351]: pgmap v134: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-16T05:39:19.485 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:19 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2011849442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:19.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:39:19.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 -- 172.21.15.32:0/1610265428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688101120 msgr2=0x7f6688106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 --2- 172.21.15.32:0/1610265428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688101120 0x7f6688106c50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6678009900 tx=0x7f667802bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 -- 172.21.15.32:0/1610265428 shutdown_connections 2024-04-16T05:39:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 --2- 172.21.15.32:0/1610265428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6688107330 0x7f668810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 --2- 172.21.15.32:0/1610265428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688101120 0x7f6688106c50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 --2- 172.21.15.32:0/1610265428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6688100770 0x7f6688100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 -- 172.21.15.32:0/1610265428 >> 172.21.15.32:0/1610265428 conn(0x7f6688077450 msgr2=0x7f66880fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 -- 172.21.15.32:0/1610265428 shutdown_connections 2024-04-16T05:39:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.595+0000 7f669047b640 1 -- 172.21.15.32:0/1610265428 wait complete. 2024-04-16T05:39:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f669047b640 1 Processor -- start 2024-04-16T05:39:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f669047b640 1 -- start start 2024-04-16T05:39:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f669047b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6688100770 0x7f66881016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f669047b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688107330 0x7f6688101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f669047b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6688102160 0x7f66881a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f669047b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66881123a0 con 0x7f6688107330 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f669047b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6688112220 con 0x7f6688100770 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f669047b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6688112520 con 0x7f6688102160 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668da16640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688107330 0x7f6688101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668da16640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688107330 0x7f6688101c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45778/0 (socket says 172.21.15.32:45778) 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668da16640 1 -- 172.21.15.32:0/1817874163 learned_addr learned my addr 172.21.15.32:0/1817874163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668ea18640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6688102160 0x7f66881a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668e217640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6688100770 0x7f66881016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668da16640 1 -- 172.21.15.32:0/1817874163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6688102160 msgr2=0x7f66881a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668da16640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6688102160 0x7f66881a6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668da16640 1 -- 172.21.15.32:0/1817874163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6688100770 msgr2=0x7f66881016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668da16640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6688100770 0x7f66881016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.596+0000 7f668da16640 1 -- 172.21.15.32:0/1817874163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66881a71b0 con 0x7f6688107330 2024-04-16T05:39:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.597+0000 7f668da16640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688107330 0x7f6688101c20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6678036040 tx=0x7f667802ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.597+0000 7f667f7fe640 1 -- 172.21.15.32:0/1817874163 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678041070 con 0x7f6688107330 2024-04-16T05:39:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.597+0000 7f667f7fe640 1 -- 172.21.15.32:0/1817874163 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66780054e0 con 0x7f6688107330 2024-04-16T05:39:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.597+0000 7f667f7fe640 1 -- 172.21.15.32:0/1817874163 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678034440 con 0x7f6688107330 2024-04-16T05:39:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.597+0000 7f669047b640 1 -- 172.21.15.32:0/1817874163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66881a7440 con 0x7f6688107330 2024-04-16T05:39:19.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.597+0000 7f669047b640 1 -- 172.21.15.32:0/1817874163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66881a7980 con 0x7f6688107330 2024-04-16T05:39:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.599+0000 7f667f7fe640 1 -- 172.21.15.32:0/1817874163 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6678044070 con 0x7f6688107330 2024-04-16T05:39:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.599+0000 7f669306f740 1 Processor -- start 2024-04-16T05:39:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f669306f740 1 -- start start 2024-04-16T05:39:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f669306f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 0x55c83bb06240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f669306f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f669306f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb11750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f669306f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c83ba6b0e0 con 0x55c83bb06780 2024-04-16T05:39:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f669306f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c83ba08520 con 0x55c83bb05e60 2024-04-16T05:39:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f669306f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c83ba670a0 con 0x55c83bb0f360 2024-04-16T05:39:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668e217640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f667f7fe640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f66740a3dd0 0x7f66740a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668e217640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45794/0 (socket says 172.21.15.32:45794) 2024-04-16T05:39:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668e217640 1 -- 172.21.15.32:0/1396096017 learned_addr learned my addr 172.21.15.32:0/1396096017 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668da16640 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb11750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668ea18640 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 0x55c83bb06240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668ea18640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f66740a3dd0 0x7f66740a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668e217640 1 -- 172.21.15.32:0/1396096017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 msgr2=0x55c83bb11750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668e217640 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb11750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f667f7fe640 1 -- 172.21.15.32:0/1817874163 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f66780f26c0 con 0x7f6688107330 2024-04-16T05:39:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668e217640 1 -- 172.21.15.32:0/1396096017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 msgr2=0x55c83bb06240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668e217640 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 0x55c83bb06240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.600+0000 7f668e217640 1 -- 172.21.15.32:0/1396096017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83ba308d0 con 0x55c83bb06780 2024-04-16T05:39:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.601+0000 7f668ea18640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f66740a3dd0 0x7f66740a6290 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6680005c40 tx=0x7f6680005bd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.601+0000 7f668e217640 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0ee20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f668400cbf0 tx=0x7f668400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30233adaf14dc8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.601+0000 7f667cbfe640 1 -- 172.21.15.32:0/1396096017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684010630 con 0x55c83bb06780 2024-04-16T05:39:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.601+0000 7f667cbfe640 1 -- 172.21.15.32:0/1396096017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6684010cb0 con 0x55c83bb06780 2024-04-16T05:39:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.601+0000 7f667cbfe640 1 -- 172.21.15.32:0/1396096017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684019c90 con 0x55c83bb06780 2024-04-16T05:39:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- 172.21.15.32:0/1396096017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 msgr2=0x55c83bb0ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0ee20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f668400cbf0 tx=0x7f668400e720 comp rx=0 tx=0).stop 2024-04-16T05:39:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- 172.21.15.32:0/1396096017 shutdown_connections 2024-04-16T05:39:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb11750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0ee20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 --2- 172.21.15.32:0/1396096017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 0x55c83bb06240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- 172.21.15.32:0/1396096017 >> 172.21.15.32:0/1396096017 conn(0x55c83bb00590 msgr2=0x55c83bb00970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- 172.21.15.32:0/1396096017 shutdown_connections 2024-04-16T05:39:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- 172.21.15.32:0/1396096017 wait complete. 2024-04-16T05:39:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- start start 2024-04-16T05:39:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 Processor -- start 2024-04-16T05:39:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 0x55c83bb0c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c83bb12000 con 0x55c83bb06780 2024-04-16T05:39:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c83ba244d0 con 0x55c83bb05e60 2024-04-16T05:39:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.602+0000 7f669306f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c83bb12180 con 0x55c83bb0f360 2024-04-16T05:39:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f669306f740 10 client.0 _ll_register_callbacks cb 0x55c83ba092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45802/0 (socket says 172.21.15.32:45802) 2024-04-16T05:39:19.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 -- 172.21.15.32:0/4118054153 learned_addr learned my addr 172.21.15.32:0/4118054153 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668e217640 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 0x55c83bb0c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668ea18640 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 -- 172.21.15.32:0/4118054153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 msgr2=0x55c83bb0cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb0cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 -- 172.21.15.32:0/4118054153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 msgr2=0x55c83bb0c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 0x55c83bb0c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 -- 172.21.15.32:0/4118054153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83b77d3f0 con 0x55c83bb06780 2024-04-16T05:39:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f668da16640 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0c850 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f667802cfd0 tx=0x7f6678004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9053419c52a66c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f6671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f6671ffb640 1 -- 172.21.15.32:0/4118054153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667804b070 con 0x55c83bb06780 2024-04-16T05:39:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f6671ffb640 1 -- 172.21.15.32:0/4118054153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f66780efcf0 con 0x55c83bb06780 2024-04-16T05:39:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f669306f740 1 -- 172.21.15.32:0/4118054153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6684010cb0 con 0x55c83bb06780 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f6671ffb640 1 -- 172.21.15.32:0/4118054153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f66780f24c0 con 0x55c83bb06780 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f669306f740 10 client.16665 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f669306f740 1 -- 172.21.15.32:0/4118054153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c83b8477d0 con 0x55c83bb06780 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.603+0000 7f6671ffb640 1 -- 172.21.15.32:0/4118054153 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6684010cb0 con 0x55c83bb06780 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f6671ffb640 1 -- 172.21.15.32:0/4118054153 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f66780c6bb0 con 0x55c83bb06780 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f669306f740 10 client.16665 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f669306f740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f669306f740 20 client.16665 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f669306f740 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c83bb1f320 0x55c83bb217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f669306f740 4 client.16665 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f669306f740 1 -- 172.21.15.32:0/4118054153 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83ba78b00 con 0x55c83bb1f320 2024-04-16T05:39:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f668e217640 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c83bb1f320 0x55c83bb217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f6671ffb640 10 client.16665 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.604+0000 7f668e217640 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c83bb1f320 0x55c83bb217b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310c33feec233a52 server_cookie=799e39391f06bc1e in_seq=0 out_seq=0 2024-04-16T05:39:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f6671ffb640 1 -- 172.21.15.32:0/4118054153 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c83ba78b00 con 0x55c83bb1f320 2024-04-16T05:39:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f6671ffb640 10 client.16665 handle_command_reply: tid=0 2024-04-16T05:39:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 client.16665 shutdown 2024-04-16T05:39:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 -- 172.21.15.32:0/4118054153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 msgr2=0x55c83bb0c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0c850 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f667802cfd0 tx=0x7f6678004610 comp rx=0 tx=0).stop 2024-04-16T05:39:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 -- 172.21.15.32:0/4118054153 shutdown_connections 2024-04-16T05:39:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c83bb1f320 0x55c83bb217b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb0cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb06780 0x55c83bb0c850 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f6671ffb640 0 client.16665 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 --2- 172.21.15.32:0/4118054153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb05e60 0x55c83bb0c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 -- 172.21.15.32:0/4118054153 >> 172.21.15.32:0/4118054153 conn(0x55c83bb00590 msgr2=0x55c83bb022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 -- 172.21.15.32:0/4118054153 shutdown_connections 2024-04-16T05:39:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 1 -- 172.21.15.32:0/4118054153 wait complete. 2024-04-16T05:39:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.605+0000 7f669306f740 20 client.16665 trim_cache size 0 max 16384 2024-04-16T05:39:19.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f669306f740 1 Processor -- start 2024-04-16T05:39:19.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f669306f740 1 -- start start 2024-04-16T05:39:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f669306f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb05e60 0x55c83bb1b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f669306f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f669306f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb0f360 0x55c83bb27600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f669306f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66881b3e60 con 0x55c83bb0f360 2024-04-16T05:39:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f669306f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66881b1730 con 0x55c83bb06780 2024-04-16T05:39:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f669306f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66881af580 con 0x55c83bb05e60 2024-04-16T05:39:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb0f360 0x55c83bb27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb0f360 0x55c83bb27600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45808/0 (socket says 172.21.15.32:45808) 2024-04-16T05:39:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 -- 172.21.15.32:0/2301540627 learned_addr learned my addr 172.21.15.32:0/2301540627 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668e217640 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb05e60 0x55c83bb1b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668da16640 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 -- 172.21.15.32:0/2301540627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb05e60 msgr2=0x55c83bb1b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb05e60 0x55c83bb1b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 -- 172.21.15.32:0/2301540627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 msgr2=0x55c83bb270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 -- 172.21.15.32:0/2301540627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83bb01160 con 0x55c83bb0f360 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668e217640 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb05e60 0x55c83bb1b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668da16640 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.619+0000 7f668ea18640 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb0f360 0x55c83bb27600 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6680009c40 tx=0x7f66800096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dbeb9a6f249179 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f6673fff640 1 -- 172.21.15.32:0/2301540627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66800106d0 con 0x55c83bb0f360 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f6673fff640 1 -- 172.21.15.32:0/2301540627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6680010d50 con 0x55c83bb0f360 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f6673fff640 1 -- 172.21.15.32:0/2301540627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668001fc80 con 0x55c83bb0f360 2024-04-16T05:39:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 -- 172.21.15.32:0/2301540627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb0f360 msgr2=0x55c83bb27600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb0f360 0x55c83bb27600 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6680009c40 tx=0x7f66800096c0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 -- 172.21.15.32:0/2301540627 shutdown_connections 2024-04-16T05:39:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb0f360 0x55c83bb27600 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 --2- 172.21.15.32:0/2301540627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb05e60 0x55c83bb1b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 -- 172.21.15.32:0/2301540627 >> 172.21.15.32:0/2301540627 conn(0x55c83bb00590 msgr2=0x55c83bb00970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 -- 172.21.15.32:0/2301540627 shutdown_connections 2024-04-16T05:39:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 -- 172.21.15.32:0/2301540627 wait complete. 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.620+0000 7f669306f740 1 -- start start 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f669306f740 1 Processor -- start 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f669306f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb05e60 0x55c83bb17810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f669306f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb08bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f669306f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb09100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f669306f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66881afbf0 con 0x55c83bb05e60 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f669306f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66881b1060 con 0x55c83bb06780 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f669306f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66881b02c0 con 0x55c83bb0f360 2024-04-16T05:39:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f669306f740 10 client.0 _ll_register_callbacks cb 0x7f6678003ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb05e60 0x55c83bb17810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb05e60 0x55c83bb17810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45812/0 (socket says 172.21.15.32:45812) 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 -- 172.21.15.32:0/1546309638 learned_addr learned my addr 172.21.15.32:0/1546309638 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668da16640 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb08bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668ea18640 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb09100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 -- 172.21.15.32:0/1546309638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 msgr2=0x55c83bb09100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb09100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 -- 172.21.15.32:0/1546309638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 msgr2=0x55c83bb08bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb08bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 -- 172.21.15.32:0/1546309638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83bb097a0 con 0x55c83bb05e60 2024-04-16T05:39:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f668e217640 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb05e60 0x55c83bb17810 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f668400c750 tx=0x7f6684005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88debc078ad783c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.621+0000 7f6671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f6671ffb640 1 -- 172.21.15.32:0/1546309638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684010e60 con 0x55c83bb05e60 2024-04-16T05:39:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f6671ffb640 1 -- 172.21.15.32:0/1546309638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6684019af0 con 0x55c83bb05e60 2024-04-16T05:39:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 1 -- 172.21.15.32:0/1546309638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6680010d50 con 0x55c83bb05e60 2024-04-16T05:39:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f6671ffb640 1 -- 172.21.15.32:0/1546309638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6684019d30 con 0x55c83bb05e60 2024-04-16T05:39:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 10 client.16671 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 1 -- 172.21.15.32:0/1546309638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c83bb85730 con 0x55c83bb05e60 2024-04-16T05:39:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f6671ffb640 1 -- 172.21.15.32:0/1546309638 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6680010d50 con 0x55c83bb05e60 2024-04-16T05:39:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f6671ffb640 1 -- 172.21.15.32:0/1546309638 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6684004780 con 0x55c83bb05e60 2024-04-16T05:39:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 10 client.16671 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 20 client.16671 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c83bb85d70 0x55c83bb88190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 4 client.16671 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f669306f740 1 -- 172.21.15.32:0/1546309638 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c83ba72970 con 0x55c83bb85d70 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.622+0000 7f668da16640 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c83bb85d70 0x55c83bb88190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f6671ffb640 10 client.16671 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f668da16640 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c83bb85d70 0x55c83bb88190 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04775eac602bd10 server_cookie=828a3fc97a9cde31 in_seq=0 out_seq=0 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f6671ffb640 1 -- 172.21.15.32:0/1546309638 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c83ba72970 con 0x55c83bb85d70 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f6671ffb640 10 client.16671 handle_command_reply: tid=0 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 client.16671 shutdown 2024-04-16T05:39:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 -- 172.21.15.32:0/1546309638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb05e60 msgr2=0x55c83bb17810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb05e60 0x55c83bb17810 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f668400c750 tx=0x7f6684005ce0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 -- 172.21.15.32:0/1546309638 shutdown_connections 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55c83bb85d70 0x55c83bb88190 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c83bb0f360 0x55c83bb09100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c83bb06780 0x55c83bb08bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f6671ffb640 0 client.16671 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 --2- 172.21.15.32:0/1546309638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c83bb05e60 0x55c83bb17810 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 -- 172.21.15.32:0/1546309638 >> 172.21.15.32:0/1546309638 conn(0x55c83bb00590 msgr2=0x55c83bafeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 -- 172.21.15.32:0/1546309638 shutdown_connections 2024-04-16T05:39:19.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 1 -- 172.21.15.32:0/1546309638 wait complete. 2024-04-16T05:39:19.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.623+0000 7f669306f740 20 client.16671 trim_cache size 0 max 16384 2024-04-16T05:39:19.637 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:39:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 -- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f66740a3dd0 msgr2=0x7f66740a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f66740a3dd0 0x7f66740a6290 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6680005c40 tx=0x7f6680005bd0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 -- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688107330 msgr2=0x7f6688101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688107330 0x7f6688101c20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6678036040 tx=0x7f667802ce20 comp rx=0 tx=0).stop 2024-04-16T05:39:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 -- 172.21.15.32:0/1817874163 shutdown_connections 2024-04-16T05:39:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6688102160 0x7f66881a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f66740a3dd0 0x7f66740a6290 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6688107330 0x7f6688101c20 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 --2- 172.21.15.32:0/1817874163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6688100770 0x7f66881016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 -- 172.21.15.32:0/1817874163 >> 172.21.15.32:0/1817874163 conn(0x7f6688077450 msgr2=0x7f66880fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 -- 172.21.15.32:0/1817874163 shutdown_connections 2024-04-16T05:39:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:19.624+0000 7f669047b640 1 -- 172.21.15.32:0/1817874163 wait complete. 2024-04-16T05:39:19.641 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:19 smithi134 ceph-mon[36153]: pgmap v134: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-16T05:39:19.641 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:19 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2011849442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:19.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:19 smithi082 ceph-mon[36195]: pgmap v134: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-16T05:39:19.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:19 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2011849442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:20.641 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:39:20.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:39:20.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:39:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3228044062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 msgr2=0x7f7d8810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3228044062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 0x7f7d8810dde0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009ca0 tx=0x7f7d8002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3228044062 shutdown_connections 2024-04-16T05:39:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3228044062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 0x7f7d8810dde0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3228044062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d88101af0 0x7f7d88105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3228044062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d88101140 0x7f7d88101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3228044062 >> 172.21.15.32:0/3228044062 conn(0x7f7d88078000 msgr2=0x7f7d88078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3228044062 shutdown_connections 2024-04-16T05:39:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.765+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3228044062 wait complete. 2024-04-16T05:39:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8f4f2640 1 Processor -- start 2024-04-16T05:39:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8f4f2640 1 -- start start 2024-04-16T05:39:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8f4f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d88101140 0x7f7d881996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8f4f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 0x7f7d88199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8f4f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d8819a1c0 0x7f7d881a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8f4f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881112d0 con 0x7f7d88106210 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8f4f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88111150 con 0x7f7d8819a1c0 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8f4f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88111450 con 0x7f7d88101140 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 0x7f7d88199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 0x7f7d88199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45844/0 (socket says 172.21.15.32:45844) 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/3183496847 learned_addr learned my addr 172.21.15.32:0/3183496847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d8819a1c0 0x7f7d881a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d88101140 0x7f7d881996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/3183496847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d88101140 msgr2=0x7f7d881996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d88101140 0x7f7d881996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/3183496847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d8819a1c0 msgr2=0x7f7d881a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d8819a1c0 0x7f7d881a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/3183496847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d881a4f70 con 0x7f7d88106210 2024-04-16T05:39:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d88101140 0x7f7d881996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.766+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 0x7f7d88199c30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00c970 tx=0x7f7d7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.767+0000 7f7d7a7fc640 1 -- 172.21.15.32:0/3183496847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c007bb0 con 0x7f7d88106210 2024-04-16T05:39:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.767+0000 7f7d7a7fc640 1 -- 172.21.15.32:0/3183496847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7c007d50 con 0x7f7d88106210 2024-04-16T05:39:20.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.767+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3183496847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d881a5260 con 0x7f7d88106210 2024-04-16T05:39:20.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.767+0000 7f7d7a7fc640 1 -- 172.21.15.32:0/3183496847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c00f450 con 0x7f7d88106210 2024-04-16T05:39:20.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.767+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3183496847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d881a57b0 con 0x7f7d88106210 2024-04-16T05:39:20.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.769+0000 7f7d7a7fc640 1 -- 172.21.15.32:0/3183496847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d7c0046c0 con 0x7f7d88106210 2024-04-16T05:39:20.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.769+0000 7f7d921ba740 1 Processor -- start 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d921ba740 1 -- start start 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d921ba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563af2749e90 con 0x563af27e53d0 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d921ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563af26e74c0 con 0x563af27e4ab0 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d921ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563af2745e50 con 0x563af27edfb0 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8d28e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d7a7fc640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7d740a3d20 0x7f7d740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8d28e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45860/0 (socket says 172.21.15.32:45860) 2024-04-16T05:39:20.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8d28e640 1 -- 172.21.15.32:0/667662480 learned_addr learned my addr 172.21.15.32:0/667662480 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7d740a3d20 0x7f7d740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8d28e640 1 -- 172.21.15.32:0/667662480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 msgr2=0x563af27f03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8d28e640 1 -- 172.21.15.32:0/667662480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 msgr2=0x563af27e4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27e4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8d28e640 1 -- 172.21.15.32:0/667662480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af25247e0 con 0x563af27e53d0 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d7a7fc640 1 -- 172.21.15.32:0/3183496847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7d7c0d0b90 con 0x7f7d88106210 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.770+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.771+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eda70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400bd40 tx=0x7f7d8400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e33381ab085d280 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.771+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7d740a3d20 0x7f7d740a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d80004af0 tx=0x7f7d8002e040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.771+0000 7f7d737fe640 1 -- 172.21.15.32:0/667662480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84015070 con 0x563af27e53d0 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.771+0000 7f7d737fe640 1 -- 172.21.15.32:0/667662480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8400be30 con 0x563af27e53d0 2024-04-16T05:39:20.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.771+0000 7f7d737fe640 1 -- 172.21.15.32:0/667662480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84011a10 con 0x563af27e53d0 2024-04-16T05:39:20.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.771+0000 7f7d921ba740 1 -- 172.21.15.32:0/667662480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 msgr2=0x563af27eda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.771+0000 7f7d921ba740 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eda70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400bd40 tx=0x7f7d8400f730 comp rx=0 tx=0).stop 2024-04-16T05:39:20.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 -- 172.21.15.32:0/667662480 shutdown_connections 2024-04-16T05:39:20.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eda70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 --2- 172.21.15.32:0/667662480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 -- 172.21.15.32:0/667662480 >> 172.21.15.32:0/667662480 conn(0x563af27df1e0 msgr2=0x563af27df5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 -- 172.21.15.32:0/667662480 shutdown_connections 2024-04-16T05:39:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 -- 172.21.15.32:0/667662480 wait complete. 2024-04-16T05:39:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 -- start start 2024-04-16T05:39:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 Processor -- start 2024-04-16T05:39:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563af27f0da0 con 0x563af27e53d0 2024-04-16T05:39:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563af26d2410 con 0x563af27e4ab0 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563af27f0f20 con 0x563af27edfb0 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.772+0000 7f7d921ba740 10 client.0 _ll_register_callbacks cb 0x563af26cdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45870/0 (socket says 172.21.15.32:45870) 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/1063145587 learned_addr learned my addr 172.21.15.32:0/1063145587 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/1063145587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 msgr2=0x563af27ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 msgr2=0x563af27eb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27eb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/1063145587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af2584f70 con 0x563af27e53d0 2024-04-16T05:39:20.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:20.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:20.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eb5b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c007590 tx=0x7f7d7c0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf1ff9989c2f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:20.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d70ff9640 1 -- 172.21.15.32:0/1063145587 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c01d040 con 0x563af27e53d0 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d70ff9640 1 -- 172.21.15.32:0/1063145587 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7d7c0ce1c0 con 0x563af27e53d0 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d921ba740 1 -- 172.21.15.32:0/1063145587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8400be30 con 0x563af27e53d0 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d70ff9640 1 -- 172.21.15.32:0/1063145587 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c0ce3c0 con 0x563af27e53d0 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d921ba740 10 client.16683 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.773+0000 7f7d921ba740 1 -- 172.21.15.32:0/1063145587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563af241c170 con 0x563af27e53d0 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d70ff9640 1 -- 172.21.15.32:0/1063145587 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8400be30 con 0x563af27e53d0 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d70ff9640 1 -- 172.21.15.32:0/1063145587 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d7c09bbe0 con 0x563af27e53d0 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d921ba740 10 client.16683 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:20.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d921ba740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d921ba740 20 client.16683 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d921ba740 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563af27fe2e0 0x563af28006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d921ba740 4 client.16683 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d921ba740 1 -- 172.21.15.32:0/1063145587 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563af27578b0 con 0x563af27fe2e0 2024-04-16T05:39:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.774+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563af27fe2e0 0x563af28006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.775+0000 7f7d70ff9640 10 client.16683 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.775+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563af27fe2e0 0x563af28006d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca014fcd903a9999 server_cookie=e83bd286be096eba in_seq=0 out_seq=0 2024-04-16T05:39:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.775+0000 7f7d70ff9640 1 -- 172.21.15.32:0/1063145587 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563af27578b0 con 0x563af27fe2e0 2024-04-16T05:39:20.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.775+0000 7f7d70ff9640 10 client.16683 handle_command_reply: tid=0 2024-04-16T05:39:20.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.775+0000 7f7d921ba740 1 client.16683 shutdown 2024-04-16T05:39:20.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.775+0000 7f7d921ba740 1 -- 172.21.15.32:0/1063145587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 msgr2=0x563af27eb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.775+0000 7f7d921ba740 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eb5b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c007590 tx=0x7f7d7c0049f0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 1 -- 172.21.15.32:0/1063145587 shutdown_connections 2024-04-16T05:39:20.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563af27fe2e0 0x563af28006d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27eb5b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d70ff9640 0 client.16683 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:20.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 1 --2- 172.21.15.32:0/1063145587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e4ab0 0x563af27eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 1 -- 172.21.15.32:0/1063145587 >> 172.21.15.32:0/1063145587 conn(0x563af27df1e0 msgr2=0x563af27e0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 1 -- 172.21.15.32:0/1063145587 shutdown_connections 2024-04-16T05:39:20.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 1 -- 172.21.15.32:0/1063145587 wait complete. 2024-04-16T05:39:20.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.776+0000 7f7d921ba740 20 client.16683 trim_cache size 0 max 16384 2024-04-16T05:39:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.793+0000 7f7d921ba740 1 Processor -- start 2024-04-16T05:39:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.793+0000 7f7d921ba740 1 -- start start 2024-04-16T05:39:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27e4ab0 0x563af27ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27edfb0 0x563af2805f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d921ba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881b1bd0 con 0x563af27e53d0 2024-04-16T05:39:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d921ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881af4a0 con 0x563af27edfb0 2024-04-16T05:39:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d921ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881ad2f0 con 0x563af27e4ab0 2024-04-16T05:39:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27f9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45886/0 (socket says 172.21.15.32:45886) 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/3255854663 learned_addr learned my addr 172.21.15.32:0/3255854663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/3255854663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27e4ab0 0x563af27ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/3255854663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27e4ab0 msgr2=0x563af27ed7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/3255854663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27e4ab0 0x563af27ed7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/3255854663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27edfb0 msgr2=0x563af2805f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/3255854663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27edfb0 0x563af2805f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 -- 172.21.15.32:0/3255854663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af27e3dd0 con 0x563af27e53d0 2024-04-16T05:39:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/3255854663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27f9c60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c09c490 tx=0x7f7d7c098a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a40d7aba51c6cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d72ffd640 1 -- 172.21.15.32:0/3255854663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c09c790 con 0x563af27e53d0 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d72ffd640 1 -- 172.21.15.32:0/3255854663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7c0a0b40 con 0x563af27e53d0 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.794+0000 7f7d72ffd640 1 -- 172.21.15.32:0/3255854663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c0a5190 con 0x563af27e53d0 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 -- 172.21.15.32:0/3255854663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 msgr2=0x563af27f9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 --2- 172.21.15.32:0/3255854663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27f9c60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c09c490 tx=0x7f7d7c098a30 comp rx=0 tx=0).stop 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 -- 172.21.15.32:0/3255854663 shutdown_connections 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 --2- 172.21.15.32:0/3255854663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27edfb0 0x563af2805f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 --2- 172.21.15.32:0/3255854663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e53d0 0x563af27f9c60 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 --2- 172.21.15.32:0/3255854663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27e4ab0 0x563af27ed7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 -- 172.21.15.32:0/3255854663 >> 172.21.15.32:0/3255854663 conn(0x563af27df1e0 msgr2=0x563af27de250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 -- 172.21.15.32:0/3255854663 shutdown_connections 2024-04-16T05:39:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 -- 172.21.15.32:0/3255854663 wait complete. 2024-04-16T05:39:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 -- start start 2024-04-16T05:39:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.795+0000 7f7d921ba740 1 Processor -- start 2024-04-16T05:39:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e4ab0 0x563af27f6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e53d0 0x563af27f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d921ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d921ba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881ad960 con 0x563af27e4ab0 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d921ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881aedd0 con 0x563af27e53d0 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d921ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881ae030 con 0x563af27edfb0 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d921ba740 10 client.0 _ll_register_callbacks cb 0x7f7d7c007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e4ab0 0x563af27f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e4ab0 0x563af27f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45888/0 (socket says 172.21.15.32:45888) 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 -- 172.21.15.32:0/2459095736 learned_addr learned my addr 172.21.15.32:0/2459095736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e53d0 0x563af27f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 -- 172.21.15.32:0/2459095736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 msgr2=0x563af27f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 -- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e53d0 msgr2=0x563af27f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e53d0 0x563af27f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 -- 172.21.15.32:0/2459095736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af27f2ec0 con 0x563af27e4ab0 2024-04-16T05:39:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8da8f640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e53d0 0x563af27f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d8d28e640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e4ab0 0x563af27f6140 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7d84011700 tx=0x7f7d84005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08a71143fe5a43d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.796+0000 7f7d70ff9640 1 -- 172.21.15.32:0/2459095736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8402d060 con 0x563af27e4ab0 2024-04-16T05:39:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d70ff9640 1 -- 172.21.15.32:0/2459095736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f7d840052c0 con 0x563af27e4ab0 2024-04-16T05:39:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 1 -- 172.21.15.32:0/2459095736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d7c0a0b40 con 0x563af27e4ab0 2024-04-16T05:39:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d70ff9640 1 -- 172.21.15.32:0/2459095736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d84034050 con 0x563af27e4ab0 2024-04-16T05:39:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 10 client.16689 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 1 -- 172.21.15.32:0/2459095736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563af2865470 con 0x563af27e4ab0 2024-04-16T05:39:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d70ff9640 1 -- 172.21.15.32:0/2459095736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7c0a0b40 con 0x563af27e4ab0 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d70ff9640 1 -- 172.21.15.32:0/2459095736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d84004680 con 0x563af27e4ab0 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 10 client.16689 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 20 client.16689 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563af2865910 0x563af2867d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 4 client.16689 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d921ba740 1 -- 172.21.15.32:0/2459095736 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563af2751720 con 0x563af2865910 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.797+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563af2865910 0x563af2867d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.798+0000 7f7d70ff9640 10 client.16689 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.798+0000 7f7d8ca8d640 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563af2865910 0x563af2867d00 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370112cbf2ee8eb0 server_cookie=7db9a5b3d776cb3d in_seq=0 out_seq=0 2024-04-16T05:39:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d70ff9640 1 -- 172.21.15.32:0/2459095736 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563af2751720 con 0x563af2865910 2024-04-16T05:39:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d70ff9640 10 client.16689 handle_command_reply: tid=0 2024-04-16T05:39:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 client.16689 shutdown 2024-04-16T05:39:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 -- 172.21.15.32:0/2459095736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e4ab0 msgr2=0x563af27f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e4ab0 0x563af27f6140 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7d84011700 tx=0x7f7d84005b60 comp rx=0 tx=0).stop 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 -- 172.21.15.32:0/2459095736 shutdown_connections 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563af2865910 0x563af2867d00 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563af27edfb0 0x563af27f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563af27e53d0 0x563af27f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 --2- 172.21.15.32:0/2459095736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563af27e4ab0 0x563af27f6140 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 -- 172.21.15.32:0/2459095736 >> 172.21.15.32:0/2459095736 conn(0x563af27df1e0 msgr2=0x563af27eb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d70ff9640 0 client.16689 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 -- 172.21.15.32:0/2459095736 shutdown_connections 2024-04-16T05:39:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 1 -- 172.21.15.32:0/2459095736 wait complete. 2024-04-16T05:39:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.799+0000 7f7d921ba740 20 client.16689 trim_cache size 0 max 16384 2024-04-16T05:39:20.808 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:39:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:39:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "6ee82a35-9f39-4e5e-bb7e-dd2ba7ae4c2e", 2024-04-16T05:39:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:39:20.809 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:39:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7d740a3d20 msgr2=0x7f7d740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7d740a3d20 0x7f7d740a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d80004af0 tx=0x7f7d8002e040 comp rx=0 tx=0).stop 2024-04-16T05:39:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 msgr2=0x7f7d88199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 0x7f7d88199c30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00c970 tx=0x7f7d7c00ce30 comp rx=0 tx=0).stop 2024-04-16T05:39:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3183496847 shutdown_connections 2024-04-16T05:39:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d8819a1c0 0x7f7d881a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f7d740a3d20 0x7f7d740a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d88106210 0x7f7d88199c30 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 --2- 172.21.15.32:0/3183496847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d88101140 0x7f7d881996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3183496847 >> 172.21.15.32:0/3183496847 conn(0x7f7d88078000 msgr2=0x7f7d880ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3183496847 shutdown_connections 2024-04-16T05:39:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.800+0000 7f7d8f4f2640 1 -- 172.21.15.32:0/3183496847 wait complete. 2024-04-16T05:39:20.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:39:20.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:39:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 -- 172.21.15.32:0/1167563678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 msgr2=0x7f5508102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 --2- 172.21.15.32:0/1167563678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 0x7f5508102620 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5500009900 tx=0x7f550002bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 -- 172.21.15.32:0/1167563678 shutdown_connections 2024-04-16T05:39:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 --2- 172.21.15.32:0/1167563678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5508107310 0x7f550810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 --2- 172.21.15.32:0/1167563678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5508102bf0 0x7f5508106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 --2- 172.21.15.32:0/1167563678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 0x7f5508102620 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 -- 172.21.15.32:0/1167563678 >> 172.21.15.32:0/1167563678 conn(0x7f55080fda40 msgr2=0x7f55080ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 -- 172.21.15.32:0/1167563678 shutdown_connections 2024-04-16T05:39:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.921+0000 7f550f749640 1 -- 172.21.15.32:0/1167563678 wait complete. 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550f749640 1 Processor -- start 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550f749640 1 -- start start 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550f749640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 0x7f550819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550f749640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5508107310 0x7f550819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550f749640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5508198700 0x7f5508198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550f749640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55081123a0 con 0x7f5508102240 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550f749640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5508112220 con 0x7f5508198700 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550f749640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5508112520 con 0x7f5508107310 2024-04-16T05:39:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550d4e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 0x7f550819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550d4e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 0x7f550819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45918/0 (socket says 172.21.15.32:45918) 2024-04-16T05:39:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550d4e5640 1 -- 172.21.15.32:0/3408480429 learned_addr learned my addr 172.21.15.32:0/3408480429 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.922+0000 7f550dce6640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5508198700 0x7f5508198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550cce4640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5508107310 0x7f550819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550d4e5640 1 -- 172.21.15.32:0/3408480429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5508107310 msgr2=0x7f550819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550d4e5640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5508107310 0x7f550819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550d4e5640 1 -- 172.21.15.32:0/3408480429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5508198700 msgr2=0x7f5508198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550d4e5640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5508198700 0x7f5508198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550d4e5640 1 -- 172.21.15.32:0/3408480429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5508199300 con 0x7f5508102240 2024-04-16T05:39:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550d4e5640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 0x7f550819e640 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5500006010 tx=0x7f550002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f54fe7fc640 1 -- 172.21.15.32:0/3408480429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500041070 con 0x7f5508102240 2024-04-16T05:39:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f54fe7fc640 1 -- 172.21.15.32:0/3408480429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5500004450 con 0x7f5508102240 2024-04-16T05:39:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550f749640 1 -- 172.21.15.32:0/3408480429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5508199590 con 0x7f5508102240 2024-04-16T05:39:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f54fe7fc640 1 -- 172.21.15.32:0/3408480429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500003e00 con 0x7f5508102240 2024-04-16T05:39:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.923+0000 7f550f749640 1 -- 172.21.15.32:0/3408480429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55081a56d0 con 0x7f5508102240 2024-04-16T05:39:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.925+0000 7f54fe7fc640 1 -- 172.21.15.32:0/3408480429 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5500044070 con 0x7f5508102240 2024-04-16T05:39:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f551240c740 1 Processor -- start 2024-04-16T05:39:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f551240c740 1 -- start start 2024-04-16T05:39:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f551240c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984cebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f551240c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984cf9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f551240c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 0x55d984cfbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f551240c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d984c51180 con 0x55d984ceb830 2024-04-16T05:39:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f551240c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d984bee360 con 0x55d984cec1e0 2024-04-16T05:39:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f551240c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d984c4d140 con 0x55d984cf9760 2024-04-16T05:39:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f54fe7fc640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54f40a3d20 0x7f54f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f550dce6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984cebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f550dce6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984cebc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45928/0 (socket says 172.21.15.32:45928) 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f550dce6640 1 -- 172.21.15.32:0/3926324032 learned_addr learned my addr 172.21.15.32:0/3926324032 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f550dce6640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54f40a3d20 0x7f54f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f54fe7fc640 1 -- 172.21.15.32:0/3408480429 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f55000f2590 con 0x7f5508102240 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f550cce4640 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 0x55d984cfbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f550d4e5640 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984cf9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.926+0000 7f550dce6640 1 -- 172.21.15.32:0/3926324032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 msgr2=0x55d984cfbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f550dce6640 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 0x55d984cfbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f550dce6640 1 -- 172.21.15.32:0/3926324032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 msgr2=0x55d984cf9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f550dce6640 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984cf9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f550dce6640 1 -- 172.21.15.32:0/3926324032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d984a2d7d0 con 0x55d984ceb830 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f550dce6640 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984cebc10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5504012d00 tx=0x7f5504010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c620e78c1d5b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f550dce6640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54f40a3d20 0x7f54f40a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5504010c40 tx=0x7f5504017000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f54f37fe640 1 -- 172.21.15.32:0/3926324032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504024070 con 0x55d984ceb830 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f54f37fe640 1 -- 172.21.15.32:0/3926324032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5504012520 con 0x55d984ceb830 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.927+0000 7f54f37fe640 1 -- 172.21.15.32:0/3926324032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504005280 con 0x55d984ceb830 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 -- 172.21.15.32:0/3926324032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 msgr2=0x55d984cebc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984cebc10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5504012d00 tx=0x7f5504010740 comp rx=0 tx=0).stop 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 -- 172.21.15.32:0/3926324032 shutdown_connections 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 0x55d984cfbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984cf9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 --2- 172.21.15.32:0/3926324032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984cebc10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 -- 172.21.15.32:0/3926324032 >> 172.21.15.32:0/3926324032 conn(0x55d984ce6580 msgr2=0x55d984ce6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 -- 172.21.15.32:0/3926324032 shutdown_connections 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 -- 172.21.15.32:0/3926324032 wait complete. 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 -- start start 2024-04-16T05:39:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.928+0000 7f551240c740 1 Processor -- start 2024-04-16T05:39:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f551240c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984ceb830 0x55d984cf6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f551240c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cec1e0 0x55d984cf6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f551240c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cf9760 0x55d984cf7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f551240c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d984cfc4c0 con 0x55d984cec1e0 2024-04-16T05:39:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f551240c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d984c0a2c0 con 0x55d984cf9760 2024-04-16T05:39:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f551240c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d984cfc640 con 0x55d984ceb830 2024-04-16T05:39:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f551240c740 10 client.0 _ll_register_callbacks cb 0x55d984bf1090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cec1e0 0x55d984cf6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cec1e0 0x55d984cf6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45942/0 (socket says 172.21.15.32:45942) 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 -- 172.21.15.32:0/1948588266 learned_addr learned my addr 172.21.15.32:0/1948588266 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550d4e5640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984ceb830 0x55d984cf6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550dce6640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cf9760 0x55d984cf7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 -- 172.21.15.32:0/1948588266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984ceb830 msgr2=0x55d984cf6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984ceb830 0x55d984cf6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 -- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cf9760 msgr2=0x55d984cf7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cf9760 0x55d984cf7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 -- 172.21.15.32:0/1948588266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9849633f0 con 0x55d984cec1e0 2024-04-16T05:39:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550d4e5640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984ceb830 0x55d984cf6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550dce6640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cf9760 0x55d984cf7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f550cce4640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cec1e0 0x55d984cf6d50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009fd0 tx=0x7f54f8012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1fdd18035a9815 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f54f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f54f0ff9640 1 -- 172.21.15.32:0/1948588266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8004220 con 0x55d984cec1e0 2024-04-16T05:39:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f54f0ff9640 1 -- 172.21.15.32:0/1948588266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f54f80043c0 con 0x55d984cec1e0 2024-04-16T05:39:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.929+0000 7f551240c740 1 -- 172.21.15.32:0/1948588266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5504012520 con 0x55d984cec1e0 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f54f0ff9640 1 -- 172.21.15.32:0/1948588266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54f80047a0 con 0x55d984cec1e0 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f551240c740 10 client.16701 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f551240c740 1 -- 172.21.15.32:0/1948588266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d9849230f0 con 0x55d984cec1e0 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f54f0ff9640 1 -- 172.21.15.32:0/1948588266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5504012520 con 0x55d984cec1e0 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f54f0ff9640 1 -- 172.21.15.32:0/1948588266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54f800f040 con 0x55d984cec1e0 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f551240c740 10 client.16701 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f551240c740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f551240c740 20 client.16701 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f551240c740 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d984d09bc0 0x55d984d0c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f551240c740 4 client.16701 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f551240c740 1 -- 172.21.15.32:0/1948588266 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d984c5eba0 con 0x55d984d09bc0 2024-04-16T05:39:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.930+0000 7f550d4e5640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d984d09bc0 0x55d984d0c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f550d4e5640 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d984d09bc0 0x55d984d0c000 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3d4a9204e3888 server_cookie=d2293b26d9337656 in_seq=0 out_seq=0 2024-04-16T05:39:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f54f0ff9640 10 client.16701 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f54f0ff9640 1 -- 172.21.15.32:0/1948588266 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d984c5eba0 con 0x55d984d09bc0 2024-04-16T05:39:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f54f0ff9640 10 client.16701 handle_command_reply: tid=0 2024-04-16T05:39:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 client.16701 shutdown 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 -- 172.21.15.32:0/1948588266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cec1e0 msgr2=0x55d984cf6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cec1e0 0x55d984cf6d50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009fd0 tx=0x7f54f8012880 comp rx=0 tx=0).stop 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 -- 172.21.15.32:0/1948588266 shutdown_connections 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cf9760 0x55d984cf7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cec1e0 0x55d984cf6d50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d984d09bc0 0x55d984d0c000 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 --2- 172.21.15.32:0/1948588266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984ceb830 0x55d984cf6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f54f0ff9640 0 client.16701 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.931+0000 7f551240c740 1 -- 172.21.15.32:0/1948588266 >> 172.21.15.32:0/1948588266 conn(0x55d984ce6580 msgr2=0x55d984ce82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.932+0000 7f551240c740 1 -- 172.21.15.32:0/1948588266 shutdown_connections 2024-04-16T05:39:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.932+0000 7f551240c740 1 -- 172.21.15.32:0/1948588266 wait complete. 2024-04-16T05:39:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.932+0000 7f551240c740 20 client.16701 trim_cache size 0 max 16384 2024-04-16T05:39:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f551240c740 1 Processor -- start 2024-04-16T05:39:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f551240c740 1 -- start start 2024-04-16T05:39:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f551240c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984ceb830 0x55d984d053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f551240c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cec1e0 0x55d984ce55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f551240c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cf9760 0x55d984d118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f551240c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55081af9f0 con 0x55d984cf9760 2024-04-16T05:39:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f551240c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55081ad2c0 con 0x55d984ceb830 2024-04-16T05:39:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f551240c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55081ab110 con 0x55d984cec1e0 2024-04-16T05:39:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f550dce6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cf9760 0x55d984d118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f550dce6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cf9760 0x55d984d118d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45952/0 (socket says 172.21.15.32:45952) 2024-04-16T05:39:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.949+0000 7f550dce6640 1 -- 172.21.15.32:0/48874973 learned_addr learned my addr 172.21.15.32:0/48874973 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f550cce4640 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cec1e0 0x55d984ce55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f550d4e5640 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984ceb830 0x55d984d053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f550dce6640 1 -- 172.21.15.32:0/48874973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cec1e0 msgr2=0x55d984ce55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f550dce6640 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cec1e0 0x55d984ce55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f550dce6640 1 -- 172.21.15.32:0/48874973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984ceb830 msgr2=0x55d984d053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f550dce6640 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984ceb830 0x55d984d053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f550dce6640 1 -- 172.21.15.32:0/48874973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d984ceaf40 con 0x55d984cf9760 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f550dce6640 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cf9760 0x55d984d118d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5504012470 tx=0x7f5504012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e054a7e62e3299 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f54f2ffd640 1 -- 172.21.15.32:0/48874973 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504024040 con 0x55d984cf9760 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f54f2ffd640 1 -- 172.21.15.32:0/48874973 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550402c070 con 0x55d984cf9760 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.950+0000 7f54f2ffd640 1 -- 172.21.15.32:0/48874973 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504004a00 con 0x55d984cf9760 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- 172.21.15.32:0/48874973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cf9760 msgr2=0x55d984d118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cf9760 0x55d984d118d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5504012470 tx=0x7f5504012f30 comp rx=0 tx=0).stop 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- 172.21.15.32:0/48874973 shutdown_connections 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984cf9760 0x55d984d118d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cec1e0 0x55d984ce55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 --2- 172.21.15.32:0/48874973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984ceb830 0x55d984d053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- 172.21.15.32:0/48874973 >> 172.21.15.32:0/48874973 conn(0x55d984ce6580 msgr2=0x55d984ce4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- 172.21.15.32:0/48874973 shutdown_connections 2024-04-16T05:39:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- 172.21.15.32:0/48874973 wait complete. 2024-04-16T05:39:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- start start 2024-04-16T05:39:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 Processor -- start 2024-04-16T05:39:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984d01610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984d01b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 0x55d984cffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55081ab780 con 0x55d984ceb830 2024-04-16T05:39:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55081acbf0 con 0x55d984cec1e0 2024-04-16T05:39:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.951+0000 7f551240c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55081abe50 con 0x55d984cf9760 2024-04-16T05:39:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f551240c740 10 client.0 _ll_register_callbacks cb 0x7f54f8009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984d01610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984d01610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45954/0 (socket says 172.21.15.32:45954) 2024-04-16T05:39:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 -- 172.21.15.32:0/4076072925 learned_addr learned my addr 172.21.15.32:0/4076072925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550cce4640 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984d01b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550dce6640 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 0x55d984cffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 -- 172.21.15.32:0/4076072925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 msgr2=0x55d984cffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 0x55d984cffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 -- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 msgr2=0x55d984d01b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984d01b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 -- 172.21.15.32:0/4076072925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d984d001e0 con 0x55d984ceb830 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550cce4640 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984d01b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:20.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f550d4e5640 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984d01610 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f55000be290 tx=0x7f550008fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3834fa695b751bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f54f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f54f0ff9640 1 -- 172.21.15.32:0/4076072925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550004b040 con 0x55d984ceb830 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f54f0ff9640 1 -- 172.21.15.32:0/4076072925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f55000efc20 con 0x55d984ceb830 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f551240c740 1 -- 172.21.15.32:0/4076072925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550402c070 con 0x55d984ceb830 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f54f0ff9640 1 -- 172.21.15.32:0/4076072925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55000be300 con 0x55d984ceb830 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f551240c740 10 client.16707 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f551240c740 1 -- 172.21.15.32:0/4076072925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d984d70cc0 con 0x55d984ceb830 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.952+0000 7f54f0ff9640 1 -- 172.21.15.32:0/4076072925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550402c070 con 0x55d984ceb830 2024-04-16T05:39:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f54f0ff9640 1 -- 172.21.15.32:0/4076072925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f55000f2260 con 0x55d984ceb830 2024-04-16T05:39:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f551240c740 10 client.16707 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f551240c740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f551240c740 20 client.16707 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f551240c740 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d984d713c0 0x55d984d737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f551240c740 4 client.16707 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:39:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f551240c740 1 -- 172.21.15.32:0/4076072925 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d984c58a10 con 0x55d984d713c0 2024-04-16T05:39:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f550cce4640 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d984d713c0 0x55d984d737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:20.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f54f0ff9640 10 client.16707 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.953+0000 7f550cce4640 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d984d713c0 0x55d984d737e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b1eb414ed4465 server_cookie=3f5376c203e8eee in_seq=0 out_seq=0 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f54f0ff9640 1 -- 172.21.15.32:0/4076072925 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d984c58a10 con 0x55d984d713c0 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f54f0ff9640 10 client.16707 handle_command_reply: tid=0 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 client.16707 shutdown 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 -- 172.21.15.32:0/4076072925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 msgr2=0x55d984d01610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984d01610 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f55000be290 tx=0x7f550008fd40 comp rx=0 tx=0).stop 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 -- 172.21.15.32:0/4076072925 shutdown_connections 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d984d713c0 0x55d984d737e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d984cf9760 0x55d984cffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d984cec1e0 0x55d984d01b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f54f0ff9640 0 client.16707 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 --2- 172.21.15.32:0/4076072925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d984ceb830 0x55d984d01610 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 -- 172.21.15.32:0/4076072925 >> 172.21.15.32:0/4076072925 conn(0x55d984ce6580 msgr2=0x55d984ce5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 -- 172.21.15.32:0/4076072925 shutdown_connections 2024-04-16T05:39:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 1 -- 172.21.15.32:0/4076072925 wait complete. 2024-04-16T05:39:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f551240c740 20 client.16707 trim_cache size 0 max 16384 2024-04-16T05:39:20.964 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:39:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:39:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:39:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "6ee82a35-9f39-4e5e-bb7e-dd2ba7ae4c2e": { 2024-04-16T05:39:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:39:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "6ee82a35-9f39-4e5e-bb7e-dd2ba7ae4c2e", 2024-04-16T05:39:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:39:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:39:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:39:20.965 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:39:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 -- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54f40a3d20 msgr2=0x7f54f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54f40a3d20 0x7f54f40a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5504010c40 tx=0x7f5504017000 comp rx=0 tx=0).stop 2024-04-16T05:39:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 -- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 msgr2=0x7f550819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 0x7f550819e640 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5500006010 tx=0x7f550002cb30 comp rx=0 tx=0).stop 2024-04-16T05:39:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 -- 172.21.15.32:0/3408480429 shutdown_connections 2024-04-16T05:39:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5508198700 0x7f5508198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5508107310 0x7f550819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54f40a3d20 0x7f54f40a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 --2- 172.21.15.32:0/3408480429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5508102240 0x7f550819e640 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.954+0000 7f550f749640 1 -- 172.21.15.32:0/3408480429 >> 172.21.15.32:0/3408480429 conn(0x7f55080fda40 msgr2=0x7f55080ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.955+0000 7f550f749640 1 -- 172.21.15.32:0/3408480429 shutdown_connections 2024-04-16T05:39:20.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:20.955+0000 7f550f749640 1 -- 172.21.15.32:0/3408480429 wait complete. 2024-04-16T05:39:20.968 INFO:tasks.cephfs.filesystem:scrub status for tag:6ee82a35-9f39-4e5e-bb7e-dd2ba7ae4c2e - {'path': '/', 'tag': '6ee82a35-9f39-4e5e-bb7e-dd2ba7ae4c2e', 'options': 'recursive,force'} 2024-04-16T05:39:21.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:21 smithi134 ceph-mon[36153]: pgmap v135: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:39:21.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:21 smithi082 ceph-mon[36195]: pgmap v135: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:39:21.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:21 smithi032 ceph-mon[32351]: pgmap v135: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:39:22.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:22 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:39:22.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:22 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:22.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:22 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:39:22.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:22 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:22 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:39:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:22 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:22 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:39:22.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:22 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:22 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:39:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:22 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:22 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:39:22.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:22 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:23.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:23 smithi134 ceph-mon[36153]: pgmap v136: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:39:23.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:23 smithi082 ceph-mon[36195]: pgmap v136: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:39:23.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:23 smithi032 ceph-mon[32351]: pgmap v136: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-16T05:39:25.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:25 smithi134 ceph-mon[36153]: pgmap v137: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:25.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:25 smithi082 ceph-mon[36195]: pgmap v137: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:25.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:25 smithi032 ceph-mon[32351]: pgmap v137: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:26.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:26 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:39:26.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:26 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:39:26.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:26 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:39:27.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:27 smithi134 ceph-mon[36153]: pgmap v138: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:27.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:27 smithi082 ceph-mon[36195]: pgmap v138: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:27.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:27 smithi032 ceph-mon[32351]: pgmap v138: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:29.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:29 smithi082 ceph-mon[36195]: pgmap v139: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:29.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:29 smithi032 ceph-mon[32351]: pgmap v139: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:29.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:29 smithi134 ceph-mon[36153]: pgmap v139: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:31.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:31 smithi082 ceph-mon[36195]: pgmap v140: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:31.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:31 smithi032 ceph-mon[32351]: pgmap v140: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:31 smithi134 ceph-mon[36153]: pgmap v140: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:39:33.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:33 smithi082 ceph-mon[36195]: pgmap v141: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:33.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:33 smithi032 ceph-mon[32351]: pgmap v141: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:33.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:33 smithi134 ceph-mon[36153]: pgmap v141: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:35.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:35 smithi082 ceph-mon[36195]: pgmap v142: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:35.686 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:35 smithi134 ceph-mon[36153]: pgmap v142: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:35.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:35 smithi032 ceph-mon[32351]: pgmap v142: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:37.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:37 smithi082 ceph-mon[36195]: pgmap v143: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:37.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:37 smithi032 ceph-mon[32351]: pgmap v143: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:37.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:37 smithi134 ceph-mon[36153]: pgmap v143: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:39.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:39 smithi032 ceph-mon[32351]: pgmap v144: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:39.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:39 smithi134 ceph-mon[36153]: pgmap v144: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:39.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:39 smithi082 ceph-mon[36195]: pgmap v144: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:41.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:41 smithi032 ceph-mon[32351]: pgmap v145: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:41.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:41 smithi134 ceph-mon[36153]: pgmap v145: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:41.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:41 smithi082 ceph-mon[36195]: pgmap v145: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:43.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:43 smithi032 ceph-mon[32351]: pgmap v146: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:43.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:43 smithi134 ceph-mon[36153]: pgmap v146: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:43.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:43 smithi082 ceph-mon[36195]: pgmap v146: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:45.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:45 smithi032 ceph-mon[32351]: pgmap v147: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:45.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:45 smithi134 ceph-mon[36153]: pgmap v147: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:45.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:45 smithi082 ceph-mon[36195]: pgmap v147: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:47.702 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:47 smithi082 ceph-mon[36195]: pgmap v148: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:47.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:47 smithi032 ceph-mon[32351]: pgmap v148: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:47.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:47 smithi134 ceph-mon[36153]: pgmap v148: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:49.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:49 smithi032 ceph-mon[32351]: pgmap v149: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:49.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:49 smithi134 ceph-mon[36153]: pgmap v149: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:49.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:49 smithi082 ceph-mon[36195]: pgmap v149: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:50.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:39:50.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:39:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.091+0000 7f927f559640 1 -- 172.21.15.32:0/182056244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9278101250 msgr2=0x7f92781016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.091+0000 7f927f559640 1 --2- 172.21.15.32:0/182056244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9278101250 0x7f92781016b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9268009930 tx=0x7f926802bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 -- 172.21.15.32:0/182056244 shutdown_connections 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 --2- 172.21.15.32:0/182056244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9278101bf0 0x7f927810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 --2- 172.21.15.32:0/182056244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9278101250 0x7f92781016b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 --2- 172.21.15.32:0/182056244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92781070e0 0x7f92781074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 -- 172.21.15.32:0/182056244 >> 172.21.15.32:0/182056244 conn(0x7f9278076d00 msgr2=0x7f9278077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 -- 172.21.15.32:0/182056244 shutdown_connections 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 -- 172.21.15.32:0/182056244 wait complete. 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 Processor -- start 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 -- start start 2024-04-16T05:39:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9278101250 0x7f927819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9278101bf0 0x7f927819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.092+0000 7f927f559640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92781070e0 0x7f9278198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927f559640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9278112420 con 0x7f92781070e0 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927f559640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92781122a0 con 0x7f9278101250 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927f559640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92781125a0 con 0x7f9278101bf0 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92781070e0 0x7f9278198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92781070e0 0x7f9278198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37324/0 (socket says 172.21.15.32:37324) 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 -- 172.21.15.32:0/2324368442 learned_addr learned my addr 172.21.15.32:0/2324368442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927d2f5640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9278101250 0x7f927819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927caf4640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9278101bf0 0x7f927819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 -- 172.21.15.32:0/2324368442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9278101bf0 msgr2=0x7f927819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9278101bf0 0x7f927819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 -- 172.21.15.32:0/2324368442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9278101250 msgr2=0x7f927819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9278101250 0x7f927819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 -- 172.21.15.32:0/2324368442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9278199000 con 0x7f92781070e0 2024-04-16T05:39:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927daf6640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92781070e0 0x7f9278198740 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f927000eaa0 tx=0x7f927000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f926e7fc640 1 -- 172.21.15.32:0/2324368442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927000cdb0 con 0x7f92781070e0 2024-04-16T05:39:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f926e7fc640 1 -- 172.21.15.32:0/2324368442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9270005020 con 0x7f92781070e0 2024-04-16T05:39:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927f559640 1 -- 172.21.15.32:0/2324368442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92781992f0 con 0x7f92781070e0 2024-04-16T05:39:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f926e7fc640 1 -- 172.21.15.32:0/2324368442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700116b0 con 0x7f92781070e0 2024-04-16T05:39:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.093+0000 7f927f559640 1 -- 172.21.15.32:0/2324368442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92781a5600 con 0x7f92781070e0 2024-04-16T05:39:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.096+0000 7f926e7fc640 1 -- 172.21.15.32:0/2324368442 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9270005560 con 0x7f92781070e0 2024-04-16T05:39:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.096+0000 7f9282221740 1 Processor -- start 2024-04-16T05:39:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.096+0000 7f9282221740 1 -- start start 2024-04-16T05:39:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f9282221740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f5f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f9282221740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f67e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f9282221740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 0x559ea8f6a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f9282221740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559ea8ec4180 con 0x559ea8f5ee50 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f9282221740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559ea8e61360 con 0x559ea8f5f770 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f9282221740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559ea8ec0140 con 0x559ea8f68350 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f5f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f926e7fc640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92640a3da0 0x7f92640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f5f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37326/0 (socket says 172.21.15.32:37326) 2024-04-16T05:39:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 -- 172.21.15.32:0/2302063766 learned_addr learned my addr 172.21.15.32:0/2302063766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927d2f5640 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f67e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927daf6640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92640a3da0 0x7f92640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 0x559ea8f6a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 -- 172.21.15.32:0/2302063766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 msgr2=0x559ea8f6a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 0x559ea8f6a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 -- 172.21.15.32:0/2302063766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 msgr2=0x559ea8f67e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f67e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 -- 172.21.15.32:0/2302063766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea8ca07d0 con 0x559ea8f5ee50 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f926e7fc640 1 -- 172.21.15.32:0/2324368442 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f92700d0d20 con 0x7f92781070e0 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927d2f5640 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f67e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.097+0000 7f927caf4640 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f5f230 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9268036040 tx=0x7f9268005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ca0ede74069a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f927daf6640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92640a3da0 0x7f92640a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f927009c680 tx=0x7f9270087040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f92637fe640 1 -- 172.21.15.32:0/2302063766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926802cb00 con 0x559ea8f5ee50 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f92637fe640 1 -- 172.21.15.32:0/2302063766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f926802cca0 con 0x559ea8f5ee50 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f92637fe640 1 -- 172.21.15.32:0/2302063766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268004920 con 0x559ea8f5ee50 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f9282221740 1 -- 172.21.15.32:0/2302063766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 msgr2=0x559ea8f5f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f9282221740 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f5f230 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9268036040 tx=0x7f9268005a20 comp rx=0 tx=0).stop 2024-04-16T05:39:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f9282221740 1 -- 172.21.15.32:0/2302063766 shutdown_connections 2024-04-16T05:39:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f9282221740 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 0x559ea8f6a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f9282221740 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f67e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f9282221740 1 --2- 172.21.15.32:0/2302063766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f5f230 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.098+0000 7f9282221740 1 -- 172.21.15.32:0/2302063766 >> 172.21.15.32:0/2302063766 conn(0x559ea8f59580 msgr2=0x559ea8f59960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 -- 172.21.15.32:0/2302063766 shutdown_connections 2024-04-16T05:39:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 -- 172.21.15.32:0/2302063766 wait complete. 2024-04-16T05:39:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 -- start start 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 Processor -- start 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 0x559ea8f65e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559ea8f6b080 con 0x559ea8f5ee50 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559ea8e7d2c0 con 0x559ea8f5f770 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559ea8f6b200 con 0x559ea8f68350 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f9282221740 10 client.0 _ll_register_callbacks cb 0x559ea8e64090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:51.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f927caf4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f927caf4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37338/0 (socket says 172.21.15.32:37338) 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.099+0000 7f927caf4640 1 -- 172.21.15.32:0/2337127171 learned_addr learned my addr 172.21.15.32:0/2337127171 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927caf4640 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 0x559ea8f65e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927d2f5640 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927caf4640 1 -- 172.21.15.32:0/2337127171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 msgr2=0x559ea8f65e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927caf4640 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 0x559ea8f65e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927caf4640 1 -- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 msgr2=0x559ea8f658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927caf4640 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927caf4640 1 -- 172.21.15.32:0/2337127171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea8bd63f0 con 0x559ea8f5ee50 2024-04-16T05:39:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927d2f5640 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f927caf4640 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f653a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9268000c00 tx=0x7f9268005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7378ff854bbcebc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f9260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f9260ff9640 1 -- 172.21.15.32:0/2337127171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268054060 con 0x559ea8f5ee50 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f9260ff9640 1 -- 172.21.15.32:0/2337127171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f92680035f0 con 0x559ea8f5ee50 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f9282221740 1 -- 172.21.15.32:0/2337127171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926802cca0 con 0x559ea8f5ee50 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f9260ff9640 1 -- 172.21.15.32:0/2337127171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9268004460 con 0x559ea8f5ee50 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f9282221740 10 client.16719 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.100+0000 7f9282221740 1 -- 172.21.15.32:0/2337127171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ea8b960f0 con 0x559ea8f5ee50 2024-04-16T05:39:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f9260ff9640 1 -- 172.21.15.32:0/2337127171 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f926802cca0 con 0x559ea8f5ee50 2024-04-16T05:39:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f9260ff9640 1 -- 172.21.15.32:0/2337127171 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f92680429d0 con 0x559ea8f5ee50 2024-04-16T05:39:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f9282221740 10 client.16719 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f9282221740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f9282221740 20 client.16719 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f9282221740 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559ea8f784f0 0x559ea8f7a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f9282221740 4 client.16719 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f9282221740 1 -- 172.21.15.32:0/2337127171 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea8ed1ba0 con 0x559ea8f784f0 2024-04-16T05:39:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.101+0000 7f927d2f5640 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559ea8f784f0 0x559ea8f7a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9260ff9640 10 client.16719 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f927d2f5640 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559ea8f784f0 0x559ea8f7a8e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa910c018714f915 server_cookie=ada5f872201ee643 in_seq=0 out_seq=0 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9260ff9640 1 -- 172.21.15.32:0/2337127171 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ea8ed1ba0 con 0x559ea8f784f0 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9260ff9640 10 client.16719 handle_command_reply: tid=0 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 client.16719 shutdown 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 -- 172.21.15.32:0/2337127171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 msgr2=0x559ea8f653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f653a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9268000c00 tx=0x7f9268005bd0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 -- 172.21.15.32:0/2337127171 shutdown_connections 2024-04-16T05:39:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559ea8f784f0 0x559ea8f7a8e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f68350 0x559ea8f65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5f770 0x559ea8f658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 --2- 172.21.15.32:0/2337127171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5ee50 0x559ea8f653a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9260ff9640 0 client.16719 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.102+0000 7f9282221740 1 -- 172.21.15.32:0/2337127171 >> 172.21.15.32:0/2337127171 conn(0x559ea8f59580 msgr2=0x559ea8f5b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.103+0000 7f9282221740 1 -- 172.21.15.32:0/2337127171 shutdown_connections 2024-04-16T05:39:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.103+0000 7f9282221740 1 -- 172.21.15.32:0/2337127171 wait complete. 2024-04-16T05:39:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.103+0000 7f9282221740 20 client.16719 trim_cache size 0 max 16384 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.120+0000 7f9282221740 1 Processor -- start 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.120+0000 7f9282221740 1 -- start start 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.120+0000 7f9282221740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5ee50 0x559ea8f67ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.120+0000 7f9282221740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5f770 0x559ea8f58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.120+0000 7f9282221740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f68350 0x559ea8f70860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.120+0000 7f9282221740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b1bd0 con 0x559ea8f68350 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.120+0000 7f9282221740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92781af4a0 con 0x559ea8f5ee50 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.120+0000 7f9282221740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92781ad2f0 con 0x559ea8f5f770 2024-04-16T05:39:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f68350 0x559ea8f70860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f68350 0x559ea8f70860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37346/0 (socket says 172.21.15.32:37346) 2024-04-16T05:39:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 -- 172.21.15.32:0/1385594568 learned_addr learned my addr 172.21.15.32:0/1385594568 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5ee50 0x559ea8f67ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927d2f5640 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5f770 0x559ea8f58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 -- 172.21.15.32:0/1385594568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5f770 msgr2=0x559ea8f58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5f770 0x559ea8f58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 -- 172.21.15.32:0/1385594568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5ee50 msgr2=0x559ea8f67ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5ee50 0x559ea8f67ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 -- 172.21.15.32:0/1385594568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea8f54fa0 con 0x559ea8f68350 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f927caf4640 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f68350 0x559ea8f70860 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9268042f10 tx=0x7f926802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69a1929236f0ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f9262ffd640 1 -- 172.21.15.32:0/1385594568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268054070 con 0x559ea8f68350 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f9262ffd640 1 -- 172.21.15.32:0/1385594568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f926804f790 con 0x559ea8f68350 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.121+0000 7f9262ffd640 1 -- 172.21.15.32:0/1385594568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92680426d0 con 0x559ea8f68350 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 -- 172.21.15.32:0/1385594568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f68350 msgr2=0x559ea8f70860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f68350 0x559ea8f70860 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9268042f10 tx=0x7f926802c6e0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 -- 172.21.15.32:0/1385594568 shutdown_connections 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f68350 0x559ea8f70860 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5f770 0x559ea8f58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 --2- 172.21.15.32:0/1385594568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f5ee50 0x559ea8f67ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 -- 172.21.15.32:0/1385594568 >> 172.21.15.32:0/1385594568 conn(0x559ea8f59580 msgr2=0x559ea8f5a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 -- 172.21.15.32:0/1385594568 shutdown_connections 2024-04-16T05:39:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 -- 172.21.15.32:0/1385594568 wait complete. 2024-04-16T05:39:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 -- start start 2024-04-16T05:39:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.122+0000 7f9282221740 1 Processor -- start 2024-04-16T05:39:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9282221740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5ee50 0x559ea8fd68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9282221740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5f770 0x559ea8fd6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9282221740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f68350 0x559ea8fe1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9282221740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92781ad960 con 0x559ea8f5f770 2024-04-16T05:39:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9282221740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92781aedd0 con 0x559ea8f68350 2024-04-16T05:39:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9282221740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92781ae030 con 0x559ea8f5ee50 2024-04-16T05:39:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9282221740 10 client.0 _ll_register_callbacks cb 0x7f9268009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5f770 0x559ea8fd6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5f770 0x559ea8fd6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37350/0 (socket says 172.21.15.32:37350) 2024-04-16T05:39:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 -- 172.21.15.32:0/1018159007 learned_addr learned my addr 172.21.15.32:0/1018159007 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927caf4640 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f68350 0x559ea8fe1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927caf4640 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5ee50 0x559ea8fd68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 -- 172.21.15.32:0/1018159007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5ee50 msgr2=0x559ea8fd68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5ee50 0x559ea8fd68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 -- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f68350 msgr2=0x559ea8fe1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f68350 0x559ea8fe1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 -- 172.21.15.32:0/1018159007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea8fe1d40 con 0x559ea8f5f770 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927caf4640 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f68350 0x559ea8fe1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f927d2f5640 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5f770 0x559ea8fd6de0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9274009c90 tx=0x7f9274004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd07f03cd43e0b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9260ff9640 1 -- 172.21.15.32:0/1018159007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274003df0 con 0x559ea8f5f770 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9260ff9640 1 -- 172.21.15.32:0/1018159007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f927401d3d0 con 0x559ea8f5f770 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.123+0000 7f9282221740 1 -- 172.21.15.32:0/1018159007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926804f790 con 0x559ea8f5f770 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9260ff9640 1 -- 172.21.15.32:0/1018159007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f927402c050 con 0x559ea8f5f770 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9282221740 10 client.16725 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9282221740 1 -- 172.21.15.32:0/1018159007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ea8fe30b0 con 0x559ea8f5f770 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9260ff9640 1 -- 172.21.15.32:0/1018159007 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f926804f790 con 0x559ea8f5f770 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9260ff9640 1 -- 172.21.15.32:0/1018159007 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f92740048c0 con 0x559ea8f5f770 2024-04-16T05:39:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9282221740 10 client.16725 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9282221740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9282221740 20 client.16725 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9282221740 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559ea8fe3850 0x559ea8fe5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9282221740 4 client.16725 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:39:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.124+0000 7f9282221740 1 -- 172.21.15.32:0/1018159007 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ea8f58de0 con 0x559ea8fe3850 2024-04-16T05:39:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f927caf4640 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559ea8fe3850 0x559ea8fe5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f927caf4640 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559ea8fe3850 0x559ea8fe5c70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1d869a7c6cc64f server_cookie=ba5469750e602723 in_seq=0 out_seq=0 2024-04-16T05:39:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9260ff9640 10 client.16725 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9260ff9640 1 -- 172.21.15.32:0/1018159007 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559ea8f58de0 con 0x559ea8fe3850 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9260ff9640 10 client.16725 handle_command_reply: tid=0 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 client.16725 shutdown 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 -- 172.21.15.32:0/1018159007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5f770 msgr2=0x559ea8fd6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5f770 0x559ea8fd6de0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9274009c90 tx=0x7f9274004410 comp rx=0 tx=0).stop 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 -- 172.21.15.32:0/1018159007 shutdown_connections 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea8f68350 0x559ea8fe1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ea8f5f770 0x559ea8fd6de0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559ea8fe3850 0x559ea8fe5c70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 --2- 172.21.15.32:0/1018159007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ea8f5ee50 0x559ea8fd68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9260ff9640 0 client.16725 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 -- 172.21.15.32:0/1018159007 >> 172.21.15.32:0/1018159007 conn(0x559ea8f59580 msgr2=0x559ea8f5ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 -- 172.21.15.32:0/1018159007 shutdown_connections 2024-04-16T05:39:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 1 -- 172.21.15.32:0/1018159007 wait complete. 2024-04-16T05:39:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.125+0000 7f9282221740 20 client.16725 trim_cache size 0 max 16384 2024-04-16T05:39:51.137 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:39:51.137 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:39:51.137 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:39:51.137 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:39:51.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 -- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92640a3da0 msgr2=0x7f92640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92640a3da0 0x7f92640a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f927009c680 tx=0x7f9270087040 comp rx=0 tx=0).stop 2024-04-16T05:39:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 -- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92781070e0 msgr2=0x7f9278198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92781070e0 0x7f9278198740 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f927000eaa0 tx=0x7f927000ef60 comp rx=0 tx=0).stop 2024-04-16T05:39:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 -- 172.21.15.32:0/2324368442 shutdown_connections 2024-04-16T05:39:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92640a3da0 0x7f92640a6260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92781070e0 0x7f9278198740 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9278101bf0 0x7f927819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 --2- 172.21.15.32:0/2324368442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9278101250 0x7f927819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 -- 172.21.15.32:0/2324368442 >> 172.21.15.32:0/2324368442 conn(0x7f9278076d00 msgr2=0x7f92780fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 -- 172.21.15.32:0/2324368442 shutdown_connections 2024-04-16T05:39:51.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.126+0000 7f927f559640 1 -- 172.21.15.32:0/2324368442 wait complete. 2024-04-16T05:39:51.140 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:39:51.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:39:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 -- 172.21.15.32:0/3071735854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 msgr2=0x7ff1b410c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 --2- 172.21.15.32:0/3071735854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 0x7ff1b410c060 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009900 tx=0x7ff1a402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 -- 172.21.15.32:0/3071735854 shutdown_connections 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 --2- 172.21.15.32:0/3071735854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b410c630 0x7ff1b410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 --2- 172.21.15.32:0/3071735854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 0x7ff1b410c060 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 --2- 172.21.15.32:0/3071735854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b4103260 0x7ff1b4103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 -- 172.21.15.32:0/3071735854 >> 172.21.15.32:0/3071735854 conn(0x7ff1b4077280 msgr2=0x7ff1b4077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 -- 172.21.15.32:0/3071735854 shutdown_connections 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.247+0000 7ff1bb338640 1 -- 172.21.15.32:0/3071735854 wait complete. 2024-04-16T05:39:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1bb338640 1 Processor -- start 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1bb338640 1 -- start start 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1bb338640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b4103260 0x7ff1b419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1bb338640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 0x7ff1b419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1bb338640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b410c630 0x7ff1b41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1bb338640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41123c0 con 0x7ff1b4103b80 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1bb338640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4112240 con 0x7ff1b4103260 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1bb338640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4112540 con 0x7ff1b410c630 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b88d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 0x7ff1b419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b88d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 0x7ff1b419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37368/0 (socket says 172.21.15.32:37368) 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b88d3640 1 -- 172.21.15.32:0/747425252 learned_addr learned my addr 172.21.15.32:0/747425252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b90d4640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b4103260 0x7ff1b419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b98d5640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b410c630 0x7ff1b41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b88d3640 1 -- 172.21.15.32:0/747425252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b410c630 msgr2=0x7ff1b41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b88d3640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b410c630 0x7ff1b41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b88d3640 1 -- 172.21.15.32:0/747425252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b4103260 msgr2=0x7ff1b419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b88d3640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b4103260 0x7ff1b419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.248+0000 7ff1b88d3640 1 -- 172.21.15.32:0/747425252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b4199060 con 0x7ff1b4103b80 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.249+0000 7ff1b88d3640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 0x7ff1b419ebd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4005940 tx=0x7ff1a402cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.249+0000 7ff1aa7fc640 1 -- 172.21.15.32:0/747425252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4041070 con 0x7ff1b4103b80 2024-04-16T05:39:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.249+0000 7ff1aa7fc640 1 -- 172.21.15.32:0/747425252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a4004450 con 0x7ff1b4103b80 2024-04-16T05:39:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.249+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b41992f0 con 0x7ff1b4103b80 2024-04-16T05:39:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.249+0000 7ff1aa7fc640 1 -- 172.21.15.32:0/747425252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4034440 con 0x7ff1b4103b80 2024-04-16T05:39:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.249+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41a5430 con 0x7ff1b4103b80 2024-04-16T05:39:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.250+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b406aec0 con 0x7ff1b4103b80 2024-04-16T05:39:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.251+0000 7ff1aa7fc640 1 -- 172.21.15.32:0/747425252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1a4044070 con 0x7ff1b4103b80 2024-04-16T05:39:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.251+0000 7ff1aa7fc640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff1a00a3d20 0x7ff1a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.252+0000 7ff1b90d4640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff1a00a3d20 0x7ff1a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.252+0000 7ff1aa7fc640 1 -- 172.21.15.32:0/747425252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff1a40f25c0 con 0x7ff1b4103b80 2024-04-16T05:39:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.252+0000 7ff1b90d4640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff1a00a3d20 0x7ff1a00a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00058c0 tx=0x7ff1b000a250 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.256+0000 7ff1aa7fc640 1 -- 172.21.15.32:0/747425252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1a40be3e0 con 0x7ff1b4103b80 2024-04-16T05:39:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.482+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1b4107720 con 0x7ff1b4103b80 2024-04-16T05:39:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.483+0000 7ff1aa7fc640 1 -- 172.21.15.32:0/747425252 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7ff1a40c26e0 con 0x7ff1b4103b80 2024-04-16T05:39:51.482 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:39:51.482 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:39:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:39:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff1a00a3d20 msgr2=0x7ff1a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff1a00a3d20 0x7ff1a00a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00058c0 tx=0x7ff1b000a250 comp rx=0 tx=0).stop 2024-04-16T05:39:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 msgr2=0x7ff1b419ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 0x7ff1b419ebd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4005940 tx=0x7ff1a402cd10 comp rx=0 tx=0).stop 2024-04-16T05:39:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 shutdown_connections 2024-04-16T05:39:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b410c630 0x7ff1b41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff1a00a3d20 0x7ff1a00a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1b4103b80 0x7ff1b419ebd0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 --2- 172.21.15.32:0/747425252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b4103260 0x7ff1b419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 >> 172.21.15.32:0/747425252 conn(0x7ff1b4077280 msgr2=0x7ff1b40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 shutdown_connections 2024-04-16T05:39:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.485+0000 7ff1bb338640 1 -- 172.21.15.32:0/747425252 wait complete. 2024-04-16T05:39:51.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:39:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 -- 172.21.15.32:0/2720708439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc106200 msgr2=0x7ff2dc10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2720708439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc106200 0x7ff2dc10ddd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009ca0 tx=0x7ff2d402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 -- 172.21.15.32:0/2720708439 shutdown_connections 2024-04-16T05:39:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2720708439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc106200 0x7ff2dc10ddd0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2720708439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2dc101ae0 0x7ff2dc105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2720708439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2dc101130 0x7ff2dc101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 -- 172.21.15.32:0/2720708439 >> 172.21.15.32:0/2720708439 conn(0x7ff2dc0780b0 msgr2=0x7ff2dc0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 -- 172.21.15.32:0/2720708439 shutdown_connections 2024-04-16T05:39:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 -- 172.21.15.32:0/2720708439 wait complete. 2024-04-16T05:39:51.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.604+0000 7ff2e087b640 1 Processor -- start 2024-04-16T05:39:51.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2e087b640 1 -- start start 2024-04-16T05:39:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2e087b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2dc101130 0x7ff2dc19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2e087b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2dc106200 0x7ff2dc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2e087b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc198730 0x7ff2dc198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2e087b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc112410 con 0x7ff2dc198730 2024-04-16T05:39:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2e087b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc112290 con 0x7ff2dc106200 2024-04-16T05:39:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2e087b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc112590 con 0x7ff2dc101130 2024-04-16T05:39:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc198730 0x7ff2dc198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc198730 0x7ff2dc198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37384/0 (socket says 172.21.15.32:37384) 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 -- 172.21.15.32:0/2872651002 learned_addr learned my addr 172.21.15.32:0/2872651002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2da59c640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2dc101130 0x7ff2dc19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2dc106200 0x7ff2dc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 -- 172.21.15.32:0/2872651002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2dc101130 msgr2=0x7ff2dc19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2dc101130 0x7ff2dc19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 -- 172.21.15.32:0/2872651002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2dc106200 msgr2=0x7ff2dc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2dc106200 0x7ff2dc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 -- 172.21.15.32:0/2872651002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc199330 con 0x7ff2dc198730 2024-04-16T05:39:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2da59c640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2dc101130 0x7ff2dc19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2dad9d640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc198730 0x7ff2dc198bb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff2d402c700 tx=0x7ff2d4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2d37fe640 1 -- 172.21.15.32:0/2872651002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4039040 con 0x7ff2dc198730 2024-04-16T05:39:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.605+0000 7ff2d37fe640 1 -- 172.21.15.32:0/2872651002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2d4044070 con 0x7ff2dc198730 2024-04-16T05:39:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.606+0000 7ff2e087b640 1 -- 172.21.15.32:0/2872651002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc1995c0 con 0x7ff2dc198730 2024-04-16T05:39:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.606+0000 7ff2d37fe640 1 -- 172.21.15.32:0/2872651002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40042a0 con 0x7ff2dc198730 2024-04-16T05:39:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.606+0000 7ff2e087b640 1 -- 172.21.15.32:0/2872651002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc1a56b0 con 0x7ff2dc198730 2024-04-16T05:39:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.608+0000 7ff2d37fe640 1 -- 172.21.15.32:0/2872651002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff2d4004440 con 0x7ff2dc198730 2024-04-16T05:39:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.608+0000 7ff2e346f740 1 Processor -- start 2024-04-16T05:39:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2e346f740 1 -- start start 2024-04-16T05:39:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 0x5596831bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2e346f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5596831170e0 con 0x5596831b1e60 2024-04-16T05:39:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2e346f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5596830b4520 con 0x5596831bb360 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2e346f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596831130a0 con 0x5596831b2780 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37400/0 (socket says 172.21.15.32:37400) 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 -- 172.21.15.32:0/453299497 learned_addr learned my addr 172.21.15.32:0/453299497 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d37fe640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff2c80a3d20 0x7ff2c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 0x5596831bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2da59c640 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2dad9d640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff2c80a3d20 0x7ff2c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 -- 172.21.15.32:0/453299497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 msgr2=0x5596831bae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831bae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 -- 172.21.15.32:0/453299497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 msgr2=0x5596831bd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 0x5596831bd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d37fe640 1 -- 172.21.15.32:0/2872651002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff2d40f2e80 con 0x7ff2dc198730 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.609+0000 7ff2d3fff640 1 -- 172.21.15.32:0/453299497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596830dc8d0 con 0x5596831b1e60 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.610+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831b2240 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400bef0 tx=0x7ff2c400b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9928353558dc2a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.610+0000 7ff2dad9d640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff2c80a3d20 0x7ff2c80a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40be8f0 tx=0x7ff2d402c0f0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.610+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/453299497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40173f0 con 0x5596831b1e60 2024-04-16T05:39:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.610+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/453299497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c4010040 con 0x5596831b1e60 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.610+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/453299497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4016880 con 0x5596831b1e60 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- 172.21.15.32:0/453299497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 msgr2=0x5596831b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831b2240 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400bef0 tx=0x7ff2c400b400 comp rx=0 tx=0).stop 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- 172.21.15.32:0/453299497 shutdown_connections 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 0x5596831bd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 --2- 172.21.15.32:0/453299497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831b2240 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- 172.21.15.32:0/453299497 >> 172.21.15.32:0/453299497 conn(0x5596831ac590 msgr2=0x5596831ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- 172.21.15.32:0/453299497 shutdown_connections 2024-04-16T05:39:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- 172.21.15.32:0/453299497 wait complete. 2024-04-16T05:39:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- start start 2024-04-16T05:39:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 Processor -- start 2024-04-16T05:39:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 0x5596831b8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5596831bdfa0 con 0x5596831bb360 2024-04-16T05:39:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5596830d04d0 con 0x5596831b1e60 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596831be120 con 0x5596831b2780 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.611+0000 7ff2e346f740 10 client.0 _ll_register_callbacks cb 0x5596830b52c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831b8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37414/0 (socket says 172.21.15.32:37414) 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 -- 172.21.15.32:0/1987449477 learned_addr learned my addr 172.21.15.32:0/1987449477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2da59c640 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 0x5596831b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 -- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 msgr2=0x5596831b8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 -- 172.21.15.32:0/1987449477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 msgr2=0x5596831b8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 0x5596831b8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 -- 172.21.15.32:0/1987449477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559682e293f0 con 0x5596831bb360 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2da59c640 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831b8ec0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4007fd0 tx=0x7ff2c40164e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19b1dd0d71ba6d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/1987449477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4029070 con 0x5596831bb360 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/1987449477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff2c40175f0 con 0x5596831bb360 2024-04-16T05:39:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2e346f740 1 -- 172.21.15.32:0/1987449477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c4010040 con 0x5596831bb360 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/1987449477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2c400fe60 con 0x5596831bb360 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2e346f740 10 client.16743 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2e346f740 1 -- 172.21.15.32:0/1987449477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559682ef37d0 con 0x5596831bb360 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.612+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/1987449477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c4010040 con 0x5596831bb360 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/1987449477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2c4006020 con 0x5596831bb360 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2e346f740 10 client.16743 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2e346f740 10 client.16743 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2e346f740 20 client.16743 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5596831cb2b0 0x5596831cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2e346f740 4 client.16743 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2e346f740 1 -- 172.21.15.32:0/1987449477 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559683124b00 con 0x5596831cb2b0 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2da59c640 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5596831cb2b0 0x5596831cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2c1ffb640 10 client.16743 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.613+0000 7ff2da59c640 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5596831cb2b0 0x5596831cd6a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec523aabe3cebe43 server_cookie=a74145843f735ce8 in_seq=0 out_seq=0 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/1987449477 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559683124b00 con 0x5596831cb2b0 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2c1ffb640 10 client.16743 handle_command_reply: tid=0 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 client.16743 shutdown 2024-04-16T05:39:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 -- 172.21.15.32:0/1987449477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 msgr2=0x5596831b8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831b8ec0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4007fd0 tx=0x7ff2c40164e0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 -- 172.21.15.32:0/1987449477 shutdown_connections 2024-04-16T05:39:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5596831cb2b0 0x5596831cd6a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831b8ec0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2c1ffb640 0 client.16743 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:39:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1987449477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 0x5596831b8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 -- 172.21.15.32:0/1987449477 >> 172.21.15.32:0/1987449477 conn(0x5596831ac590 msgr2=0x5596831ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 -- 172.21.15.32:0/1987449477 shutdown_connections 2024-04-16T05:39:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.614+0000 7ff2e346f740 1 -- 172.21.15.32:0/1987449477 wait complete. 2024-04-16T05:39:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.615+0000 7ff2e346f740 20 client.16743 trim_cache size 0 max 16384 2024-04-16T05:39:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2e346f740 1 Processor -- start 2024-04-16T05:39:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2e346f740 1 -- start start 2024-04-16T05:39:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 0x5596831bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2e346f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b1c40 con 0x5596831bb360 2024-04-16T05:39:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2e346f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1af510 con 0x5596831b1e60 2024-04-16T05:39:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2e346f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1ad360 con 0x5596831b2780 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831ab5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37424/0 (socket says 172.21.15.32:37424) 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 -- 172.21.15.32:0/1796285825 learned_addr learned my addr 172.21.15.32:0/1796285825 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 0x5596831bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2da59c640 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 -- 172.21.15.32:0/1796285825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 msgr2=0x5596831c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 -- 172.21.15.32:0/1796285825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 msgr2=0x5596831bab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 0x5596831bab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 -- 172.21.15.32:0/1796285825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596831a7e60 con 0x5596831bb360 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.628+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831ab5b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400bef0 tx=0x7ff2c4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc6a824977e346 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2c3fff640 1 -- 172.21.15.32:0/1796285825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4029070 con 0x5596831bb360 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2c3fff640 1 -- 172.21.15.32:0/1796285825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c400b150 con 0x5596831bb360 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2c3fff640 1 -- 172.21.15.32:0/1796285825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4017650 con 0x5596831bb360 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 -- 172.21.15.32:0/1796285825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 msgr2=0x5596831ab5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831ab5b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400bef0 tx=0x7ff2c4009040 comp rx=0 tx=0).stop 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 -- 172.21.15.32:0/1796285825 shutdown_connections 2024-04-16T05:39:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831bb360 0x5596831ab5b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 --2- 172.21.15.32:0/1796285825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831b1e60 0x5596831bab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 -- 172.21.15.32:0/1796285825 >> 172.21.15.32:0/1796285825 conn(0x5596831ac590 msgr2=0x5596831aaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 -- 172.21.15.32:0/1796285825 shutdown_connections 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 -- 172.21.15.32:0/1796285825 wait complete. 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.629+0000 7ff2e346f740 1 -- start start 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2e346f740 1 Processor -- start 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2e346f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 0x5596831b5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2e346f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1ad9d0 con 0x5596831b1e60 2024-04-16T05:39:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2e346f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1aee40 con 0x5596831bb360 2024-04-16T05:39:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2e346f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1ae0a0 con 0x5596831b2780 2024-04-16T05:39:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2e346f740 10 client.0 _ll_register_callbacks cb 0x7ff2cc00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831c3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37430/0 (socket says 172.21.15.32:37430) 2024-04-16T05:39:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 -- 172.21.15.32:0/4272606725 learned_addr learned my addr 172.21.15.32:0/4272606725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2da59c640 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 0x5596831b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 -- 172.21.15.32:0/4272606725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 msgr2=0x5596831b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 -- 172.21.15.32:0/4272606725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 msgr2=0x5596831b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 0x5596831b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 -- 172.21.15.32:0/4272606725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596831b5970 con 0x5596831b1e60 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2d3fff640 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831c3770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4011010 tx=0x7ff2c400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0733c5779340092 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.630+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/4272606725 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4029070 con 0x5596831b1e60 2024-04-16T05:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/4272606725 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7ff2c4005dd0 con 0x5596831b1e60 2024-04-16T05:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 1 -- 172.21.15.32:0/4272606725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c400b150 con 0x5596831b1e60 2024-04-16T05:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/4272606725 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2c400bcc0 con 0x5596831b1e60 2024-04-16T05:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 10 client.16749 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 1 -- 172.21.15.32:0/4272606725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5596832315f0 con 0x5596831b1e60 2024-04-16T05:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/4272606725 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c400b150 con 0x5596831b1e60 2024-04-16T05:39:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/4272606725 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2c4039040 con 0x5596831b1e60 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 10 client.16749 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 10 client.16749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 20 client.16749 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5596831cb2b0 0x559683233c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 4 client.16749 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2e346f740 1 -- 172.21.15.32:0/4272606725 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55968311e970 con 0x5596831cb2b0 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.631+0000 7ff2da59c640 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5596831cb2b0 0x559683233c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2da59c640 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5596831cb2b0 0x559683233c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e420a644268897cb server_cookie=1ffd2ac5cc1211b3 in_seq=0 out_seq=0 2024-04-16T05:39:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2c1ffb640 10 client.16749 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2c1ffb640 1 -- 172.21.15.32:0/4272606725 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55968311e970 con 0x5596831cb2b0 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2c1ffb640 10 client.16749 handle_command_reply: tid=0 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 client.16749 shutdown 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 -- 172.21.15.32:0/4272606725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 msgr2=0x5596831c3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831c3770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4011010 tx=0x7ff2c400bb90 comp rx=0 tx=0).stop 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 -- 172.21.15.32:0/4272606725 shutdown_connections 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5596831cb2b0 0x559683233c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596831bb360 0x5596831b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2c1ffb640 0 client.16749 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596831b2780 0x5596831b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 --2- 172.21.15.32:0/4272606725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596831b1e60 0x5596831c3770 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 -- 172.21.15.32:0/4272606725 >> 172.21.15.32:0/4272606725 conn(0x5596831ac590 msgr2=0x5596831b8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 -- 172.21.15.32:0/4272606725 shutdown_connections 2024-04-16T05:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 1 -- 172.21.15.32:0/4272606725 wait complete. 2024-04-16T05:39:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.632+0000 7ff2e346f740 20 client.16749 trim_cache size 0 max 16384 2024-04-16T05:39:51.641 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 -- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff2c80a3d20 msgr2=0x7ff2c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff2c80a3d20 0x7ff2c80a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40be8f0 tx=0x7ff2d402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 -- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc198730 msgr2=0x7ff2dc198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc198730 0x7ff2dc198bb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff2d402c700 tx=0x7ff2d4032f20 comp rx=0 tx=0).stop 2024-04-16T05:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 -- 172.21.15.32:0/2872651002 shutdown_connections 2024-04-16T05:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff2c80a3d20 0x7ff2c80a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2dc198730 0x7ff2dc198bb0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2dc106200 0x7ff2dc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 --2- 172.21.15.32:0/2872651002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2dc101130 0x7ff2dc19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 -- 172.21.15.32:0/2872651002 >> 172.21.15.32:0/2872651002 conn(0x7ff2dc0780b0 msgr2=0x7ff2dc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 -- 172.21.15.32:0/2872651002 shutdown_connections 2024-04-16T05:39:51.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.633+0000 7ff2e087b640 1 -- 172.21.15.32:0/2872651002 wait complete. 2024-04-16T05:39:51.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:39:51.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:51 smithi032 ceph-mon[32351]: pgmap v150: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 -- 172.21.15.32:0/1701019506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102c00 msgr2=0x7fd53c106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 --2- 172.21.15.32:0/1701019506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102c00 0x7fd53c106c40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009900 tx=0x7fd52c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 -- 172.21.15.32:0/1701019506 shutdown_connections 2024-04-16T05:39:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 --2- 172.21.15.32:0/1701019506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd53c107320 0x7fd53c10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 --2- 172.21.15.32:0/1701019506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102c00 0x7fd53c106c40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 --2- 172.21.15.32:0/1701019506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd53c102250 0x7fd53c102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 -- 172.21.15.32:0/1701019506 >> 172.21.15.32:0/1701019506 conn(0x7fd53c079490 msgr2=0x7fd53c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 -- 172.21.15.32:0/1701019506 shutdown_connections 2024-04-16T05:39:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.753+0000 7fd544293640 1 -- 172.21.15.32:0/1701019506 wait complete. 2024-04-16T05:39:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd544293640 1 Processor -- start 2024-04-16T05:39:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd544293640 1 -- start start 2024-04-16T05:39:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd544293640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102250 0x7fd53c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd544293640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd53c107320 0x7fd53c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd544293640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd53c198690 0x7fd53c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd544293640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c112390 con 0x7fd53c102250 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd544293640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c112210 con 0x7fd53c107320 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd544293640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c112510 con 0x7fd53c198690 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd54202f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102250 0x7fd53c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd54202f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102250 0x7fd53c19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37436/0 (socket says 172.21.15.32:37436) 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd54202f640 1 -- 172.21.15.32:0/194816200 learned_addr learned my addr 172.21.15.32:0/194816200 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd542830640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd53c198690 0x7fd53c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd54182e640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd53c107320 0x7fd53c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd54202f640 1 -- 172.21.15.32:0/194816200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd53c198690 msgr2=0x7fd53c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.754+0000 7fd54202f640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd53c198690 0x7fd53c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd54202f640 1 -- 172.21.15.32:0/194816200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd53c107320 msgr2=0x7fd53c19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd54202f640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd53c107320 0x7fd53c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd54202f640 1 -- 172.21.15.32:0/194816200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c199290 con 0x7fd53c102250 2024-04-16T05:39:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd54202f640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102250 0x7fd53c19e5d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd53800d950 tx=0x7fd53800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd5337fe640 1 -- 172.21.15.32:0/194816200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538017030 con 0x7fd53c102250 2024-04-16T05:39:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd5337fe640 1 -- 172.21.15.32:0/194816200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd538004020 con 0x7fd53c102250 2024-04-16T05:39:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd544293640 1 -- 172.21.15.32:0/194816200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c1a5370 con 0x7fd53c102250 2024-04-16T05:39:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd5337fe640 1 -- 172.21.15.32:0/194816200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5380047a0 con 0x7fd53c102250 2024-04-16T05:39:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.755+0000 7fd544293640 1 -- 172.21.15.32:0/194816200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c1a5770 con 0x7fd53c102250 2024-04-16T05:39:51.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.757+0000 7fd5337fe640 1 -- 172.21.15.32:0/194816200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd5380041d0 con 0x7fd53c102250 2024-04-16T05:39:51.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.757+0000 7fd546b22740 1 Processor -- start 2024-04-16T05:39:51.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd546b22740 1 -- start start 2024-04-16T05:39:51.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd546b22740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb12240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd546b22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb1ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd546b22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb1d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd546b22740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bfeda770e0 con 0x55bfedb11e60 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd546b22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bfeda14520 con 0x55bfedb12780 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd546b22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bfeda730a0 con 0x55bfedb1b360 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd542830640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb12240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd5337fe640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5280a3d20 0x7fd5280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd542830640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb12240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37452/0 (socket says 172.21.15.32:37452) 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd542830640 1 -- 172.21.15.32:0/1495546912 learned_addr learned my addr 172.21.15.32:0/1495546912 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd54202f640 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb1ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd542830640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5280a3d20 0x7fd5280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd54182e640 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb1d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.758+0000 7fd5337fe640 1 -- 172.21.15.32:0/194816200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fd5380d0640 con 0x7fd53c102250 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd542830640 1 -- 172.21.15.32:0/1495546912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 msgr2=0x55bfedb1d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd542830640 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb1d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd542830640 1 -- 172.21.15.32:0/1495546912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 msgr2=0x55bfedb1ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd542830640 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb1ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd542830640 1 -- 172.21.15.32:0/1495546912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfeda3c8d0 con 0x55bfedb11e60 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd542830640 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb12240 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd534014940 tx=0x7fd534014e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ffffcf87eaece6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd530bfe640 1 -- 172.21.15.32:0/1495546912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534014050 con 0x55bfedb11e60 2024-04-16T05:39:51.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd530bfe640 1 -- 172.21.15.32:0/1495546912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5340141f0 con 0x55bfedb11e60 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd530bfe640 1 -- 172.21.15.32:0/1495546912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53401e650 con 0x55bfedb11e60 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.759+0000 7fd542830640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5280a3d20 0x7fd5280a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd534021000 tx=0x7fd534020000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 -- 172.21.15.32:0/1495546912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 msgr2=0x55bfedb12240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb12240 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd534014940 tx=0x7fd534014e00 comp rx=0 tx=0).stop 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 -- 172.21.15.32:0/1495546912 shutdown_connections 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb1d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb1ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 --2- 172.21.15.32:0/1495546912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb12240 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 -- 172.21.15.32:0/1495546912 >> 172.21.15.32:0/1495546912 conn(0x55bfedb0c590 msgr2=0x55bfedb0c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 -- 172.21.15.32:0/1495546912 shutdown_connections 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 -- 172.21.15.32:0/1495546912 wait complete. 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 -- start start 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.760+0000 7fd546b22740 1 Processor -- start 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb18460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb18ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bfedb1dfa0 con 0x55bfedb11e60 2024-04-16T05:39:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bfeda304d0 con 0x55bfedb12780 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bfedb1e120 con 0x55bfedb1b360 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 10 client.0 _ll_register_callbacks cb 0x55bfeda152c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb18460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb18460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37460/0 (socket says 172.21.15.32:37460) 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 -- 172.21.15.32:0/4268957965 learned_addr learned my addr 172.21.15.32:0/4268957965 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54182e640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd542830640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb18ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 -- 172.21.15.32:0/4268957965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 msgr2=0x55bfedb18ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb18ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 -- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 msgr2=0x55bfedb189a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb189a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 -- 172.21.15.32:0/4268957965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfed7893f0 con 0x55bfedb11e60 2024-04-16T05:39:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54182e640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd542830640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb18ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd54202f640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb18460 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd538098c10 tx=0x7fd53801af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710df62fd3057616 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd525ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd525ffb640 1 -- 172.21.15.32:0/4268957965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538020160 con 0x55bfedb11e60 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd525ffb640 1 -- 172.21.15.32:0/4268957965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fd538018d30 con 0x55bfedb11e60 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 1 -- 172.21.15.32:0/4268957965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5340141f0 con 0x55bfedb11e60 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd525ffb640 1 -- 172.21.15.32:0/4268957965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd53809c240 con 0x55bfedb11e60 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 10 client.16761 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.761+0000 7fd546b22740 1 -- 172.21.15.32:0/4268957965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfed8537d0 con 0x55bfedb11e60 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd525ffb640 1 -- 172.21.15.32:0/4268957965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5340141f0 con 0x55bfedb11e60 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd525ffb640 1 -- 172.21.15.32:0/4268957965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd5380b0ce0 con 0x55bfedb11e60 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd546b22740 10 client.16761 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd546b22740 10 client.16761 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:39:51.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd546b22740 20 client.16761 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd546b22740 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55bfedb2b320 0x55bfedb2d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd546b22740 4 client.16761 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd546b22740 1 -- 172.21.15.32:0/4268957965 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfeda84b00 con 0x55bfedb2b320 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.762+0000 7fd54182e640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55bfedb2b320 0x55bfedb2d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd54182e640 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55bfedb2b320 0x55bfedb2d790 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=328b96f08a976a2a server_cookie=6808decb9c943c48 in_seq=0 out_seq=0 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd525ffb640 10 client.16761 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd525ffb640 1 -- 172.21.15.32:0/4268957965 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfeda84b00 con 0x55bfedb2b320 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd525ffb640 10 client.16761 handle_command_reply: tid=0 2024-04-16T05:39:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 client.16761 shutdown 2024-04-16T05:39:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 -- 172.21.15.32:0/4268957965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 msgr2=0x55bfedb18460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb18460 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd538098c10 tx=0x7fd53801af90 comp rx=0 tx=0).stop 2024-04-16T05:39:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 -- 172.21.15.32:0/4268957965 shutdown_connections 2024-04-16T05:39:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55bfedb2b320 0x55bfedb2d790 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb1b360 0x55bfedb18ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd525ffb640 0 client.16761 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:39:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb12780 0x55bfedb189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 --2- 172.21.15.32:0/4268957965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb11e60 0x55bfedb18460 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.763+0000 7fd546b22740 1 -- 172.21.15.32:0/4268957965 >> 172.21.15.32:0/4268957965 conn(0x55bfedb0c590 msgr2=0x55bfedb0e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.764+0000 7fd546b22740 1 -- 172.21.15.32:0/4268957965 shutdown_connections 2024-04-16T05:39:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.764+0000 7fd546b22740 1 -- 172.21.15.32:0/4268957965 wait complete. 2024-04-16T05:39:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.764+0000 7fd546b22740 20 client.16761 trim_cache size 0 max 16384 2024-04-16T05:39:51.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd546b22740 1 Processor -- start 2024-04-16T05:39:51.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd546b22740 1 -- start start 2024-04-16T05:39:51.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd546b22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb11e60 0x55bfedb1ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd546b22740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb12780 0x55bfedb26f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd546b22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb1b360 0x55bfedb175c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd546b22740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1b1c50 con 0x55bfedb12780 2024-04-16T05:39:51.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd546b22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1af520 con 0x55bfedb1b360 2024-04-16T05:39:51.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd546b22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1ad370 con 0x55bfedb11e60 2024-04-16T05:39:51.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb12780 0x55bfedb26f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb12780 0x55bfedb26f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37462/0 (socket says 172.21.15.32:37462) 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 -- 172.21.15.32:0/3346761129 learned_addr learned my addr 172.21.15.32:0/3346761129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54202f640 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb11e60 0x55bfedb1ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 -- 172.21.15.32:0/3346761129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb11e60 msgr2=0x55bfedb1ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd542830640 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb1b360 0x55bfedb175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb11e60 0x55bfedb1ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 -- 172.21.15.32:0/3346761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb1b360 msgr2=0x55bfedb175c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb1b360 0x55bfedb175c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 -- 172.21.15.32:0/3346761129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfedb07c30 con 0x55bfedb12780 2024-04-16T05:39:51.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54202f640 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb11e60 0x55bfedb1ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd542830640 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb1b360 0x55bfedb175c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.777+0000 7fd54182e640 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb12780 0x55bfedb26f40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0096c0 tx=0x7fd52c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a51d19df9ebd4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd527fff640 1 -- 172.21.15.32:0/3346761129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004380 con 0x55bfedb12780 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd527fff640 1 -- 172.21.15.32:0/3346761129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c043070 con 0x55bfedb12780 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd527fff640 1 -- 172.21.15.32:0/3346761129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004920 con 0x55bfedb12780 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 -- 172.21.15.32:0/3346761129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb12780 msgr2=0x55bfedb26f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb12780 0x55bfedb26f40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0096c0 tx=0x7fd52c005fd0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 -- 172.21.15.32:0/3346761129 shutdown_connections 2024-04-16T05:39:51.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb1b360 0x55bfedb175c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb12780 0x55bfedb26f40 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 --2- 172.21.15.32:0/3346761129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb11e60 0x55bfedb1ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 -- 172.21.15.32:0/3346761129 >> 172.21.15.32:0/3346761129 conn(0x55bfedb0c590 msgr2=0x55bfedb0aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 -- 172.21.15.32:0/3346761129 shutdown_connections 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 -- 172.21.15.32:0/3346761129 wait complete. 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.778+0000 7fd546b22740 1 -- start start 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd546b22740 1 Processor -- start 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd546b22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb11e60 0x55bfedb14fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd546b22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb12780 0x55bfedb15520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd546b22740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb1b360 0x55bfedb15a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd546b22740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1ad9e0 con 0x55bfedb1b360 2024-04-16T05:39:51.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd546b22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1aee50 con 0x55bfedb11e60 2024-04-16T05:39:51.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd546b22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1ae0b0 con 0x55bfedb12780 2024-04-16T05:39:51.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd546b22740 10 client.0 _ll_register_callbacks cb 0x7fd5380984e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:51.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb1b360 0x55bfedb15a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb1b360 0x55bfedb15a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37472/0 (socket says 172.21.15.32:37472) 2024-04-16T05:39:51.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 -- 172.21.15.32:0/3456125088 learned_addr learned my addr 172.21.15.32:0/3456125088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd54182e640 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb12780 0x55bfedb15520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd54202f640 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb11e60 0x55bfedb14fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 -- 172.21.15.32:0/3456125088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb12780 msgr2=0x55bfedb15520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb12780 0x55bfedb15520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 -- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb11e60 msgr2=0x55bfedb14fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb11e60 0x55bfedb14fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 -- 172.21.15.32:0/3456125088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfedb94ce0 con 0x55bfedb1b360 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd54202f640 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb11e60 0x55bfedb14fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:51.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd542830640 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb1b360 0x55bfedb15a60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd534009e80 tx=0x7fd534005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52bed3f391d8e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd525ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.779+0000 7fd525ffb640 1 -- 172.21.15.32:0/3456125088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53401e3f0 con 0x55bfedb1b360 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd525ffb640 1 -- 172.21.15.32:0/3456125088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fd53401e590 con 0x55bfedb1b360 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 1 -- 172.21.15.32:0/3456125088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd52c043070 con 0x55bfedb1b360 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd525ffb640 1 -- 172.21.15.32:0/3456125088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd534038050 con 0x55bfedb1b360 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 10 client.16767 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 1 -- 172.21.15.32:0/3456125088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfedb95e10 con 0x55bfedb1b360 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd525ffb640 1 -- 172.21.15.32:0/3456125088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c043070 con 0x55bfedb1b360 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd525ffb640 1 -- 172.21.15.32:0/3456125088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd534031060 con 0x55bfedb1b360 2024-04-16T05:39:51.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 10 client.16767 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:51.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 10 client.16767 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:39:51.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 20 client.16767 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:51.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55bfedb96410 0x55bfedb98830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 4 client.16767 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:51.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd546b22740 1 -- 172.21.15.32:0/3456125088 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfeda7e970 con 0x55bfedb96410 2024-04-16T05:39:51.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.780+0000 7fd54202f640 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55bfedb96410 0x55bfedb98830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd525ffb640 10 client.16767 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:39:51.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd54202f640 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55bfedb96410 0x55bfedb98830 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9c5d20fedafb45f server_cookie=b91534de5a31eedb in_seq=0 out_seq=0 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd525ffb640 1 -- 172.21.15.32:0/3456125088 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfeda7e970 con 0x55bfedb96410 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd525ffb640 10 client.16767 handle_command_reply: tid=0 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 client.16767 shutdown 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 -- 172.21.15.32:0/3456125088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb1b360 msgr2=0x55bfedb15a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb1b360 0x55bfedb15a60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd534009e80 tx=0x7fd534005f10 comp rx=0 tx=0).stop 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 -- 172.21.15.32:0/3456125088 shutdown_connections 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55bfedb96410 0x55bfedb98830 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfedb1b360 0x55bfedb15a60 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd525ffb640 0 client.16767 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfedb12780 0x55bfedb15520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 --2- 172.21.15.32:0/3456125088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfedb11e60 0x55bfedb14fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 -- 172.21.15.32:0/3456125088 >> 172.21.15.32:0/3456125088 conn(0x55bfedb0c590 msgr2=0x55bfedb18420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 -- 172.21.15.32:0/3456125088 shutdown_connections 2024-04-16T05:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 1 -- 172.21.15.32:0/3456125088 wait complete. 2024-04-16T05:39:51.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.781+0000 7fd546b22740 20 client.16767 trim_cache size 0 max 16384 2024-04-16T05:39:51.790 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 -- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5280a3d20 msgr2=0x7fd5280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5280a3d20 0x7fd5280a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd534021000 tx=0x7fd534020000 comp rx=0 tx=0).stop 2024-04-16T05:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 -- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102250 msgr2=0x7fd53c19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102250 0x7fd53c19e5d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd53800d950 tx=0x7fd53800de10 comp rx=0 tx=0).stop 2024-04-16T05:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 -- 172.21.15.32:0/194816200 shutdown_connections 2024-04-16T05:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd53c198690 0x7fd53c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd53c107320 0x7fd53c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5280a3d20 0x7fd5280a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 --2- 172.21.15.32:0/194816200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c102250 0x7fd53c19e5d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 -- 172.21.15.32:0/194816200 >> 172.21.15.32:0/194816200 conn(0x7fd53c079490 msgr2=0x7fd53c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 -- 172.21.15.32:0/194816200 shutdown_connections 2024-04-16T05:39:51.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.782+0000 7fd544293640 1 -- 172.21.15.32:0/194816200 wait complete. 2024-04-16T05:39:51.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:39:51.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:51 smithi134 ceph-mon[36153]: pgmap v150: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:51.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:51 smithi082 ceph-mon[36195]: pgmap v150: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 -- 172.21.15.32:0/3004267250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103b90 msgr2=0x7fae5010c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 --2- 172.21.15.32:0/3004267250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103b90 0x7fae5010c070 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fae44009900 tx=0x7fae4402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 -- 172.21.15.32:0/3004267250 shutdown_connections 2024-04-16T05:39:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 --2- 172.21.15.32:0/3004267250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae5010c640 0x7fae5010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 --2- 172.21.15.32:0/3004267250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103b90 0x7fae5010c070 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 --2- 172.21.15.32:0/3004267250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae50103270 0x7fae50103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 -- 172.21.15.32:0/3004267250 >> 172.21.15.32:0/3004267250 conn(0x7fae50077280 msgr2=0x7fae50077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 -- 172.21.15.32:0/3004267250 shutdown_connections 2024-04-16T05:39:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.941+0000 7fae563e9640 1 -- 172.21.15.32:0/3004267250 wait complete. 2024-04-16T05:39:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae563e9640 1 Processor -- start 2024-04-16T05:39:51.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae563e9640 1 -- start start 2024-04-16T05:39:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae563e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103270 0x7fae5019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae563e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae50103b90 0x7fae5019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae563e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae5010c640 0x7fae501986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae563e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fae501123d0 con 0x7fae50103270 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae563e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fae50112250 con 0x7fae5010c640 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae563e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae50112550 con 0x7fae50103b90 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae4ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103270 0x7fae5019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae4ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103270 0x7fae5019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37494/0 (socket says 172.21.15.32:37494) 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae4ffff640 1 -- 172.21.15.32:0/2660401928 learned_addr learned my addr 172.21.15.32:0/2660401928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.942+0000 7fae54986640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae5010c640 0x7fae501986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae50103b90 0x7fae5019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4ffff640 1 -- 172.21.15.32:0/2660401928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae50103b90 msgr2=0x7fae5019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4ffff640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae50103b90 0x7fae5019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4ffff640 1 -- 172.21.15.32:0/2660401928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae5010c640 msgr2=0x7fae501986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4ffff640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae5010c640 0x7fae501986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4ffff640 1 -- 172.21.15.32:0/2660401928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae50198f90 con 0x7fae50103270 2024-04-16T05:39:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae54986640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae5010c640 0x7fae501986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae50103b90 0x7fae5019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4ffff640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103270 0x7fae5019e610 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fae4000c890 tx=0x7fae4000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4d7fa640 1 -- 172.21.15.32:0/2660401928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40004220 con 0x7fae50103270 2024-04-16T05:39:51.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae563e9640 1 -- 172.21.15.32:0/2660401928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae50199280 con 0x7fae50103270 2024-04-16T05:39:51.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4d7fa640 1 -- 172.21.15.32:0/2660401928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae400043c0 con 0x7fae50103270 2024-04-16T05:39:51.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae4d7fa640 1 -- 172.21.15.32:0/2660401928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40004a00 con 0x7fae50103270 2024-04-16T05:39:51.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.943+0000 7fae563e9640 1 -- 172.21.15.32:0/2660401928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae501a5590 con 0x7fae50103270 2024-04-16T05:39:51.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.945+0000 7fae4d7fa640 1 -- 172.21.15.32:0/2660401928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae400079b0 con 0x7fae50103270 2024-04-16T05:39:51.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae58fd1740 1 Processor -- start 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae58fd1740 1 -- start start 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266780240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 0x561266788e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 0x56126678b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae58fd1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5612666e50e0 con 0x56126677fe60 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae58fd1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561266682520 con 0x561266789360 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae58fd1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612666e10a0 con 0x561266780780 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae4d7fa640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fae380a3d20 0x7fae380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266780240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266780240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37510/0 (socket says 172.21.15.32:37510) 2024-04-16T05:39:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 -- 172.21.15.32:0/3510278730 learned_addr learned my addr 172.21.15.32:0/3510278730 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae4ffff640 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 0x561266788e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fae380a3d20 0x7fae380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 0x56126678b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae4d7fa640 1 -- 172.21.15.32:0/2660401928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fae400d0900 con 0x7fae50103270 2024-04-16T05:39:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 -- 172.21.15.32:0/3510278730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 msgr2=0x561266788e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 0x561266788e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 -- 172.21.15.32:0/3510278730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 msgr2=0x56126678b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 0x56126678b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.946+0000 7fae54986640 1 -- 172.21.15.32:0/3510278730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612666aa8d0 con 0x56126677fe60 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.947+0000 7fae54986640 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266780240 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fae48013de0 tx=0x7fae480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c51420e44882c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.947+0000 7fae54986640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fae380a3d20 0x7fae380a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fae48010ca0 tx=0x7fae4801b000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.947+0000 7fae367fc640 1 -- 172.21.15.32:0/3510278730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4800be10 con 0x56126677fe60 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.947+0000 7fae367fc640 1 -- 172.21.15.32:0/3510278730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4800ab60 con 0x56126677fe60 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.947+0000 7fae367fc640 1 -- 172.21.15.32:0/3510278730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48004480 con 0x56126677fe60 2024-04-16T05:39:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 -- 172.21.15.32:0/3510278730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 msgr2=0x561266780240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266780240 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fae48013de0 tx=0x7fae480107a0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 -- 172.21.15.32:0/3510278730 shutdown_connections 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 0x56126678b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 0x561266788e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3510278730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266780240 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 -- 172.21.15.32:0/3510278730 >> 172.21.15.32:0/3510278730 conn(0x56126677a590 msgr2=0x56126677a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 -- 172.21.15.32:0/3510278730 shutdown_connections 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 -- 172.21.15.32:0/3510278730 wait complete. 2024-04-16T05:39:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 -- start start 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 Processor -- start 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.948+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56126677fe60 0x5612667864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 0x561266786a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266789360 0x561266786f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae58fd1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56126678bfa0 con 0x561266789360 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae58fd1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56126669e4d0 con 0x56126677fe60 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae58fd1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56126678c120 con 0x561266780780 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae58fd1740 10 client.0 _ll_register_callbacks cb 0x5612666832c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266789360 0x561266786f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266789360 0x561266786f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37514/0 (socket says 172.21.15.32:37514) 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 -- 172.21.15.32:0/3717679612 learned_addr learned my addr 172.21.15.32:0/3717679612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae4ffff640 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56126677fe60 0x5612667864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 -- 172.21.15.32:0/3717679612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 msgr2=0x561266786a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 0x561266786a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 0x561266786a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 -- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56126677fe60 msgr2=0x5612667864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56126677fe60 0x5612667864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 -- 172.21.15.32:0/3717679612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612663f73f0 con 0x561266789360 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae4ffff640 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56126677fe60 0x5612667864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae54986640 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266789360 0x561266786f50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae4800a2c0 tx=0x7fae48010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d7cccba62e559a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae2f7fe640 1 -- 172.21.15.32:0/3717679612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48004e10 con 0x561266789360 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae2f7fe640 1 -- 172.21.15.32:0/3717679612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fae480040c0 con 0x561266789360 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.949+0000 7fae58fd1740 1 -- 172.21.15.32:0/3717679612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4800ab60 con 0x561266789360 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae2f7fe640 1 -- 172.21.15.32:0/3717679612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae48013940 con 0x561266789360 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae58fd1740 10 client.16779 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae58fd1740 1 -- 172.21.15.32:0/3717679612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5612664c17d0 con 0x561266789360 2024-04-16T05:39:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae2f7fe640 1 -- 172.21.15.32:0/3717679612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4800ab60 con 0x561266789360 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae2f7fe640 1 -- 172.21.15.32:0/3717679612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fae4802c070 con 0x561266789360 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae58fd1740 10 client.16779 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae58fd1740 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae58fd1740 20 client.16779 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612667992d0 0x56126679b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae58fd1740 4 client.16779 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae58fd1740 1 -- 172.21.15.32:0/3717679612 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612666f2b00 con 0x5612667992d0 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.950+0000 7fae4ffff640 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612667992d0 0x56126679b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.951+0000 7fae4ffff640 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612667992d0 0x56126679b6c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67de1f3c30326e21 server_cookie=bab462f346ca0197 in_seq=0 out_seq=0 2024-04-16T05:39:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.951+0000 7fae2f7fe640 10 client.16779 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.951+0000 7fae2f7fe640 1 -- 172.21.15.32:0/3717679612 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612666f2b00 con 0x5612667992d0 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.951+0000 7fae2f7fe640 10 client.16779 handle_command_reply: tid=0 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 client.16779 shutdown 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 -- 172.21.15.32:0/3717679612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266789360 msgr2=0x561266786f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266789360 0x561266786f50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae4800a2c0 tx=0x7fae48010fd0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 -- 172.21.15.32:0/3717679612 shutdown_connections 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266789360 0x561266786f50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612667992d0 0x56126679b6c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266780780 0x561266786a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 --2- 172.21.15.32:0/3717679612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56126677fe60 0x5612667864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae2f7fe640 0 client.16779 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:51.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 -- 172.21.15.32:0/3717679612 >> 172.21.15.32:0/3717679612 conn(0x56126677a590 msgr2=0x56126677c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 -- 172.21.15.32:0/3717679612 shutdown_connections 2024-04-16T05:39:51.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 1 -- 172.21.15.32:0/3717679612 wait complete. 2024-04-16T05:39:51.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.952+0000 7fae58fd1740 20 client.16779 trim_cache size 0 max 16384 2024-04-16T05:39:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae58fd1740 1 Processor -- start 2024-04-16T05:39:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae58fd1740 1 -- start start 2024-04-16T05:39:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56126677fe60 0x561266788c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266780780 0x561266794f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 0x561266785740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae58fd1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fae501b1cb0 con 0x561266780780 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae58fd1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fae501af580 con 0x561266789360 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae58fd1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae501ad3d0 con 0x56126677fe60 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266780780 0x561266794f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266780780 0x561266794f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37516/0 (socket says 172.21.15.32:37516) 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.965+0000 7fae4f7fe640 1 -- 172.21.15.32:0/1619049910 learned_addr learned my addr 172.21.15.32:0/1619049910 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae4ffff640 1 --2- 172.21.15.32:0/1619049910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56126677fe60 0x561266788c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae4f7fe640 1 -- 172.21.15.32:0/1619049910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56126677fe60 msgr2=0x561266788c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/1619049910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56126677fe60 0x561266788c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae4f7fe640 1 -- 172.21.15.32:0/1619049910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 msgr2=0x561266785740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/1619049910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 0x561266785740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae4f7fe640 1 -- 172.21.15.32:0/1619049910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56126678bac0 con 0x561266780780 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/1619049910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266780780 0x561266794f40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fae44035000 tx=0x7fae44005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c8c677ff9c93e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae35ffb640 1 -- 172.21.15.32:0/1619049910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4402cbe0 con 0x561266780780 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae35ffb640 1 -- 172.21.15.32:0/1619049910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4402cd80 con 0x561266780780 2024-04-16T05:39:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.966+0000 7fae35ffb640 1 -- 172.21.15.32:0/1619049910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae440049b0 con 0x561266780780 2024-04-16T05:39:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 -- 172.21.15.32:0/1619049910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266780780 msgr2=0x561266794f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 --2- 172.21.15.32:0/1619049910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266780780 0x561266794f40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fae44035000 tx=0x7fae44005a20 comp rx=0 tx=0).stop 2024-04-16T05:39:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 -- 172.21.15.32:0/1619049910 shutdown_connections 2024-04-16T05:39:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 --2- 172.21.15.32:0/1619049910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266789360 0x561266785740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 --2- 172.21.15.32:0/1619049910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561266780780 0x561266794f40 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 --2- 172.21.15.32:0/1619049910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56126677fe60 0x561266788c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 -- 172.21.15.32:0/1619049910 >> 172.21.15.32:0/1619049910 conn(0x56126677a590 msgr2=0x561266779730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 -- 172.21.15.32:0/1619049910 shutdown_connections 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 -- 172.21.15.32:0/1619049910 wait complete. 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 -- start start 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 Processor -- start 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266791690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266780780 0x56126678ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266789360 0x56126678cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fae501ada40 con 0x56126677fe60 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.967+0000 7fae58fd1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fae501aeeb0 con 0x561266780780 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae58fd1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae501ae110 con 0x561266789360 2024-04-16T05:39:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae58fd1740 10 client.0 _ll_register_callbacks cb 0x7fae44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266791690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266791690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37528/0 (socket says 172.21.15.32:37528) 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 -- 172.21.15.32:0/2876202871 learned_addr learned my addr 172.21.15.32:0/2876202871 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266780780 0x56126678ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 -- 172.21.15.32:0/2876202871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266789360 msgr2=0x56126678cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266789360 0x56126678cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 -- 172.21.15.32:0/2876202871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266780780 msgr2=0x56126678ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266780780 0x56126678ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 -- 172.21.15.32:0/2876202871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56126678d600 con 0x56126677fe60 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae4ffff640 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266791690 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fae40005fd0 tx=0x7fae40008ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7458c8989051c52b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae2f7fe640 1 -- 172.21.15.32:0/2876202871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4009cc00 con 0x56126677fe60 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae2f7fe640 1 -- 172.21.15.32:0/2876202871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fae40007ca0 con 0x56126677fe60 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae58fd1740 1 -- 172.21.15.32:0/2876202871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612666ec970 con 0x56126677fe60 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae2f7fe640 1 -- 172.21.15.32:0/2876202871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae4009c500 con 0x56126677fe60 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae58fd1740 10 client.16785 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae58fd1740 1 -- 172.21.15.32:0/2876202871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5612667ff630 con 0x56126677fe60 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.968+0000 7fae2f7fe640 1 -- 172.21.15.32:0/2876202871 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612666ec970 con 0x56126677fe60 2024-04-16T05:39:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae2f7fe640 1 -- 172.21.15.32:0/2876202871 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fae400cdf30 con 0x56126677fe60 2024-04-16T05:39:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae58fd1740 10 client.16785 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae58fd1740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae58fd1740 20 client.16785 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae58fd1740 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612667ffd00 0x561266802140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae58fd1740 4 client.16785 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:39:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae58fd1740 1 -- 172.21.15.32:0/2876202871 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612666686b0 con 0x5612667ffd00 2024-04-16T05:39:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612667ffd00 0x561266802140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae4f7fe640 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612667ffd00 0x561266802140 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5038e8d6cf9aac09 server_cookie=d165e2042a1ec86f in_seq=0 out_seq=0 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae2f7fe640 10 client.16785 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae2f7fe640 1 -- 172.21.15.32:0/2876202871 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612666686b0 con 0x5612667ffd00 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae2f7fe640 10 client.16785 handle_command_reply: tid=0 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.969+0000 7fae58fd1740 1 client.16785 shutdown 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 -- 172.21.15.32:0/2876202871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 msgr2=0x561266791690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266791690 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fae40005fd0 tx=0x7fae40008ed0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 -- 172.21.15.32:0/2876202871 shutdown_connections 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5612667ffd00 0x561266802140 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561266789360 0x56126678cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561266780780 0x56126678ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae2f7fe640 0 client.16785 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:51.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 --2- 172.21.15.32:0/2876202871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56126677fe60 0x561266791690 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 -- 172.21.15.32:0/2876202871 >> 172.21.15.32:0/2876202871 conn(0x56126677a590 msgr2=0x561266778ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 -- 172.21.15.32:0/2876202871 shutdown_connections 2024-04-16T05:39:51.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 1 -- 172.21.15.32:0/2876202871 wait complete. 2024-04-16T05:39:51.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae58fd1740 20 client.16785 trim_cache size 0 max 16384 2024-04-16T05:39:51.978 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:39:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae563e9640 1 -- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fae380a3d20 msgr2=0x7fae380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae563e9640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fae380a3d20 0x7fae380a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fae48010ca0 tx=0x7fae4801b000 comp rx=0 tx=0).stop 2024-04-16T05:39:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae563e9640 1 -- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103270 msgr2=0x7fae5019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae563e9640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103270 0x7fae5019e610 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fae4000c890 tx=0x7fae4000cd50 comp rx=0 tx=0).stop 2024-04-16T05:39:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.970+0000 7fae563e9640 1 -- 172.21.15.32:0/2660401928 shutdown_connections 2024-04-16T05:39:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.971+0000 7fae563e9640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae5010c640 0x7fae501986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.971+0000 7fae563e9640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae50103b90 0x7fae5019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.971+0000 7fae563e9640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fae380a3d20 0x7fae380a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.971+0000 7fae563e9640 1 --2- 172.21.15.32:0/2660401928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae50103270 0x7fae5019e610 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.971+0000 7fae563e9640 1 -- 172.21.15.32:0/2660401928 >> 172.21.15.32:0/2660401928 conn(0x7fae50077280 msgr2=0x7fae500fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.971+0000 7fae563e9640 1 -- 172.21.15.32:0/2660401928 shutdown_connections 2024-04-16T05:39:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:51.971+0000 7fae563e9640 1 -- 172.21.15.32:0/2660401928 wait complete. 2024-04-16T05:39:52.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:52 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/747425252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:52.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:52 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/747425252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:52.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:52 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/747425252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:39:52.982 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:39:52.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:39:52.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:39:53.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.106+0000 7f4e05c14640 1 -- 172.21.15.32:0/3646739619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e00103290 msgr2=0x7f4e00103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.106+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3646739619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e00103290 0x7f4e00103670 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009900 tx=0x7f4df002bd60 comp rx=0 tx=0).stop 2024-04-16T05:39:53.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 -- 172.21.15.32:0/3646739619 shutdown_connections 2024-04-16T05:39:53.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3646739619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010c5d0 0x7f4e0010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3646739619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00103bb0 0x7f4e0010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3646739619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e00103290 0x7f4e00103670 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 -- 172.21.15.32:0/3646739619 >> 172.21.15.32:0/3646739619 conn(0x7f4e00077280 msgr2=0x7f4e00077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 -- 172.21.15.32:0/3646739619 shutdown_connections 2024-04-16T05:39:53.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 -- 172.21.15.32:0/3646739619 wait complete. 2024-04-16T05:39:53.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 Processor -- start 2024-04-16T05:39:53.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.107+0000 7f4e05c14640 1 -- start start 2024-04-16T05:39:53.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4e05c14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00103290 0x7f4e001a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4e05c14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00103bb0 0x7f4e001a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4e05c14640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0010c5d0 0x7f4e0019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4e05c14640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e00112400 con 0x7f4e0010c5d0 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4e05c14640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e00112280 con 0x7f4e00103bb0 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4e05c14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e00112580 con 0x7f4e00103290 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0010c5d0 0x7f4e0019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0010c5d0 0x7f4e0019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37554/0 (socket says 172.21.15.32:37554) 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 -- 172.21.15.32:0/3240706911 learned_addr learned my addr 172.21.15.32:0/3240706911 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00103bb0 0x7f4e001a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00103290 0x7f4e001a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 -- 172.21.15.32:0/3240706911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00103290 msgr2=0x7f4e001a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00103290 0x7f4e001a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 -- 172.21.15.32:0/3240706911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00103bb0 msgr2=0x7f4e001a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00103bb0 0x7f4e001a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 -- 172.21.15.32:0/3240706911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0019b040 con 0x7f4e0010c5d0 2024-04-16T05:39:53.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0010c5d0 0x7f4e0019a8c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4df800eb10 tx=0x7f4df800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfcff9640 1 -- 172.21.15.32:0/3240706911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8004b20 con 0x7f4e0010c5d0 2024-04-16T05:39:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4dfcff9640 1 -- 172.21.15.32:0/3240706911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df8004cc0 con 0x7f4e0010c5d0 2024-04-16T05:39:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.108+0000 7f4e05c14640 1 -- 172.21.15.32:0/3240706911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0019b330 con 0x7f4e0010c5d0 2024-04-16T05:39:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.109+0000 7f4dfcff9640 1 -- 172.21.15.32:0/3240706911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8010760 con 0x7f4e0010c5d0 2024-04-16T05:39:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.109+0000 7f4e05c14640 1 -- 172.21.15.32:0/3240706911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e001a75d0 con 0x7f4e0010c5d0 2024-04-16T05:39:53.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4dfcff9640 1 -- 172.21.15.32:0/3240706911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4df8005020 con 0x7f4e0010c5d0 2024-04-16T05:39:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4e08800740 1 Processor -- start 2024-04-16T05:39:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4e08800740 1 -- start start 2024-04-16T05:39:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4e08800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c82183e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4e08800740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c8218ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4e08800740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c8218f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4dfcff9640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4dec0a3d20 0x7f4dec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4e08800740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555c820e8e90 con 0x555c8218cfb0 2024-04-16T05:39:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4e08800740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555c820864c0 con 0x555c821843d0 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.111+0000 7f4e08800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555c820e4e50 con 0x555c82183ab0 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c8218f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4dec0a3d20 0x7f4dec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c8218f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37556/0 (socket says 172.21.15.32:37556) 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 -- 172.21.15.32:0/580675913 learned_addr learned my addr 172.21.15.32:0/580675913 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c8218ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfcff9640 1 -- 172.21.15.32:0/3240706911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4df80d12e0 con 0x7f4e0010c5d0 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c82183e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 -- 172.21.15.32:0/580675913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 msgr2=0x555c82183e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c82183e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 -- 172.21.15.32:0/580675913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 msgr2=0x555c8218ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c8218ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 -- 172.21.15.32:0/580675913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c81ec37e0 con 0x555c8218cfb0 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfffff640 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c8218f3a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4df80a5680 tx=0x7f4df800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163a41863c5fa03 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4dec0a3d20 0x7f4dec0a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4e00102e70 tx=0x7f4de8009a50 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4df5ffb640 1 -- 172.21.15.32:0/580675913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df809ca80 con 0x555c8218cfb0 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4df5ffb640 1 -- 172.21.15.32:0/580675913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df8016dd0 con 0x555c8218cfb0 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.112+0000 7f4df5ffb640 1 -- 172.21.15.32:0/580675913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df809c8a0 con 0x555c8218cfb0 2024-04-16T05:39:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 -- 172.21.15.32:0/580675913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 msgr2=0x555c8218f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c8218f3a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4df80a5680 tx=0x7f4df800efd0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 -- 172.21.15.32:0/580675913 shutdown_connections 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c8218f3a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c8218ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 --2- 172.21.15.32:0/580675913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c82183e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 -- 172.21.15.32:0/580675913 >> 172.21.15.32:0/580675913 conn(0x555c8217e1e0 msgr2=0x555c8217e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 -- 172.21.15.32:0/580675913 shutdown_connections 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.113+0000 7f4e08800740 1 -- 172.21.15.32:0/580675913 wait complete. 2024-04-16T05:39:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 1 -- start start 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 1 Processor -- start 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c82183ab0 0x555c8218a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c821843d0 0x555c8218a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c8218cfb0 0x555c8218aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555c8218fd40 con 0x555c821843d0 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555c82071410 con 0x555c82183ab0 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555c8218fec0 con 0x555c8218cfb0 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4e08800740 10 client.0 _ll_register_callbacks cb 0x555c8206cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4dfeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c821843d0 0x555c8218a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4dfeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c821843d0 0x555c8218a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37572/0 (socket says 172.21.15.32:37572) 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2193872601 learned_addr learned my addr 172.21.15.32:0/2193872601 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c82183ab0 0x555c8218a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.114+0000 7f4dfffff640 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c8218cfb0 0x555c8218aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2193872601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c8218cfb0 msgr2=0x555c8218aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c8218cfb0 0x555c8218aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2193872601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c82183ab0 msgr2=0x555c8218a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c82183ab0 0x555c8218a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2193872601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c81f23f70 con 0x555c821843d0 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c821843d0 0x555c8218a550 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009870 tx=0x7f4de8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f1a9d4d9c47139 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4ddeffd640 1 -- 172.21.15.32:0/2193872601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de80036b0 con 0x555c821843d0 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4e08800740 1 -- 172.21.15.32:0/2193872601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c820f68b0 con 0x555c821843d0 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4ddeffd640 1 -- 172.21.15.32:0/2193872601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4de8003850 con 0x555c821843d0 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4ddeffd640 1 -- 172.21.15.32:0/2193872601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4de8003a50 con 0x555c821843d0 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4e08800740 10 client.16797 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4e08800740 1 -- 172.21.15.32:0/2193872601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c81dbb170 con 0x555c821843d0 2024-04-16T05:39:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.115+0000 7f4ddeffd640 1 -- 172.21.15.32:0/2193872601 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555c820f68b0 con 0x555c821843d0 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.116+0000 7f4ddeffd640 1 -- 172.21.15.32:0/2193872601 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4de801f050 con 0x555c821843d0 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.116+0000 7f4e08800740 10 client.16797 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.116+0000 7f4e08800740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.116+0000 7f4e08800740 20 client.16797 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.116+0000 7f4e08800740 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555c8219d300 0x555c8219f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.116+0000 7f4e08800740 4 client.16797 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.116+0000 7f4e08800740 1 -- 172.21.15.32:0/2193872601 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c820f0720 con 0x555c8219d300 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.116+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555c8219d300 0x555c8219f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.117+0000 7f4ddeffd640 10 client.16797 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:53.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.117+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555c8219d300 0x555c8219f770 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26601b92460de3f server_cookie=45e289939fe38d6b in_seq=0 out_seq=0 2024-04-16T05:39:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.117+0000 7f4ddeffd640 1 -- 172.21.15.32:0/2193872601 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c820f0720 con 0x555c8219d300 2024-04-16T05:39:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.117+0000 7f4ddeffd640 10 client.16797 handle_command_reply: tid=0 2024-04-16T05:39:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.117+0000 7f4e08800740 1 client.16797 shutdown 2024-04-16T05:39:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.117+0000 7f4e08800740 1 -- 172.21.15.32:0/2193872601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c821843d0 msgr2=0x555c8218a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.117+0000 7f4e08800740 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c821843d0 0x555c8218a550 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009870 tx=0x7f4de8005f90 comp rx=0 tx=0).stop 2024-04-16T05:39:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 1 -- 172.21.15.32:0/2193872601 shutdown_connections 2024-04-16T05:39:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555c8219d300 0x555c8219f770 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c8218cfb0 0x555c8218aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c821843d0 0x555c8218a550 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4ddeffd640 0 client.16797 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 1 --2- 172.21.15.32:0/2193872601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c82183ab0 0x555c8218a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 1 -- 172.21.15.32:0/2193872601 >> 172.21.15.32:0/2193872601 conn(0x555c8217e1e0 msgr2=0x555c8217ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 1 -- 172.21.15.32:0/2193872601 shutdown_connections 2024-04-16T05:39:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 1 -- 172.21.15.32:0/2193872601 wait complete. 2024-04-16T05:39:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.118+0000 7f4e08800740 20 client.16797 trim_cache size 0 max 16384 2024-04-16T05:39:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4e08800740 1 Processor -- start 2024-04-16T05:39:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4e08800740 1 -- start start 2024-04-16T05:39:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4e08800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c8218c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4e08800740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82198c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4e08800740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c821a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4e08800740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e001b3f10 con 0x555c8218cfb0 2024-04-16T05:39:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4e08800740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e001b17e0 con 0x555c821843d0 2024-04-16T05:39:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4e08800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e001af630 con 0x555c82183ab0 2024-04-16T05:39:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c821a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c821a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37578/0 (socket says 172.21.15.32:37578) 2024-04-16T05:39:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4dfffff640 1 -- 172.21.15.32:0/2195265012 learned_addr learned my addr 172.21.15.32:0/2195265012 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c8218c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4dfffff640 1 -- 172.21.15.32:0/2195265012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 msgr2=0x555c8218c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.136+0000 7f4dfffff640 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c8218c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4dfffff640 1 -- 172.21.15.32:0/2195265012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 msgr2=0x555c82198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4dfffff640 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4dfffff640 1 -- 172.21.15.32:0/2195265012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c82188fe0 con 0x555c8218cfb0 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4dfffff640 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c821a5080 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4df80cecf0 tx=0x7f4df80992d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b675ef0f976ed154 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4df57fa640 1 -- 172.21.15.32:0/2195265012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df809ca80 con 0x555c8218cfb0 2024-04-16T05:39:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4df57fa640 1 -- 172.21.15.32:0/2195265012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df80d1120 con 0x555c8218cfb0 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4df57fa640 1 -- 172.21.15.32:0/2195265012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df80995b0 con 0x555c8218cfb0 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4e08800740 1 -- 172.21.15.32:0/2195265012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 msgr2=0x555c821a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.137+0000 7f4e08800740 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c821a5080 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4df80cecf0 tx=0x7f4df80992d0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 -- 172.21.15.32:0/2195265012 shutdown_connections 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c821a5080 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 --2- 172.21.15.32:0/2195265012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c8218c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 -- 172.21.15.32:0/2195265012 >> 172.21.15.32:0/2195265012 conn(0x555c8217e1e0 msgr2=0x555c8218a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 -- 172.21.15.32:0/2195265012 shutdown_connections 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 -- 172.21.15.32:0/2195265012 wait complete. 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 -- start start 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 Processor -- start 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c821920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82192620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c82192b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e001afca0 con 0x555c8218cfb0 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e001b1110 con 0x555c821843d0 2024-04-16T05:39:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e001b0370 con 0x555c82183ab0 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4e08800740 10 client.0 _ll_register_callbacks cb 0x7f4de8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.138+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c82192b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c82192b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37580/0 (socket says 172.21.15.32:37580) 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfffff640 1 -- 172.21.15.32:0/3790948673 learned_addr learned my addr 172.21.15.32:0/3790948673 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c821920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82192620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfffff640 1 -- 172.21.15.32:0/3790948673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 msgr2=0x555c821920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c821920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfffff640 1 -- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 msgr2=0x555c82192620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82192620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfffff640 1 -- 172.21.15.32:0/3790948673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c82203900 con 0x555c8218cfb0 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82192620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4dfffff640 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c82192b60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4df8005910 tx=0x7f4df809ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e21f3ec95a32a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4ddeffd640 1 -- 172.21.15.32:0/3790948673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df809d3a0 con 0x555c8218cfb0 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4ddeffd640 1 -- 172.21.15.32:0/3790948673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4df80a5720 con 0x555c8218cfb0 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4e08800740 1 -- 172.21.15.32:0/3790948673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c8206cbe0 con 0x555c8218cfb0 2024-04-16T05:39:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4ddeffd640 1 -- 172.21.15.32:0/3790948673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4df80a5bb0 con 0x555c8218cfb0 2024-04-16T05:39:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4e08800740 10 client.16803 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4e08800740 1 -- 172.21.15.32:0/3790948673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c82204af0 con 0x555c8218cfb0 2024-04-16T05:39:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.139+0000 7f4ddeffd640 1 -- 172.21.15.32:0/3790948673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555c8206cbe0 con 0x555c8218cfb0 2024-04-16T05:39:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4ddeffd640 1 -- 172.21.15.32:0/3790948673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4df8005310 con 0x555c8218cfb0 2024-04-16T05:39:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4e08800740 10 client.16803 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4e08800740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4e08800740 20 client.16803 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4e08800740 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555c82204d70 0x555c82207160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4e08800740 4 client.16803 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:39:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4e08800740 1 -- 172.21.15.32:0/3790948673 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555c820b96b0 con 0x555c82204d70 2024-04-16T05:39:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555c82204d70 0x555c82207160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4ddeffd640 10 client.16803 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.140+0000 7f4dff7fe640 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555c82204d70 0x555c82207160 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdbc3a39ca37e64a server_cookie=1023c128f62e230f in_seq=0 out_seq=0 2024-04-16T05:39:53.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.141+0000 7f4ddeffd640 1 -- 172.21.15.32:0/3790948673 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555c820b96b0 con 0x555c82204d70 2024-04-16T05:39:53.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.141+0000 7f4ddeffd640 10 client.16803 handle_command_reply: tid=0 2024-04-16T05:39:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.141+0000 7f4e08800740 1 client.16803 shutdown 2024-04-16T05:39:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 -- 172.21.15.32:0/3790948673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 msgr2=0x555c82192b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c82192b60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4df8005910 tx=0x7f4df809ca80 comp rx=0 tx=0).stop 2024-04-16T05:39:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 -- 172.21.15.32:0/3790948673 shutdown_connections 2024-04-16T05:39:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c8218cfb0 0x555c82192b60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c821843d0 0x555c82192620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555c82204d70 0x555c82207160 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 --2- 172.21.15.32:0/3790948673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c82183ab0 0x555c821920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 -- 172.21.15.32:0/3790948673 >> 172.21.15.32:0/3790948673 conn(0x555c8217e1e0 msgr2=0x555c8217f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4ddeffd640 0 client.16803 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 -- 172.21.15.32:0/3790948673 shutdown_connections 2024-04-16T05:39:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 1 -- 172.21.15.32:0/3790948673 wait complete. 2024-04-16T05:39:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e08800740 20 client.16803 trim_cache size 0 max 16384 2024-04-16T05:39:53.150 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:39:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:39:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "e0ec24e6-e5b9-4cdf-ab37-16d4bf8544bb", 2024-04-16T05:39:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:39:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:39:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 -- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4dec0a3d20 msgr2=0x7f4dec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4dec0a3d20 0x7f4dec0a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4e00102e70 tx=0x7f4de8009a50 comp rx=0 tx=0).stop 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 -- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0010c5d0 msgr2=0x7f4e0019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0010c5d0 0x7f4e0019a8c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4df800eb10 tx=0x7f4df800c2d0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 -- 172.21.15.32:0/3240706911 shutdown_connections 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4dec0a3d20 0x7f4dec0a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0010c5d0 0x7f4e0019a8c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e00103bb0 0x7f4e001a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 --2- 172.21.15.32:0/3240706911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e00103290 0x7f4e001a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.142+0000 7f4e05c14640 1 -- 172.21.15.32:0/3240706911 >> 172.21.15.32:0/3240706911 conn(0x7f4e00077280 msgr2=0x7f4e000fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.143+0000 7f4e05c14640 1 -- 172.21.15.32:0/3240706911 shutdown_connections 2024-04-16T05:39:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.143+0000 7f4e05c14640 1 -- 172.21.15.32:0/3240706911 wait complete. 2024-04-16T05:39:53.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:39:53.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:39:53.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 -- 172.21.15.32:0/3717380324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8900106210 msgr2=0x7f890010dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 --2- 172.21.15.32:0/3717380324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8900106210 0x7f890010dde0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009900 tx=0x7f88f002bc30 comp rx=0 tx=0).stop 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 -- 172.21.15.32:0/3717380324 shutdown_connections 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 --2- 172.21.15.32:0/3717380324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8900106210 0x7f890010dde0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 --2- 172.21.15.32:0/3717380324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8900101af0 0x7f8900105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 --2- 172.21.15.32:0/3717380324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8900101140 0x7f8900101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 -- 172.21.15.32:0/3717380324 >> 172.21.15.32:0/3717380324 conn(0x7f8900078000 msgr2=0x7f8900078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 -- 172.21.15.32:0/3717380324 shutdown_connections 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 -- 172.21.15.32:0/3717380324 wait complete. 2024-04-16T05:39:53.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 Processor -- start 2024-04-16T05:39:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.263+0000 7f8906fc8640 1 -- start start 2024-04-16T05:39:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8906fc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8900101140 0x7f890019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8906fc8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8900106210 0x7f890019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8906fc8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89001986c0 0x7f8900198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8906fc8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89001123e0 con 0x7f89001986c0 2024-04-16T05:39:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8906fc8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8900112260 con 0x7f8900106210 2024-04-16T05:39:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8906fc8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8900112560 con 0x7f8900101140 2024-04-16T05:39:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89001986c0 0x7f8900198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89001986c0 0x7f8900198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37598/0 (socket says 172.21.15.32:37598) 2024-04-16T05:39:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 -- 172.21.15.32:0/2894403932 learned_addr learned my addr 172.21.15.32:0/2894403932 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f88fffff640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8900106210 0x7f890019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8904d64640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8900101140 0x7f890019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 -- 172.21.15.32:0/2894403932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8900101140 msgr2=0x7f890019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8900101140 0x7f890019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 -- 172.21.15.32:0/2894403932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8900106210 msgr2=0x7f890019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8900106210 0x7f890019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 -- 172.21.15.32:0/2894403932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89001992c0 con 0x7f89001986c0 2024-04-16T05:39:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f8905565640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89001986c0 0x7f8900198b40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009230 tx=0x7f88f002cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.264+0000 7f88fdffb640 1 -- 172.21.15.32:0/2894403932 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0041070 con 0x7f89001986c0 2024-04-16T05:39:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.265+0000 7f88fdffb640 1 -- 172.21.15.32:0/2894403932 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f0004450 con 0x7f89001986c0 2024-04-16T05:39:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.265+0000 7f8906fc8640 1 -- 172.21.15.32:0/2894403932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8900199550 con 0x7f89001986c0 2024-04-16T05:39:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.265+0000 7f88fdffb640 1 -- 172.21.15.32:0/2894403932 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0003d90 con 0x7f89001986c0 2024-04-16T05:39:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.265+0000 7f8906fc8640 1 -- 172.21.15.32:0/2894403932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001a5690 con 0x7f89001986c0 2024-04-16T05:39:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.267+0000 7f88fdffb640 1 -- 172.21.15.32:0/2894403932 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88f0044070 con 0x7f89001986c0 2024-04-16T05:39:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.267+0000 7f8909bb1740 1 Processor -- start 2024-04-16T05:39:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.267+0000 7f8909bb1740 1 -- start start 2024-04-16T05:39:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e45e50 0x559a51e46230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e4f350 0x559a51e51740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8909bb1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559a51dab180 con 0x559a51e45e50 2024-04-16T05:39:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8909bb1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559a51d48360 con 0x559a51e4f350 2024-04-16T05:39:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8909bb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559a51da7140 con 0x559a51e46770 2024-04-16T05:39:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e45e50 0x559a51e46230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fdffb640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f88e80a3d20 0x7f88e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e45e50 0x559a51e46230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37606/0 (socket says 172.21.15.32:37606) 2024-04-16T05:39:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 -- 172.21.15.32:0/3283452014 learned_addr learned my addr 172.21.15.32:0/3283452014 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8905565640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f88e80a3d20 0x7f88e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e4f350 0x559a51e51740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8904d64640 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 -- 172.21.15.32:0/3283452014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 msgr2=0x559a51e4ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fdffb640 1 -- 172.21.15.32:0/2894403932 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f88f00f2560 con 0x7f89001986c0 2024-04-16T05:39:53.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 -- 172.21.15.32:0/3283452014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e4f350 msgr2=0x559a51e51740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e4f350 0x559a51e51740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 -- 172.21.15.32:0/3283452014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a51b877d0 con 0x559a51e45e50 2024-04-16T05:39:53.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8904d64640 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:39:53.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88fffff640 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e45e50 0x559a51e46230 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009f70 tx=0x7f88f4009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e25cce74ecc4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f8905565640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f88e80a3d20 0x7f88e80a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f88f00eff70 tx=0x7f88f00b4040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88e6ffd640 1 -- 172.21.15.32:0/3283452014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4004050 con 0x559a51e45e50 2024-04-16T05:39:53.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88e6ffd640 1 -- 172.21.15.32:0/3283452014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f40041f0 con 0x559a51e45e50 2024-04-16T05:39:53.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.268+0000 7f88e6ffd640 1 -- 172.21.15.32:0/3283452014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f400f760 con 0x559a51e45e50 2024-04-16T05:39:53.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 -- 172.21.15.32:0/3283452014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e45e50 msgr2=0x559a51e46230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e45e50 0x559a51e46230 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009f70 tx=0x7f88f4009370 comp rx=0 tx=0).stop 2024-04-16T05:39:53.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 -- 172.21.15.32:0/3283452014 shutdown_connections 2024-04-16T05:39:53.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e4f350 0x559a51e51740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3283452014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e45e50 0x559a51e46230 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 -- 172.21.15.32:0/3283452014 >> 172.21.15.32:0/3283452014 conn(0x559a51e40580 msgr2=0x559a51e40960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 -- 172.21.15.32:0/3283452014 shutdown_connections 2024-04-16T05:39:53.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 -- 172.21.15.32:0/3283452014 wait complete. 2024-04-16T05:39:53.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.269+0000 7f8909bb1740 1 -- start start 2024-04-16T05:39:53.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8909bb1740 1 Processor -- start 2024-04-16T05:39:53.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e45e50 0x559a51e4c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e4cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8909bb1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559a51e52140 con 0x559a51e4f350 2024-04-16T05:39:53.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8909bb1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559a51d642c0 con 0x559a51e45e50 2024-04-16T05:39:53.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8909bb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559a51e522c0 con 0x559a51e46770 2024-04-16T05:39:53.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8909bb1740 10 client.0 _ll_register_callbacks cb 0x559a51d4b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f88fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e4cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f88fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e4cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37614/0 (socket says 172.21.15.32:37614) 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f88fffff640 1 -- 172.21.15.32:0/3471022754 learned_addr learned my addr 172.21.15.32:0/3471022754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f88fffff640 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e45e50 0x559a51e4c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.270+0000 7f8904d64640 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88fffff640 1 -- 172.21.15.32:0/3471022754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 msgr2=0x559a51e4c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88fffff640 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88fffff640 1 -- 172.21.15.32:0/3471022754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e45e50 msgr2=0x559a51e4c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88fffff640 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e45e50 0x559a51e4c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88fffff640 1 -- 172.21.15.32:0/3471022754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a51abd3f0 con 0x559a51e4f350 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88fffff640 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e4cdf0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009460 tx=0x7f88f4003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc959f4193e3b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88dffff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88dffff640 1 -- 172.21.15.32:0/3471022754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4029040 con 0x559a51e4f350 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88dffff640 1 -- 172.21.15.32:0/3471022754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f88f4005230 con 0x559a51e4f350 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f8909bb1740 1 -- 172.21.15.32:0/3471022754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f40041f0 con 0x559a51e4f350 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88dffff640 1 -- 172.21.15.32:0/3471022754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88f4003ba0 con 0x559a51e4f350 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f8909bb1740 10 client.16815 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f8909bb1740 1 -- 172.21.15.32:0/3471022754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a51a7d0f0 con 0x559a51e4f350 2024-04-16T05:39:53.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88dffff640 1 -- 172.21.15.32:0/3471022754 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f40041f0 con 0x559a51e4f350 2024-04-16T05:39:53.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f88dffff640 1 -- 172.21.15.32:0/3471022754 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f88f400f500 con 0x559a51e4f350 2024-04-16T05:39:53.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f8909bb1740 10 client.16815 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:53.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f8909bb1740 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:53.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f8909bb1740 20 client.16815 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:53.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.271+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559a51e5f4b0 0x559a51e618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.272+0000 7f8909bb1740 4 client.16815 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:39:53.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.272+0000 7f8909bb1740 1 -- 172.21.15.32:0/3471022754 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a51db8ba0 con 0x559a51e5f4b0 2024-04-16T05:39:53.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.272+0000 7f8904d64640 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559a51e5f4b0 0x559a51e618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.272+0000 7f8904d64640 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559a51e5f4b0 0x559a51e618a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f166725f34277636 server_cookie=a4f317f11387f50d in_seq=0 out_seq=0 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.272+0000 7f88dffff640 10 client.16815 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f88dffff640 1 -- 172.21.15.32:0/3471022754 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a51db8ba0 con 0x559a51e5f4b0 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f88dffff640 10 client.16815 handle_command_reply: tid=0 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 client.16815 shutdown 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 -- 172.21.15.32:0/3471022754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 msgr2=0x559a51e4cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e4cdf0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009460 tx=0x7f88f4003dc0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 -- 172.21.15.32:0/3471022754 shutdown_connections 2024-04-16T05:39:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e4cdf0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559a51e5f4b0 0x559a51e618a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e46770 0x559a51e4c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 --2- 172.21.15.32:0/3471022754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e45e50 0x559a51e4c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f88dffff640 0 client.16815 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 -- 172.21.15.32:0/3471022754 >> 172.21.15.32:0/3471022754 conn(0x559a51e40580 msgr2=0x559a51e422d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 -- 172.21.15.32:0/3471022754 shutdown_connections 2024-04-16T05:39:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.273+0000 7f8909bb1740 1 -- 172.21.15.32:0/3471022754 wait complete. 2024-04-16T05:39:53.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.274+0000 7f8909bb1740 20 client.16815 trim_cache size 0 max 16384 2024-04-16T05:39:53.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8909bb1740 1 Processor -- start 2024-04-16T05:39:53.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8909bb1740 1 -- start start 2024-04-16T05:39:53.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 0x559a51e4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e5b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e66fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8909bb1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89001b1b60 con 0x559a51e4f350 2024-04-16T05:39:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8909bb1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f89001af430 con 0x559a51e46770 2024-04-16T05:39:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8909bb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89001ad280 con 0x559a51e45e50 2024-04-16T05:39:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f88fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e66fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f88fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e66fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37622/0 (socket says 172.21.15.32:37622) 2024-04-16T05:39:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f88fffff640 1 -- 172.21.15.32:0/103898967 learned_addr learned my addr 172.21.15.32:0/103898967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f8904d64640 1 --2- 172.21.15.32:0/103898967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e5b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f88fffff640 1 -- 172.21.15.32:0/103898967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 msgr2=0x559a51e4eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f88fffff640 1 --2- 172.21.15.32:0/103898967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 0x559a51e4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f88fffff640 1 -- 172.21.15.32:0/103898967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 msgr2=0x559a51e5b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f88fffff640 1 --2- 172.21.15.32:0/103898967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e5b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.291+0000 7f88fffff640 1 -- 172.21.15.32:0/103898967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a51e471c0 con 0x559a51e4f350 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f88fffff640 1 --2- 172.21.15.32:0/103898967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e66fe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f88f4010010 tx=0x7f88f4005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6b5de61416333c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f88e67fc640 1 -- 172.21.15.32:0/103898967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4024070 con 0x559a51e4f350 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f88e67fc640 1 -- 172.21.15.32:0/103898967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f4003e30 con 0x559a51e4f350 2024-04-16T05:39:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f88e67fc640 1 -- 172.21.15.32:0/103898967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f400faf0 con 0x559a51e4f350 2024-04-16T05:39:53.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 -- 172.21.15.32:0/103898967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 msgr2=0x559a51e66fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 --2- 172.21.15.32:0/103898967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e66fe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f88f4010010 tx=0x7f88f4005b90 comp rx=0 tx=0).stop 2024-04-16T05:39:53.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 -- 172.21.15.32:0/103898967 shutdown_connections 2024-04-16T05:39:53.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 --2- 172.21.15.32:0/103898967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e66fe0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 --2- 172.21.15.32:0/103898967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e5b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 --2- 172.21.15.32:0/103898967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 0x559a51e4eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 -- 172.21.15.32:0/103898967 >> 172.21.15.32:0/103898967 conn(0x559a51e40580 msgr2=0x559a51e4b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 -- 172.21.15.32:0/103898967 shutdown_connections 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.292+0000 7f8909bb1740 1 -- 172.21.15.32:0/103898967 wait complete. 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 1 -- start start 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 1 Processor -- start 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 0x559a51e54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e55400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e55940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89001ad8f0 con 0x559a51e4f350 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f89001aed60 con 0x559a51e46770 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89001adfc0 con 0x559a51e45e50 2024-04-16T05:39:53.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8909bb1740 10 client.0 _ll_register_callbacks cb 0x7f88f4013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e55940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e55940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37626/0 (socket says 172.21.15.32:37626) 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 -- 172.21.15.32:0/1412484853 learned_addr learned my addr 172.21.15.32:0/1412484853 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8904d64640 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e55400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 0x559a51e54ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 -- 172.21.15.32:0/1412484853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 msgr2=0x559a51e54ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 0x559a51e54ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 -- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 msgr2=0x559a51e55400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:39:53.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e55400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f88fffff640 1 -- 172.21.15.32:0/1412484853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a51ec4860 con 0x559a51e4f350 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.293+0000 7f8904d64640 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e55400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f88fffff640 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e55940 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f88f4003e00 tx=0x7f88f40055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1078bc8a8b957f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f88dffff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f88dffff640 1 -- 172.21.15.32:0/1412484853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4024040 con 0x559a51e4f350 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f88dffff640 1 -- 172.21.15.32:0/1412484853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f88f4004190 con 0x559a51e4f350 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 1 -- 172.21.15.32:0/1412484853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f4003e30 con 0x559a51e4f350 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f88dffff640 1 -- 172.21.15.32:0/1412484853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88f4044050 con 0x559a51e4f350 2024-04-16T05:39:53.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 10 client.16821 fetch_fsmap learned FSMap version 25 2024-04-16T05:39:53.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 1 -- 172.21.15.32:0/1412484853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a51ec5a80 con 0x559a51e4f350 2024-04-16T05:39:53.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f88dffff640 1 -- 172.21.15.32:0/1412484853 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f4003e30 con 0x559a51e4f350 2024-04-16T05:39:53.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f88dffff640 1 -- 172.21.15.32:0/1412484853 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f88f401f910 con 0x559a51e4f350 2024-04-16T05:39:53.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 10 client.16821 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:39:53.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:39:53.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 20 client.16821 populate_metadata read hostname 'smithi032' 2024-04-16T05:39:53.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559a51ec6010 0x559a51ec8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:39:53.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 4 client.16821 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.294+0000 7f8909bb1740 1 -- 172.21.15.32:0/1412484853 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a51db2a10 con 0x559a51ec6010 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8904d64640 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559a51ec6010 0x559a51ec8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8904d64640 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559a51ec6010 0x559a51ec8430 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e4a95ef4c656b6 server_cookie=22edeeef5a0c0536 in_seq=0 out_seq=0 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f88dffff640 10 client.16821 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f88dffff640 1 -- 172.21.15.32:0/1412484853 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559a51db2a10 con 0x559a51ec6010 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f88dffff640 10 client.16821 handle_command_reply: tid=0 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 client.16821 shutdown 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 -- 172.21.15.32:0/1412484853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 msgr2=0x559a51e55940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e55940 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f88f4003e00 tx=0x7f88f40055b0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 -- 172.21.15.32:0/1412484853 shutdown_connections 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559a51e4f350 0x559a51e55940 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a51e46770 0x559a51e55400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559a51ec6010 0x559a51ec8430 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 --2- 172.21.15.32:0/1412484853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a51e45e50 0x559a51e54ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f88dffff640 0 client.16821 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.295+0000 7f8909bb1740 1 -- 172.21.15.32:0/1412484853 >> 172.21.15.32:0/1412484853 conn(0x559a51e40580 msgr2=0x559a51e3eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8909bb1740 1 -- 172.21.15.32:0/1412484853 shutdown_connections 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8909bb1740 1 -- 172.21.15.32:0/1412484853 wait complete. 2024-04-16T05:39:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8909bb1740 20 client.16821 trim_cache size 0 max 16384 2024-04-16T05:39:53.305 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:39:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:39:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:39:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "e0ec24e6-e5b9-4cdf-ab37-16d4bf8544bb": { 2024-04-16T05:39:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:39:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "e0ec24e6-e5b9-4cdf-ab37-16d4bf8544bb", 2024-04-16T05:39:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:39:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:39:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:39:53.306 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:39:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 -- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f88e80a3d20 msgr2=0x7f88e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f88e80a3d20 0x7f88e80a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f88f00eff70 tx=0x7f88f00b4040 comp rx=0 tx=0).stop 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 -- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89001986c0 msgr2=0x7f8900198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89001986c0 0x7f8900198b40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009230 tx=0x7f88f002cba0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 -- 172.21.15.32:0/2894403932 shutdown_connections 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f88e80a3d20 0x7f88e80a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89001986c0 0x7f8900198b40 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8900106210 0x7f890019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 --2- 172.21.15.32:0/2894403932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8900101140 0x7f890019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 -- 172.21.15.32:0/2894403932 >> 172.21.15.32:0/2894403932 conn(0x7f8900078000 msgr2=0x7f89000feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 -- 172.21.15.32:0/2894403932 shutdown_connections 2024-04-16T05:39:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:39:53.296+0000 7f8906fc8640 1 -- 172.21.15.32:0/2894403932 wait complete. 2024-04-16T05:39:53.309 INFO:tasks.cephfs.filesystem:scrub status for tag:e0ec24e6-e5b9-4cdf-ab37-16d4bf8544bb - {'path': '/', 'tag': 'e0ec24e6-e5b9-4cdf-ab37-16d4bf8544bb', 'options': 'recursive,force'} 2024-04-16T05:39:53.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:53 smithi032 ceph-mon[32351]: pgmap v151: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:53.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:53 smithi134 ceph-mon[36153]: pgmap v151: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:53.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:53 smithi082 ceph-mon[36195]: pgmap v151: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:39:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:54 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:39:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:54 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:54 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:39:54.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:54 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:54.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:54 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:39:54.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:54 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:54.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:54 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:39:54.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:54 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:54 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:39:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:54 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:54 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:39:54.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:54 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:39:55.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:55 smithi032 ceph-mon[32351]: pgmap v152: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:55.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:55 smithi134 ceph-mon[36153]: pgmap v152: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:55.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:55 smithi082 ceph-mon[36195]: pgmap v152: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:56.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:56 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:39:56.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:56 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:39:56.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:56 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:39:57.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:57 smithi032 ceph-mon[32351]: pgmap v153: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:57.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:57 smithi134 ceph-mon[36153]: pgmap v153: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:57.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:57 smithi082 ceph-mon[36195]: pgmap v153: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:39:59.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:39:59 smithi032 ceph-mon[32351]: pgmap v154: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:59.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:39:59 smithi134 ceph-mon[36153]: pgmap v154: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:39:59.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:39:59 smithi082 ceph-mon[36195]: pgmap v154: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:40:00.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:00 smithi032 ceph-mon[32351]: overall HEALTH_OK 2024-04-16T05:40:00.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:00 smithi134 ceph-mon[36153]: overall HEALTH_OK 2024-04-16T05:40:00.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:00 smithi082 ceph-mon[36195]: overall HEALTH_OK 2024-04-16T05:40:01.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:01 smithi032 ceph-mon[32351]: pgmap v155: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:01.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:01 smithi134 ceph-mon[36153]: pgmap v155: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:01.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:01 smithi082 ceph-mon[36195]: pgmap v155: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:03.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:03 smithi032 ceph-mon[32351]: pgmap v156: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:03.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:03 smithi134 ceph-mon[36153]: pgmap v156: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:03.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:03 smithi082 ceph-mon[36195]: pgmap v156: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:05.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:05 smithi032 ceph-mon[32351]: pgmap v157: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:40:05.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:05 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:40:05.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:05 smithi134 ceph-mon[36153]: pgmap v157: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:40:05.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:05 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:40:05.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:05 smithi082 ceph-mon[36195]: pgmap v157: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:40:05.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:05 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:40:06.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:06 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:40:06.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:06 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:40:06.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:06 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:40:06.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:06 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:40:06.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:06 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:40:06.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:06 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:40:06.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:06 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:40:06.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:06 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:40:06.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:06 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:40:07.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:07 smithi032 ceph-mon[32351]: pgmap v158: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:40:07.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:07 smithi134 ceph-mon[36153]: pgmap v158: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:40:07.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:07 smithi082 ceph-mon[36195]: pgmap v158: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:40:09.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:09 smithi032 ceph-mon[32351]: pgmap v159: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:09.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:09 smithi134 ceph-mon[36153]: pgmap v159: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:09.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:09 smithi082 ceph-mon[36195]: pgmap v159: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:40:11.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:11 smithi032 ceph-mon[32351]: pgmap v160: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:40:11.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:11 smithi134 ceph-mon[36153]: pgmap v160: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:40:11.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:11 smithi082 ceph-mon[36195]: pgmap v160: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-16T05:40:13.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:13 smithi032 ceph-mon[32351]: pgmap v161: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:40:13.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:13 smithi134 ceph-mon[36153]: pgmap v161: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:40:13.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:13 smithi082 ceph-mon[36195]: pgmap v161: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-16T05:40:15.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:15 smithi032 ceph-mon[32351]: pgmap v162: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:40:15.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:15 smithi134 ceph-mon[36153]: pgmap v162: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:40:15.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:15 smithi082 ceph-mon[36195]: pgmap v162: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:40:17.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:17 smithi032 ceph-mon[32351]: pgmap v163: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:40:17.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:17 smithi134 ceph-mon[36153]: pgmap v163: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:40:17.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:17 smithi082 ceph-mon[36195]: pgmap v163: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:40:19.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:19 smithi134 ceph-mon[36153]: pgmap v164: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:40:19.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:19 smithi082 ceph-mon[36195]: pgmap v164: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:40:19.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:19 smithi032 ceph-mon[32351]: pgmap v164: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:40:21.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:21 smithi134 ceph-mon[36153]: pgmap v165: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:21.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:21 smithi082 ceph-mon[36195]: pgmap v165: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:21.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:21 smithi032 ceph-mon[32351]: pgmap v165: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:23.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:23.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:40:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 -- 172.21.15.32:0/1748983825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 msgr2=0x7f850c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 --2- 172.21.15.32:0/1748983825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 0x7f850c1016b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009930 tx=0x7f84f402bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 -- 172.21.15.32:0/1748983825 shutdown_connections 2024-04-16T05:40:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 --2- 172.21.15.32:0/1748983825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f850c101bf0 0x7f850c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 --2- 172.21.15.32:0/1748983825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 0x7f850c1016b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 --2- 172.21.15.32:0/1748983825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c1070e0 0x7f850c1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 -- 172.21.15.32:0/1748983825 >> 172.21.15.32:0/1748983825 conn(0x7f850c076d00 msgr2=0x7f850c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 -- 172.21.15.32:0/1748983825 shutdown_connections 2024-04-16T05:40:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.434+0000 7f85121db640 1 -- 172.21.15.32:0/1748983825 wait complete. 2024-04-16T05:40:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f85121db640 1 Processor -- start 2024-04-16T05:40:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f85121db640 1 -- start start 2024-04-16T05:40:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f85121db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 0x7f850c10e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f85121db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f850c101bf0 0x7f850c10ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f85121db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c1070e0 0x7f850c10f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f85121db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f850c112460 con 0x7f850c101250 2024-04-16T05:40:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f85121db640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1122e0 con 0x7f850c101bf0 2024-04-16T05:40:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f85121db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1125e0 con 0x7f850c1070e0 2024-04-16T05:40:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f850b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 0x7f850c10e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.435+0000 7f850b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 0x7f850c10e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49692/0 (socket says 172.21.15.32:49692) 2024-04-16T05:40:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f850b7fe640 1 -- 172.21.15.32:0/431405753 learned_addr learned my addr 172.21.15.32:0/431405753 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f8503fff640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f850c101bf0 0x7f850c10ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f850b7fe640 1 -- 172.21.15.32:0/431405753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c1070e0 msgr2=0x7f850c10f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f850b7fe640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c1070e0 0x7f850c10f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f850b7fe640 1 -- 172.21.15.32:0/431405753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f850c101bf0 msgr2=0x7f850c10ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f850b7fe640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f850c101bf0 0x7f850c10ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f850b7fe640 1 -- 172.21.15.32:0/431405753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c1a6e40 con 0x7f850c101250 2024-04-16T05:40:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f8503fff640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f850c101bf0 0x7f850c10ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f850b7fe640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 0x7f850c10e850 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c970 tx=0x7f84fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f85097fa640 1 -- 172.21.15.32:0/431405753 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc007bb0 con 0x7f850c101250 2024-04-16T05:40:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f85121db640 1 -- 172.21.15.32:0/431405753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c1a7130 con 0x7f850c101250 2024-04-16T05:40:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f85097fa640 1 -- 172.21.15.32:0/431405753 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84fc007d50 con 0x7f850c101250 2024-04-16T05:40:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f85097fa640 1 -- 172.21.15.32:0/431405753 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00f450 con 0x7f850c101250 2024-04-16T05:40:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.436+0000 7f85121db640 1 -- 172.21.15.32:0/431405753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c1a7630 con 0x7f850c101250 2024-04-16T05:40:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.438+0000 7f85097fa640 1 -- 172.21.15.32:0/431405753 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84fc005020 con 0x7f850c101250 2024-04-16T05:40:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8514dc8740 1 Processor -- start 2024-04-16T05:40:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8514dc8740 1 -- start start 2024-04-16T05:40:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74619230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 0x557d74621e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74622350 0x557d74624740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8514dc8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557d7457e180 con 0x557d74618e50 2024-04-16T05:40:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8514dc8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557d7451b360 con 0x557d74622350 2024-04-16T05:40:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8514dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d7457a140 con 0x557d74619770 2024-04-16T05:40:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74619230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74619230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49702/0 (socket says 172.21.15.32:49702) 2024-04-16T05:40:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850bfff640 1 -- 172.21.15.32:0/421949085 learned_addr learned my addr 172.21.15.32:0/421949085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f85097fa640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84f80a3d20 0x7f84f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f8503fff640 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74622350 0x557d74624740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850b7fe640 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 0x557d74621e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850bfff640 1 -- 172.21.15.32:0/421949085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 msgr2=0x557d74621e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850bfff640 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 0x557d74621e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850bfff640 1 -- 172.21.15.32:0/421949085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74622350 msgr2=0x557d74624740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850bfff640 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74622350 0x557d74624740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.439+0000 7f850bfff640 1 -- 172.21.15.32:0/421949085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d7435a7d0 con 0x557d74618e50 2024-04-16T05:40:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.440+0000 7f85097fa640 1 -- 172.21.15.32:0/431405753 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f84fc0d09e0 con 0x7f850c101250 2024-04-16T05:40:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.440+0000 7f850bfff640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84f80a3d20 0x7f84f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.440+0000 7f850bfff640 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74619230 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8504009f80 tx=0x7f850400d770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9389c6742699bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.440+0000 7f8501ffb640 1 -- 172.21.15.32:0/421949085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504018070 con 0x557d74618e50 2024-04-16T05:40:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.440+0000 7f8501ffb640 1 -- 172.21.15.32:0/421949085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850401edf0 con 0x557d74618e50 2024-04-16T05:40:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.440+0000 7f8501ffb640 1 -- 172.21.15.32:0/421949085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504005480 con 0x557d74618e50 2024-04-16T05:40:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.440+0000 7f850bfff640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84f80a3d20 0x7f84f80a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f850401e000 tx=0x7f850401f000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 -- 172.21.15.32:0/421949085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 msgr2=0x557d74619230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74619230 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8504009f80 tx=0x7f850400d770 comp rx=0 tx=0).stop 2024-04-16T05:40:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 -- 172.21.15.32:0/421949085 shutdown_connections 2024-04-16T05:40:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74622350 0x557d74624740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 0x557d74621e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 --2- 172.21.15.32:0/421949085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74619230 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 -- 172.21.15.32:0/421949085 >> 172.21.15.32:0/421949085 conn(0x557d74613580 msgr2=0x557d74613960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 -- 172.21.15.32:0/421949085 shutdown_connections 2024-04-16T05:40:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 -- 172.21.15.32:0/421949085 wait complete. 2024-04-16T05:40:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 -- start start 2024-04-16T05:40:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 Processor -- start 2024-04-16T05:40:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 0x557d7461f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 0x557d7461f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.441+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74622350 0x557d7461fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f8514dc8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557d74625140 con 0x557d74622350 2024-04-16T05:40:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f8514dc8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557d745372c0 con 0x557d74618e50 2024-04-16T05:40:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f8514dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d746252c0 con 0x557d74619770 2024-04-16T05:40:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f8514dc8740 10 client.0 _ll_register_callbacks cb 0x557d7451e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74622350 0x557d7461fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74622350 0x557d7461fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49706/0 (socket says 172.21.15.32:49706) 2024-04-16T05:40:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 -- 172.21.15.32:0/1643407480 learned_addr learned my addr 172.21.15.32:0/1643407480 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850b7fe640 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 0x557d7461f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f8503fff640 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 0x557d7461f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 -- 172.21.15.32:0/1643407480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 msgr2=0x557d7461f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 0x557d7461f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 -- 172.21.15.32:0/1643407480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 msgr2=0x557d7461f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 0x557d7461f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 -- 172.21.15.32:0/1643407480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d742903f0 con 0x557d74622350 2024-04-16T05:40:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f850bfff640 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74622350 0x557d7461fe00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f850400de10 tx=0x7f8504004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77303d7a2369376a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f84eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f84eaffd640 1 -- 172.21.15.32:0/1643407480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850402b070 con 0x557d74622350 2024-04-16T05:40:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f84eaffd640 1 -- 172.21.15.32:0/1643407480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8504005460 con 0x557d74622350 2024-04-16T05:40:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.442+0000 7f8514dc8740 1 -- 172.21.15.32:0/1643407480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850401edf0 con 0x557d74622350 2024-04-16T05:40:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f84eaffd640 1 -- 172.21.15.32:0/1643407480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8504037050 con 0x557d74622350 2024-04-16T05:40:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f8514dc8740 10 client.16833 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f8514dc8740 1 -- 172.21.15.32:0/1643407480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d742500f0 con 0x557d74622350 2024-04-16T05:40:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f84eaffd640 1 -- 172.21.15.32:0/1643407480 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850401edf0 con 0x557d74622350 2024-04-16T05:40:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f84eaffd640 1 -- 172.21.15.32:0/1643407480 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f85040108a0 con 0x557d74622350 2024-04-16T05:40:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f8514dc8740 10 client.16833 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f8514dc8740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f8514dc8740 20 client.16833 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f8514dc8740 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x557d746325c0 0x557d746349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f8514dc8740 4 client.16833 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f8514dc8740 1 -- 172.21.15.32:0/1643407480 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d7458bba0 con 0x557d746325c0 2024-04-16T05:40:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.443+0000 7f850b7fe640 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x557d746325c0 0x557d746349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.444+0000 7f850b7fe640 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x557d746325c0 0x557d746349b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2084f75f7105c31d server_cookie=202d5bf9832de6ed in_seq=0 out_seq=0 2024-04-16T05:40:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.444+0000 7f84eaffd640 10 client.16833 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.444+0000 7f84eaffd640 1 -- 172.21.15.32:0/1643407480 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d7458bba0 con 0x557d746325c0 2024-04-16T05:40:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.444+0000 7f84eaffd640 10 client.16833 handle_command_reply: tid=0 2024-04-16T05:40:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.444+0000 7f8514dc8740 1 client.16833 shutdown 2024-04-16T05:40:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.444+0000 7f8514dc8740 1 -- 172.21.15.32:0/1643407480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74622350 msgr2=0x557d7461fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74622350 0x557d7461fe00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f850400de10 tx=0x7f8504004d90 comp rx=0 tx=0).stop 2024-04-16T05:40:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 -- 172.21.15.32:0/1643407480 shutdown_connections 2024-04-16T05:40:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74622350 0x557d7461fe00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x557d746325c0 0x557d746349b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74619770 0x557d7461f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 --2- 172.21.15.32:0/1643407480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 0x557d7461f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f84eaffd640 0 client.16833 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 -- 172.21.15.32:0/1643407480 >> 172.21.15.32:0/1643407480 conn(0x557d74613580 msgr2=0x557d746152d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 -- 172.21.15.32:0/1643407480 shutdown_connections 2024-04-16T05:40:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 1 -- 172.21.15.32:0/1643407480 wait complete. 2024-04-16T05:40:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.445+0000 7f8514dc8740 20 client.16833 trim_cache size 0 max 16384 2024-04-16T05:40:23.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.462+0000 7f8514dc8740 1 Processor -- start 2024-04-16T05:40:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.462+0000 7f8514dc8740 1 -- start start 2024-04-16T05:40:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 0x557d74621ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74619770 0x557d7462e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 0x557d7463a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8514dc8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b3ee0 con 0x557d74619770 2024-04-16T05:40:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8514dc8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b17b0 con 0x557d74618e50 2024-04-16T05:40:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8514dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1af600 con 0x557d74622350 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74619770 0x557d7462e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74619770 0x557d7462e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49720/0 (socket says 172.21.15.32:49720) 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 -- 172.21.15.32:0/2435011032 learned_addr learned my addr 172.21.15.32:0/2435011032 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f850b7fe640 1 --2- 172.21.15.32:0/2435011032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 0x557d74621ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 -- 172.21.15.32:0/2435011032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 msgr2=0x557d7463a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 --2- 172.21.15.32:0/2435011032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 0x557d7463a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 -- 172.21.15.32:0/2435011032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 msgr2=0x557d74621ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 --2- 172.21.15.32:0/2435011032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 0x557d74621ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 -- 172.21.15.32:0/2435011032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d74618170 con 0x557d74619770 2024-04-16T05:40:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f8503fff640 1 --2- 172.21.15.32:0/2435011032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74619770 0x557d7462e000 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009930 tx=0x7f84f402c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd2d892b7c959fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f85017fa640 1 -- 172.21.15.32:0/2435011032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f402cca0 con 0x557d74619770 2024-04-16T05:40:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f85017fa640 1 -- 172.21.15.32:0/2435011032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f4041070 con 0x557d74619770 2024-04-16T05:40:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.463+0000 7f85017fa640 1 -- 172.21.15.32:0/2435011032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4004a40 con 0x557d74619770 2024-04-16T05:40:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 -- 172.21.15.32:0/2435011032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74619770 msgr2=0x557d7462e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2435011032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74619770 0x557d7462e000 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009930 tx=0x7f84f402c930 comp rx=0 tx=0).stop 2024-04-16T05:40:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 -- 172.21.15.32:0/2435011032 shutdown_connections 2024-04-16T05:40:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2435011032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 0x557d7463a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2435011032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74619770 0x557d7462e000 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2435011032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74618e50 0x557d74621ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 -- 172.21.15.32:0/2435011032 >> 172.21.15.32:0/2435011032 conn(0x557d74613580 msgr2=0x557d7461e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 -- 172.21.15.32:0/2435011032 shutdown_connections 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 -- 172.21.15.32:0/2435011032 wait complete. 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.464+0000 7f8514dc8740 1 -- start start 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8514dc8740 1 Processor -- start 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74627e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74619770 0x557d74628390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8514dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 0x557d746288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8514dc8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1afc70 con 0x557d74618e50 2024-04-16T05:40:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8514dc8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b10e0 con 0x557d74619770 2024-04-16T05:40:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8514dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b0340 con 0x557d74622350 2024-04-16T05:40:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8514dc8740 10 client.0 _ll_register_callbacks cb 0x7f84f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74627e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74627e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49726/0 (socket says 172.21.15.32:49726) 2024-04-16T05:40:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 -- 172.21.15.32:0/2348451852 learned_addr learned my addr 172.21.15.32:0/2348451852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850bfff640 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 0x557d746288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8503fff640 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74619770 0x557d74628390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 -- 172.21.15.32:0/2348451852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 msgr2=0x557d746288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 0x557d746288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 -- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74619770 msgr2=0x557d74628390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74619770 0x557d74628390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 -- 172.21.15.32:0/2348451852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d74697920 con 0x557d74618e50 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f8503fff640 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74619770 0x557d74628390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f850b7fe640 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74627e50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f84fc098a40 tx=0x7f84fc09c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708c854e70d76fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.465+0000 7f84eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f84eaffd640 1 -- 172.21.15.32:0/2348451852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0b4060 con 0x557d74618e50 2024-04-16T05:40:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f84eaffd640 1 -- 172.21.15.32:0/2348451852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f84fc098c50 con 0x557d74618e50 2024-04-16T05:40:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 1 -- 172.21.15.32:0/2348451852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84f4041070 con 0x557d74618e50 2024-04-16T05:40:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f84eaffd640 1 -- 172.21.15.32:0/2348451852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84fc01d040 con 0x557d74618e50 2024-04-16T05:40:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 10 client.16839 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 1 -- 172.21.15.32:0/2348451852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d74698b90 con 0x557d74618e50 2024-04-16T05:40:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f84eaffd640 1 -- 172.21.15.32:0/2348451852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f4041070 con 0x557d74618e50 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f84eaffd640 1 -- 172.21.15.32:0/2348451852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f84fc0bb350 con 0x557d74618e50 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 10 client.16839 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 20 client.16839 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x557d74699260 0x557d7469b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 4 client.16839 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8514dc8740 1 -- 172.21.15.32:0/2348451852 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d74585a10 con 0x557d74699260 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.466+0000 7f8503fff640 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x557d74699260 0x557d7469b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8503fff640 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x557d74699260 0x557d7469b680 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=644b5c430e00a37d server_cookie=88a0f69b5cbaf17e in_seq=0 out_seq=0 2024-04-16T05:40:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f84eaffd640 10 client.16839 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f84eaffd640 1 -- 172.21.15.32:0/2348451852 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557d74585a10 con 0x557d74699260 2024-04-16T05:40:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f84eaffd640 10 client.16839 handle_command_reply: tid=0 2024-04-16T05:40:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 client.16839 shutdown 2024-04-16T05:40:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 -- 172.21.15.32:0/2348451852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 msgr2=0x557d74627e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74627e50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f84fc098a40 tx=0x7f84fc09c5e0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 -- 172.21.15.32:0/2348451852 shutdown_connections 2024-04-16T05:40:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x557d74699260 0x557d7469b680 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d74622350 0x557d746288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d74619770 0x557d74628390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f84eaffd640 0 client.16839 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 --2- 172.21.15.32:0/2348451852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d74618e50 0x557d74627e50 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 -- 172.21.15.32:0/2348451852 >> 172.21.15.32:0/2348451852 conn(0x557d74613580 msgr2=0x557d74611ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 -- 172.21.15.32:0/2348451852 shutdown_connections 2024-04-16T05:40:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.467+0000 7f8514dc8740 1 -- 172.21.15.32:0/2348451852 wait complete. 2024-04-16T05:40:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f8514dc8740 20 client.16839 trim_cache size 0 max 16384 2024-04-16T05:40:23.477 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:40:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:40:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:40:23.477 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:40:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 -- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84f80a3d20 msgr2=0x7f84f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84f80a3d20 0x7f84f80a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f850401e000 tx=0x7f850401f000 comp rx=0 tx=0).stop 2024-04-16T05:40:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 -- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 msgr2=0x7f850c10e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 0x7f850c10e850 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c970 tx=0x7f84fc00ce30 comp rx=0 tx=0).stop 2024-04-16T05:40:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 -- 172.21.15.32:0/431405753 shutdown_connections 2024-04-16T05:40:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c1070e0 0x7f850c10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f850c101bf0 0x7f850c10ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f84f80a3d20 0x7f84f80a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 --2- 172.21.15.32:0/431405753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f850c101250 0x7f850c10e850 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 -- 172.21.15.32:0/431405753 >> 172.21.15.32:0/431405753 conn(0x7f850c076d00 msgr2=0x7f850c0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 -- 172.21.15.32:0/431405753 shutdown_connections 2024-04-16T05:40:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.468+0000 7f85121db640 1 -- 172.21.15.32:0/431405753 wait complete. 2024-04-16T05:40:23.480 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:40:23.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:40:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- 172.21.15.32:0/44274810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101af0 msgr2=0x7fa45c105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 --2- 172.21.15.32:0/44274810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101af0 0x7fa45c105b30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009900 tx=0x7fa44c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- 172.21.15.32:0/44274810 shutdown_connections 2024-04-16T05:40:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 --2- 172.21.15.32:0/44274810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c106210 0x7fa45c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 --2- 172.21.15.32:0/44274810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101af0 0x7fa45c105b30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 --2- 172.21.15.32:0/44274810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c101140 0x7fa45c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- 172.21.15.32:0/44274810 >> 172.21.15.32:0/44274810 conn(0x7fa45c077280 msgr2=0x7fa45c077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- 172.21.15.32:0/44274810 shutdown_connections 2024-04-16T05:40:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- 172.21.15.32:0/44274810 wait complete. 2024-04-16T05:40:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 Processor -- start 2024-04-16T05:40:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- start start 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101140 0x7fa45c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c106210 0x7fa45c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c19a1d0 0x7fa45c1a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1112a0 con 0x7fa45c101140 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c111120 con 0x7fa45c106210 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.589+0000 7fa46427b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c111420 con 0x7fa45c19a1d0 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101140 0x7fa45c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101140 0x7fa45c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49734/0 (socket says 172.21.15.32:49734) 2024-04-16T05:40:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 -- 172.21.15.32:0/4239075025 learned_addr learned my addr 172.21.15.32:0/4239075025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa461816640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c106210 0x7fa45c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 -- 172.21.15.32:0/4239075025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c19a1d0 msgr2=0x7fa45c1a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c19a1d0 0x7fa45c1a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 -- 172.21.15.32:0/4239075025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c106210 msgr2=0x7fa45c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c106210 0x7fa45c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 -- 172.21.15.32:0/4239075025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c1a4f80 con 0x7fa45c101140 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa462017640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101140 0x7fa45c199700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa45800d950 tx=0x7fa45800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa4537fe640 1 -- 172.21.15.32:0/4239075025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458017070 con 0x7fa45c101140 2024-04-16T05:40:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c1a5270 con 0x7fa45c101140 2024-04-16T05:40:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa4537fe640 1 -- 172.21.15.32:0/4239075025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa458009c30 con 0x7fa45c101140 2024-04-16T05:40:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa4537fe640 1 -- 172.21.15.32:0/4239075025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45800f6b0 con 0x7fa45c101140 2024-04-16T05:40:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.590+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c1a5790 con 0x7fa45c101140 2024-04-16T05:40:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.591+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45c102610 con 0x7fa45c101140 2024-04-16T05:40:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.592+0000 7fa4537fe640 1 -- 172.21.15.32:0/4239075025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4580109d0 con 0x7fa45c101140 2024-04-16T05:40:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.593+0000 7fa4537fe640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4480a3d20 0x7fa4480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.593+0000 7fa461816640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4480a3d20 0x7fa4480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.593+0000 7fa4537fe640 1 -- 172.21.15.32:0/4239075025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fa4580d08a0 con 0x7fa45c101140 2024-04-16T05:40:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.593+0000 7fa461816640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4480a3d20 0x7fa4480a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa44c006010 tx=0x7fa44c000bc0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.598+0000 7fa4537fe640 1 -- 172.21.15.32:0/4239075025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa458009de0 con 0x7fa45c101140 2024-04-16T05:40:23.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:23 smithi032 ceph-mon[32351]: pgmap v166: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.825+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa45c101af0 con 0x7fa45c101140 2024-04-16T05:40:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.826+0000 7fa4537fe640 1 -- 172.21.15.32:0/4239075025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7fa45809cba0 con 0x7fa45c101140 2024-04-16T05:40:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:40:23.825 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:40:23.825 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:40:23.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4480a3d20 msgr2=0x7fa4480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4480a3d20 0x7fa4480a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa44c006010 tx=0x7fa44c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101140 msgr2=0x7fa45c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101140 0x7fa45c199700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa45800d950 tx=0x7fa45800de10 comp rx=0 tx=0).stop 2024-04-16T05:40:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 shutdown_connections 2024-04-16T05:40:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c19a1d0 0x7fa45c1a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c106210 0x7fa45c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4480a3d20 0x7fa4480a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 --2- 172.21.15.32:0/4239075025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa45c101140 0x7fa45c199700 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 >> 172.21.15.32:0/4239075025 conn(0x7fa45c077280 msgr2=0x7fa45c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 shutdown_connections 2024-04-16T05:40:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.828+0000 7fa46427b640 1 -- 172.21.15.32:0/4239075025 wait complete. 2024-04-16T05:40:23.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:40:23.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:23 smithi134 ceph-mon[36153]: pgmap v166: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:23.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:23 smithi082 ceph-mon[36195]: pgmap v166: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:23.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 -- 172.21.15.32:0/1627345269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06b1f0 msgr2=0x7f005c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1627345269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06b1f0 0x7f005c104a50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0054009900 tx=0x7f005402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 -- 172.21.15.32:0/1627345269 shutdown_connections 2024-04-16T05:40:23.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1627345269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06b1f0 0x7f005c104a50 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1627345269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c06a680 0x7f005c06aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1627345269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c106f50 0x7f005c107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 -- 172.21.15.32:0/1627345269 >> 172.21.15.32:0/1627345269 conn(0x7f005c076d00 msgr2=0x7f005c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 -- 172.21.15.32:0/1627345269 shutdown_connections 2024-04-16T05:40:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.947+0000 7f0063ba8640 1 -- 172.21.15.32:0/1627345269 wait complete. 2024-04-16T05:40:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0063ba8640 1 Processor -- start 2024-04-16T05:40:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0063ba8640 1 -- start start 2024-04-16T05:40:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0063ba8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06a680 0x7f005c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0063ba8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c06b1f0 0x7f005c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0063ba8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106f50 0x7f005c1a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0063ba8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1189b0 con 0x7f005c06a680 2024-04-16T05:40:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0063ba8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f005c118830 con 0x7f005c06b1f0 2024-04-16T05:40:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0063ba8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f005c118b30 con 0x7f005c106f50 2024-04-16T05:40:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0061944640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06a680 0x7f005c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0061944640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06a680 0x7f005c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49758/0 (socket says 172.21.15.32:49758) 2024-04-16T05:40:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0061944640 1 -- 172.21.15.32:0/1657161782 learned_addr learned my addr 172.21.15.32:0/1657161782 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0061143640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c06b1f0 0x7f005c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.948+0000 7f0062145640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106f50 0x7f005c1a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0061944640 1 -- 172.21.15.32:0/1657161782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106f50 msgr2=0x7f005c1a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0061944640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106f50 0x7f005c1a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0061944640 1 -- 172.21.15.32:0/1657161782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c06b1f0 msgr2=0x7f005c19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0061944640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c06b1f0 0x7f005c19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0061944640 1 -- 172.21.15.32:0/1657161782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c1a3de0 con 0x7f005c06a680 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0061944640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06a680 0x7f005c19edf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f005800eab0 tx=0x7f005800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0052ffd640 1 -- 172.21.15.32:0/1657161782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005800cd80 con 0x7f005c06a680 2024-04-16T05:40:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0052ffd640 1 -- 172.21.15.32:0/1657161782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0058011950 con 0x7f005c06a680 2024-04-16T05:40:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0063ba8640 1 -- 172.21.15.32:0/1657161782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c1a40d0 con 0x7f005c06a680 2024-04-16T05:40:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0052ffd640 1 -- 172.21.15.32:0/1657161782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058010a00 con 0x7f005c06a680 2024-04-16T05:40:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.949+0000 7f0063ba8640 1 -- 172.21.15.32:0/1657161782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1ab960 con 0x7f005c06a680 2024-04-16T05:40:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.951+0000 7f0052ffd640 1 -- 172.21.15.32:0/1657161782 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0058011b00 con 0x7f005c06a680 2024-04-16T05:40:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f006678f740 1 Processor -- start 2024-04-16T05:40:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f006678f740 1 -- start start 2024-04-16T05:40:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f0052ffd640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00480a3da0 0x7f00480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f006678f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b8e60 0x56102e2b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f006678f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b9780 0x56102e2c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f006678f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f006678f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56102e21e0e0 con 0x56102e2b8e60 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f006678f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56102e1bb520 con 0x56102e2b9780 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f006678f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56102e21a0a0 con 0x56102e2c2360 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f0061944640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b8e60 0x56102e2b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f0062145640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00480a3da0 0x7f00480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f0061944640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b8e60 0x56102e2b9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49764/0 (socket says 172.21.15.32:49764) 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f0061944640 1 -- 172.21.15.32:0/276917535 learned_addr learned my addr 172.21.15.32:0/276917535 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f0052ffd640 1 -- 172.21.15.32:0/1657161782 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f00580d10a0 con 0x7f005c06a680 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.952+0000 7f0061143640 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b9780 0x56102e2c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0062145640 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0061944640 1 -- 172.21.15.32:0/276917535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 msgr2=0x56102e2c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0061944640 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0061944640 1 -- 172.21.15.32:0/276917535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b9780 msgr2=0x56102e2c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0061944640 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b9780 0x56102e2c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0061944640 1 -- 172.21.15.32:0/276917535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56102e1e38d0 con 0x56102e2b8e60 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0061143640 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b9780 0x56102e2c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0062145640 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0061944640 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b8e60 0x56102e2b9240 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0058099670 tx=0x7f0058014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16f1b99053ff802 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0062145640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00480a3da0 0x7f00480a6260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0054006010 tx=0x7f0054005de0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0047fff640 1 -- 172.21.15.32:0/276917535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005809c600 con 0x56102e2b8e60 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0047fff640 1 -- 172.21.15.32:0/276917535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005809d3a0 con 0x56102e2b8e60 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.953+0000 7f0047fff640 1 -- 172.21.15.32:0/276917535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005800b2b0 con 0x56102e2b8e60 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 -- 172.21.15.32:0/276917535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b8e60 msgr2=0x56102e2b9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b8e60 0x56102e2b9240 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0058099670 tx=0x7f0058014000 comp rx=0 tx=0).stop 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 -- 172.21.15.32:0/276917535 shutdown_connections 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b9780 0x56102e2c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 --2- 172.21.15.32:0/276917535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b8e60 0x56102e2b9240 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 -- 172.21.15.32:0/276917535 >> 172.21.15.32:0/276917535 conn(0x56102e2b3590 msgr2=0x56102e2b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 -- 172.21.15.32:0/276917535 shutdown_connections 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 -- 172.21.15.32:0/276917535 wait complete. 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 -- start start 2024-04-16T05:40:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.954+0000 7f006678f740 1 Processor -- start 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f006678f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f006678f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2b9780 0x56102e2bf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f006678f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2c2360 0x56102e2bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f006678f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56102e2c4fa0 con 0x56102e2c2360 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f006678f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56102e1d74d0 con 0x56102e2b8e60 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f006678f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56102e2c5120 con 0x56102e2b9780 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f006678f740 10 client.0 _ll_register_callbacks cb 0x56102e1bc2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2c2360 0x56102e2bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2c2360 0x56102e2bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49770/0 (socket says 172.21.15.32:49770) 2024-04-16T05:40:23.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 -- 172.21.15.32:0/1597553343 learned_addr learned my addr 172.21.15.32:0/1597553343 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0061944640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0061143640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2b9780 0x56102e2bf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 -- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2b9780 msgr2=0x56102e2bf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2b9780 0x56102e2bf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 -- 172.21.15.32:0/1597553343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 msgr2=0x56102e2bf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2bf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 -- 172.21.15.32:0/1597553343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56102df303f0 con 0x56102e2c2360 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0061143640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2b9780 0x56102e2bf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0061944640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2bf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:40:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f0062145640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2c2360 0x56102e2bfe20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0054004440 tx=0x7f0054009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9707d8172bdae3c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.955+0000 7f00457fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f00457fa640 1 -- 172.21.15.32:0/1597553343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005404e070 con 0x56102e2c2360 2024-04-16T05:40:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f00457fa640 1 -- 172.21.15.32:0/1597553343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f00540048c0 con 0x56102e2c2360 2024-04-16T05:40:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 1 -- 172.21.15.32:0/1597553343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005809d3a0 con 0x56102e2c2360 2024-04-16T05:40:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f00457fa640 1 -- 172.21.15.32:0/1597553343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f005405a050 con 0x56102e2c2360 2024-04-16T05:40:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 10 client.16857 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 1 -- 172.21.15.32:0/1597553343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56102dffa7d0 con 0x56102e2c2360 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f00457fa640 1 -- 172.21.15.32:0/1597553343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005809d3a0 con 0x56102e2c2360 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f00457fa640 1 -- 172.21.15.32:0/1597553343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0054049070 con 0x56102e2c2360 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 10 client.16857 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 10 client.16857 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 20 client.16857 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56102e2d21e0 0x56102e2d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 4 client.16857 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f006678f740 1 -- 172.21.15.32:0/1597553343 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56102e22bb00 con 0x56102e2d21e0 2024-04-16T05:40:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.956+0000 7f0061944640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56102e2d21e0 0x56102e2d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.957+0000 7f00457fa640 10 client.16857 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.957+0000 7f0061944640 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56102e2d21e0 0x56102e2d4660 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=733e6eb242b3158e server_cookie=d267e7767d7df411 in_seq=0 out_seq=0 2024-04-16T05:40:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f00457fa640 1 -- 172.21.15.32:0/1597553343 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56102e22bb00 con 0x56102e2d21e0 2024-04-16T05:40:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f00457fa640 10 client.16857 handle_command_reply: tid=0 2024-04-16T05:40:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 client.16857 shutdown 2024-04-16T05:40:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 -- 172.21.15.32:0/1597553343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2c2360 msgr2=0x56102e2bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2c2360 0x56102e2bfe20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0054004440 tx=0x7f0054009260 comp rx=0 tx=0).stop 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 -- 172.21.15.32:0/1597553343 shutdown_connections 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56102e2d21e0 0x56102e2d4660 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2c2360 0x56102e2bfe20 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2b9780 0x56102e2bf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 --2- 172.21.15.32:0/1597553343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2bf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f00457fa640 0 client.16857 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 -- 172.21.15.32:0/1597553343 >> 172.21.15.32:0/1597553343 conn(0x56102e2b3590 msgr2=0x56102e2b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 -- 172.21.15.32:0/1597553343 shutdown_connections 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 1 -- 172.21.15.32:0/1597553343 wait complete. 2024-04-16T05:40:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.958+0000 7f006678f740 20 client.16857 trim_cache size 0 max 16384 2024-04-16T05:40:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.971+0000 7f006678f740 1 Processor -- start 2024-04-16T05:40:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.971+0000 7f006678f740 1 -- start start 2024-04-16T05:40:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.971+0000 7f006678f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.971+0000 7f006678f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.971+0000 7f006678f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2be500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f006678f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b81f0 con 0x56102e2b9780 2024-04-16T05:40:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f006678f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b5ac0 con 0x56102e2b8e60 2024-04-16T05:40:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f006678f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b3910 con 0x56102e2c2360 2024-04-16T05:40:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49786/0 (socket says 172.21.15.32:49786) 2024-04-16T05:40:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 -- 172.21.15.32:0/1515397851 learned_addr learned my addr 172.21.15.32:0/1515397851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061944640 1 --2- 172.21.15.32:0/1515397851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 -- 172.21.15.32:0/1515397851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 msgr2=0x56102e2be500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 --2- 172.21.15.32:0/1515397851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2be500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 -- 172.21.15.32:0/1515397851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 msgr2=0x56102e2c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 --2- 172.21.15.32:0/1515397851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2c1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 -- 172.21.15.32:0/1515397851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56102e2c4750 con 0x56102e2b9780 2024-04-16T05:40:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f0061143640 1 --2- 172.21.15.32:0/1515397851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2cdef0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f004c00ab90 tx=0x7f004c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d627c7684294f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f00477fe640 1 -- 172.21.15.32:0/1515397851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c005020 con 0x56102e2b9780 2024-04-16T05:40:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f00477fe640 1 -- 172.21.15.32:0/1515397851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c0056a0 con 0x56102e2b9780 2024-04-16T05:40:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.972+0000 7f00477fe640 1 -- 172.21.15.32:0/1515397851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c00f790 con 0x56102e2b9780 2024-04-16T05:40:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 -- 172.21.15.32:0/1515397851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 msgr2=0x56102e2cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 --2- 172.21.15.32:0/1515397851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2cdef0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f004c00ab90 tx=0x7f004c012820 comp rx=0 tx=0).stop 2024-04-16T05:40:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 -- 172.21.15.32:0/1515397851 shutdown_connections 2024-04-16T05:40:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 --2- 172.21.15.32:0/1515397851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2be500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 --2- 172.21.15.32:0/1515397851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2cdef0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 --2- 172.21.15.32:0/1515397851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 -- 172.21.15.32:0/1515397851 >> 172.21.15.32:0/1515397851 conn(0x56102e2b3590 msgr2=0x56102e2b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 -- 172.21.15.32:0/1515397851 shutdown_connections 2024-04-16T05:40:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 -- 172.21.15.32:0/1515397851 wait complete. 2024-04-16T05:40:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 -- start start 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.973+0000 7f006678f740 1 Processor -- start 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f006678f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f006678f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f006678f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f006678f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b3f80 con 0x56102e2b9780 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f006678f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b53f0 con 0x56102e2b8e60 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f006678f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b4650 con 0x56102e2c2360 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f006678f740 10 client.0 _ll_register_callbacks cb 0x7f00540040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49802/0 (socket says 172.21.15.32:49802) 2024-04-16T05:40:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 -- 172.21.15.32:0/1889874612 learned_addr learned my addr 172.21.15.32:0/1889874612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061944640 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0062145640 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 -- 172.21.15.32:0/1889874612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 msgr2=0x56102e2c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 -- 172.21.15.32:0/1889874612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 msgr2=0x56102e2ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 -- 172.21.15.32:0/1889874612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56102e2c6850 con 0x56102e2b9780 2024-04-16T05:40:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f0061143640 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2c5b00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f004c00ab60 tx=0x7f004c0052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99abf9c1ec86e140 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f00457fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f00457fa640 1 -- 172.21.15.32:0/1889874612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c029020 con 0x56102e2b9780 2024-04-16T05:40:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f00457fa640 1 -- 172.21.15.32:0/1889874612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f004c00f490 con 0x56102e2b9780 2024-04-16T05:40:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.974+0000 7f006678f740 1 -- 172.21.15.32:0/1889874612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004c0056a0 con 0x56102e2b9780 2024-04-16T05:40:23.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f00457fa640 1 -- 172.21.15.32:0/1889874612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f004c034050 con 0x56102e2b9780 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f006678f740 10 client.16863 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f006678f740 1 -- 172.21.15.32:0/1889874612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56102e338770 con 0x56102e2b9780 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f00457fa640 1 -- 172.21.15.32:0/1889874612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c0056a0 con 0x56102e2b9780 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f00457fa640 1 -- 172.21.15.32:0/1889874612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f004c0247d0 con 0x56102e2b9780 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f006678f740 10 client.16863 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f006678f740 10 client.16863 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f006678f740 20 client.16863 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f006678f740 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56102e338e60 0x56102e33b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f006678f740 4 client.16863 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f006678f740 1 -- 172.21.15.32:0/1889874612 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56102e225970 con 0x56102e338e60 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.975+0000 7f0061944640 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56102e338e60 0x56102e33b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f0061944640 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56102e338e60 0x56102e33b280 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f086737fd3fbabdb server_cookie=d1de8bf40bf5b84b in_seq=0 out_seq=0 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f00457fa640 10 client.16863 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f00457fa640 1 -- 172.21.15.32:0/1889874612 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56102e225970 con 0x56102e338e60 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f00457fa640 10 client.16863 handle_command_reply: tid=0 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 client.16863 shutdown 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 -- 172.21.15.32:0/1889874612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 msgr2=0x56102e2c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2c5b00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f004c00ab60 tx=0x7f004c0052d0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 -- 172.21.15.32:0/1889874612 shutdown_connections 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56102e338e60 0x56102e33b280 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56102e2c2360 0x56102e2c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f00457fa640 0 client.16863 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56102e2b9780 0x56102e2c5b00 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 --2- 172.21.15.32:0/1889874612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56102e2b8e60 0x56102e2ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 -- 172.21.15.32:0/1889874612 >> 172.21.15.32:0/1889874612 conn(0x56102e2b3590 msgr2=0x56102e2bf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 -- 172.21.15.32:0/1889874612 shutdown_connections 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 1 -- 172.21.15.32:0/1889874612 wait complete. 2024-04-16T05:40:23.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.976+0000 7f006678f740 20 client.16863 trim_cache size 0 max 16384 2024-04-16T05:40:23.986 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:23.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 -- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00480a3da0 msgr2=0x7f00480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00480a3da0 0x7f00480a6260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0054006010 tx=0x7f0054005de0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 -- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06a680 msgr2=0x7f005c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06a680 0x7f005c19edf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f005800eab0 tx=0x7f005800eeb0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 -- 172.21.15.32:0/1657161782 shutdown_connections 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f005c106f50 0x7f005c1a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f005c06b1f0 0x7f005c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f00480a3da0 0x7f00480a6260 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 --2- 172.21.15.32:0/1657161782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f005c06a680 0x7f005c19edf0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 -- 172.21.15.32:0/1657161782 >> 172.21.15.32:0/1657161782 conn(0x7f005c076d00 msgr2=0x7f005c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 -- 172.21.15.32:0/1657161782 shutdown_connections 2024-04-16T05:40:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:23.977+0000 7f0063ba8640 1 -- 172.21.15.32:0/1657161782 wait complete. 2024-04-16T05:40:23.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:40:24.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 -- 172.21.15.32:0/1299151639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab781032a0 msgr2=0x7fab78103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 --2- 172.21.15.32:0/1299151639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab781032a0 0x7fab78103680 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fab68009ca0 tx=0x7fab6802c230 comp rx=0 tx=0).stop 2024-04-16T05:40:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 -- 172.21.15.32:0/1299151639 shutdown_connections 2024-04-16T05:40:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 --2- 172.21.15.32:0/1299151639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab7810c5e0 0x7fab7810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 --2- 172.21.15.32:0/1299151639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab78103bc0 0x7fab7810c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 --2- 172.21.15.32:0/1299151639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab781032a0 0x7fab78103680 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 -- 172.21.15.32:0/1299151639 >> 172.21.15.32:0/1299151639 conn(0x7fab78077280 msgr2=0x7fab78077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 -- 172.21.15.32:0/1299151639 shutdown_connections 2024-04-16T05:40:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.097+0000 7fab7d10e640 1 -- 172.21.15.32:0/1299151639 wait complete. 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.098+0000 7fab7d10e640 1 Processor -- start 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.098+0000 7fab7d10e640 1 -- start start 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7d10e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab781032a0 0x7fab7819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7d10e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab78103bc0 0x7fab7819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7d10e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7810c5e0 0x7fab781986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7d10e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab781123c0 con 0x7fab7810c5e0 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7d10e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fab78112240 con 0x7fab78103bc0 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7d10e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab78112540 con 0x7fab781032a0 2024-04-16T05:40:24.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7810c5e0 0x7fab781986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7810c5e0 0x7fab781986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49828/0 (socket says 172.21.15.32:49828) 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 -- 172.21.15.32:0/404426977 learned_addr learned my addr 172.21.15.32:0/404426977 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7659c640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab78103bc0 0x7fab7819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab76d9d640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab781032a0 0x7fab7819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 -- 172.21.15.32:0/404426977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab781032a0 msgr2=0x7fab7819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab781032a0 0x7fab7819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 -- 172.21.15.32:0/404426977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab78103bc0 msgr2=0x7fab7819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab78103bc0 0x7fab7819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 -- 172.21.15.32:0/404426977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab78198f60 con 0x7fab7810c5e0 2024-04-16T05:40:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab76d9d640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab781032a0 0x7fab7819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7759e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7810c5e0 0x7fab781986a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fab7000d750 tx=0x7fab7000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab6ffff640 1 -- 172.21.15.32:0/404426977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70017070 con 0x7fab7810c5e0 2024-04-16T05:40:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab6ffff640 1 -- 172.21.15.32:0/404426977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab70005500 con 0x7fab7810c5e0 2024-04-16T05:40:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7d10e640 1 -- 172.21.15.32:0/404426977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab78199250 con 0x7fab7810c5e0 2024-04-16T05:40:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab6ffff640 1 -- 172.21.15.32:0/404426977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70004d90 con 0x7fab7810c5e0 2024-04-16T05:40:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.099+0000 7fab7d10e640 1 -- 172.21.15.32:0/404426977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab781a5560 con 0x7fab7810c5e0 2024-04-16T05:40:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab6ffff640 1 -- 172.21.15.32:0/404426977 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab70004f30 con 0x7fab7810c5e0 2024-04-16T05:40:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7fdcd740 1 Processor -- start 2024-04-16T05:40:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7fdcd740 1 -- start start 2024-04-16T05:40:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e9175240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e917de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e9180750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b2e90da0e0 con 0x55b2e9174e60 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b2e9077520 con 0x55b2e9175780 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2e90d60a0 con 0x55b2e917e360 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e9175240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e9175240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49844/0 (socket says 172.21.15.32:49844) 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab7659c640 1 -- 172.21.15.32:0/3357191070 learned_addr learned my addr 172.21.15.32:0/3357191070 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.102+0000 7fab6ffff640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fab640a3d20 0x7fab640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab76d9d640 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e917de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7759e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fab640a3d20 0x7fab640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7659c640 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e9180750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7659c640 1 -- 172.21.15.32:0/3357191070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 msgr2=0x55b2e9180750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7659c640 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e9180750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7659c640 1 -- 172.21.15.32:0/3357191070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 msgr2=0x55b2e917de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7659c640 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e917de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab6ffff640 1 -- 172.21.15.32:0/404426977 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fab700d0450 con 0x7fab7810c5e0 2024-04-16T05:40:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7659c640 1 -- 172.21.15.32:0/3357191070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e909f8d0 con 0x55b2e9174e60 2024-04-16T05:40:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7659c640 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e9175240 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fab6000bfa0 tx=0x7fab6000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f697e071018dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab7759e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fab640a3d20 0x7fab640a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab700cde60 tx=0x7fab70000bc0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab6d7fa640 1 -- 172.21.15.32:0/3357191070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60004030 con 0x55b2e9174e60 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab6d7fa640 1 -- 172.21.15.32:0/3357191070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab600041d0 con 0x55b2e9174e60 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.103+0000 7fab6d7fa640 1 -- 172.21.15.32:0/3357191070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab600106c0 con 0x55b2e9174e60 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3357191070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 msgr2=0x55b2e9175240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e9175240 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fab6000bfa0 tx=0x7fab6000b840 comp rx=0 tx=0).stop 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3357191070 shutdown_connections 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e9180750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e917de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/3357191070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e9175240 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3357191070 >> 172.21.15.32:0/3357191070 conn(0x55b2e916f590 msgr2=0x55b2e916f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3357191070 shutdown_connections 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3357191070 wait complete. 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 -- start start 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.104+0000 7fab7fdcd740 1 Processor -- start 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e9174e60 0x55b2e917b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e917b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e917e360 0x55b2e917be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b2e9181060 con 0x55b2e917e360 2024-04-16T05:40:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b2e90934d0 con 0x55b2e9175780 2024-04-16T05:40:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2e91811e0 con 0x55b2e9174e60 2024-04-16T05:40:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7fdcd740 10 client.0 _ll_register_callbacks cb 0x55b2e90782c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e917e360 0x55b2e917be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e917e360 0x55b2e917be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49848/0 (socket says 172.21.15.32:49848) 2024-04-16T05:40:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 -- 172.21.15.32:0/1778977814 learned_addr learned my addr 172.21.15.32:0/1778977814 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab76d9d640 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e917b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e9174e60 0x55b2e917b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 -- 172.21.15.32:0/1778977814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e9174e60 msgr2=0x55b2e917b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e9174e60 0x55b2e917b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 -- 172.21.15.32:0/1778977814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 msgr2=0x55b2e917b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e917b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 -- 172.21.15.32:0/1778977814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e8dec3f0 con 0x55b2e917e360 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7659c640 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e917e360 0x55b2e917be00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fab6000bee0 tx=0x7fab60003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440470956ee6166f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1778977814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab600037f0 con 0x55b2e917e360 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1778977814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fab60005050 con 0x55b2e917e360 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.105+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1778977814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab600041d0 con 0x55b2e917e360 2024-04-16T05:40:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1778977814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab60034050 con 0x55b2e917e360 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab7fdcd740 10 client.16875 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1778977814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b2e8eb67d0 con 0x55b2e917e360 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1778977814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab600041d0 con 0x55b2e917e360 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1778977814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab6002c020 con 0x55b2e917e360 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab7fdcd740 10 client.16875 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab7fdcd740 10 client.16875 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab7fdcd740 20 client.16875 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b2e918e270 0x55b2e9190660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab7fdcd740 4 client.16875 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1778977814 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2e90e7b00 con 0x55b2e918e270 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.106+0000 7fab76d9d640 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b2e918e270 0x55b2e9190660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.107+0000 7fab5e7fc640 10 client.16875 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.107+0000 7fab76d9d640 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b2e918e270 0x55b2e9190660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc375aab5b695a0a server_cookie=2c44ffe81ae661c5 in_seq=0 out_seq=0 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.107+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1778977814 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2e90e7b00 con 0x55b2e918e270 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.107+0000 7fab5e7fc640 10 client.16875 handle_command_reply: tid=0 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.107+0000 7fab7fdcd740 1 client.16875 shutdown 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.107+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1778977814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e917e360 msgr2=0x55b2e917be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.107+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e917e360 0x55b2e917be00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fab6000bee0 tx=0x7fab60003cb0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1778977814 shutdown_connections 2024-04-16T05:40:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b2e918e270 0x55b2e9190660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e917e360 0x55b2e917be00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e917b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab5e7fc640 0 client.16875 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1778977814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e9174e60 0x55b2e917b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1778977814 >> 172.21.15.32:0/1778977814 conn(0x55b2e916f590 msgr2=0x55b2e91712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1778977814 shutdown_connections 2024-04-16T05:40:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1778977814 wait complete. 2024-04-16T05:40:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.108+0000 7fab7fdcd740 20 client.16875 trim_cache size 0 max 16384 2024-04-16T05:40:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.121+0000 7fab7fdcd740 1 Processor -- start 2024-04-16T05:40:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.121+0000 7fab7fdcd740 1 -- start start 2024-04-16T05:40:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.121+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9174e60 0x55b2e917dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.121+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9175780 0x55b2e918a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.121+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e916e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.121+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab781b1c00 con 0x55b2e9175780 2024-04-16T05:40:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.121+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fab781af4d0 con 0x55b2e9174e60 2024-04-16T05:40:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.121+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab781ad320 con 0x55b2e917e360 2024-04-16T05:40:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9175780 0x55b2e918a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9175780 0x55b2e918a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49860/0 (socket says 172.21.15.32:49860) 2024-04-16T05:40:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 -- 172.21.15.32:0/3226415317 learned_addr learned my addr 172.21.15.32:0/3226415317 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab7659c640 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e916e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 -- 172.21.15.32:0/3226415317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 msgr2=0x55b2e916e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e916e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 -- 172.21.15.32:0/3226415317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9174e60 msgr2=0x55b2e917dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9174e60 0x55b2e917dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 -- 172.21.15.32:0/3226415317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e9190bf0 con 0x55b2e9175780 2024-04-16T05:40:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab7659c640 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e916e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab76d9d640 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9175780 0x55b2e918a030 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fab6802c810 tx=0x7fab68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d842d2f4b551038 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab6cff9640 1 -- 172.21.15.32:0/3226415317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68050070 con 0x55b2e9175780 2024-04-16T05:40:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab6cff9640 1 -- 172.21.15.32:0/3226415317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab68003e50 con 0x55b2e9175780 2024-04-16T05:40:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.122+0000 7fab6cff9640 1 -- 172.21.15.32:0/3226415317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68039040 con 0x55b2e9175780 2024-04-16T05:40:24.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3226415317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9175780 msgr2=0x55b2e918a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9175780 0x55b2e918a030 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fab6802c810 tx=0x7fab68004a40 comp rx=0 tx=0).stop 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3226415317 shutdown_connections 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e916e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9175780 0x55b2e918a030 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/3226415317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9174e60 0x55b2e917dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3226415317 >> 172.21.15.32:0/3226415317 conn(0x55b2e916f590 msgr2=0x55b2e917bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3226415317 shutdown_connections 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 -- 172.21.15.32:0/3226415317 wait complete. 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 -- start start 2024-04-16T05:40:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.123+0000 7fab7fdcd740 1 Processor -- start 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e91869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e9181d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7fdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e9182370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab781ad990 con 0x55b2e9174e60 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fab781aee00 con 0x55b2e9175780 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7fdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab781ae060 con 0x55b2e917e360 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7fdcd740 10 client.0 _ll_register_callbacks cb 0x7fab68037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e91869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e91869f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49862/0 (socket says 172.21.15.32:49862) 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 -- 172.21.15.32:0/1723775330 learned_addr learned my addr 172.21.15.32:0/1723775330 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab76d9d640 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e9181d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e9182370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 -- 172.21.15.32:0/1723775330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 msgr2=0x55b2e9182370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e9182370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 -- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 msgr2=0x55b2e9181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e9181d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 -- 172.21.15.32:0/1723775330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e9180750 con 0x55b2e9174e60 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab76d9d640 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e9181d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:40:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7659c640 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e91869f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fab600122d0 tx=0x7fab6000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255e65b49b556d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1723775330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60004790 con 0x55b2e9174e60 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1723775330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fab600035f0 con 0x55b2e9174e60 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.124+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1723775330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab68003e50 con 0x55b2e9174e60 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1723775330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab60044050 con 0x55b2e9174e60 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab7fdcd740 10 client.16881 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1723775330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b2e91f8940 con 0x55b2e9174e60 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1723775330 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab68003e50 con 0x55b2e9174e60 2024-04-16T05:40:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1723775330 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab6002c040 con 0x55b2e9174e60 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab7fdcd740 10 client.16881 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab7fdcd740 10 client.16881 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab7fdcd740 20 client.16881 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b2e91f9060 0x55b2e91fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab7fdcd740 4 client.16881 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1723775330 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2e90e1970 con 0x55b2e91f9060 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.125+0000 7fab76d9d640 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b2e91f9060 0x55b2e91fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab76d9d640 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b2e91f9060 0x55b2e91fb480 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5561396e6e7f839e server_cookie=536fbedaccf8987d in_seq=0 out_seq=0 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab5e7fc640 10 client.16881 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab5e7fc640 1 -- 172.21.15.32:0/1723775330 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2e90e1970 con 0x55b2e91f9060 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab5e7fc640 10 client.16881 handle_command_reply: tid=0 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 client.16881 shutdown 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1723775330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 msgr2=0x55b2e91869f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e91869f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fab600122d0 tx=0x7fab6000bfa0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1723775330 shutdown_connections 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b2e91f9060 0x55b2e91fb480 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e917e360 0x55b2e9182370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2e9175780 0x55b2e9181d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 --2- 172.21.15.32:0/1723775330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2e9174e60 0x55b2e91869f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab5e7fc640 0 client.16881 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.126+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1723775330 >> 172.21.15.32:0/1723775330 conn(0x55b2e916f590 msgr2=0x55b2e91707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1723775330 shutdown_connections 2024-04-16T05:40:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7fdcd740 1 -- 172.21.15.32:0/1723775330 wait complete. 2024-04-16T05:40:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7fdcd740 20 client.16881 trim_cache size 0 max 16384 2024-04-16T05:40:24.134 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 -- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fab640a3d20 msgr2=0x7fab640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fab640a3d20 0x7fab640a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab700cde60 tx=0x7fab70000bc0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 -- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7810c5e0 msgr2=0x7fab781986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7810c5e0 0x7fab781986a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fab7000d750 tx=0x7fab7000dc10 comp rx=0 tx=0).stop 2024-04-16T05:40:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 -- 172.21.15.32:0/404426977 shutdown_connections 2024-04-16T05:40:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fab640a3d20 0x7fab640a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7810c5e0 0x7fab781986a0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab78103bc0 0x7fab7819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 --2- 172.21.15.32:0/404426977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab781032a0 0x7fab7819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 -- 172.21.15.32:0/404426977 >> 172.21.15.32:0/404426977 conn(0x7fab78077280 msgr2=0x7fab780fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 -- 172.21.15.32:0/404426977 shutdown_connections 2024-04-16T05:40:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.127+0000 7fab7d10e640 1 -- 172.21.15.32:0/404426977 wait complete. 2024-04-16T05:40:24.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:40:24.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 -- 172.21.15.32:0/1855977318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c1007c0 msgr2=0x7fbf0c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 --2- 172.21.15.32:0/1855977318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c1007c0 0x7fbf0c100ba0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009900 tx=0x7fbefc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:24.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 -- 172.21.15.32:0/1855977318 shutdown_connections 2024-04-16T05:40:24.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 --2- 172.21.15.32:0/1855977318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c107380 0x7fbf0c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 --2- 172.21.15.32:0/1855977318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf0c101170 0x7fbf0c106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 --2- 172.21.15.32:0/1855977318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c1007c0 0x7fbf0c100ba0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 -- 172.21.15.32:0/1855977318 >> 172.21.15.32:0/1855977318 conn(0x7fbf0c077410 msgr2=0x7fbf0c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 -- 172.21.15.32:0/1855977318 shutdown_connections 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.246+0000 7fbf1161b640 1 -- 172.21.15.32:0/1855977318 wait complete. 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf1161b640 1 Processor -- start 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf1161b640 1 -- start start 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf1161b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf0c1007c0 0x7fbf0c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf1161b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c107380 0x7fbf0c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf1161b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c198680 0x7fbf0c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf1161b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1123a0 con 0x7fbf0c107380 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf1161b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c112220 con 0x7fbf0c1007c0 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf1161b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c112520 con 0x7fbf0c198680 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c107380 0x7fbf0c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c107380 0x7fbf0c19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49872/0 (socket says 172.21.15.32:49872) 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf03fff640 1 -- 172.21.15.32:0/2238547915 learned_addr learned my addr 172.21.15.32:0/2238547915 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf0c1007c0 0x7fbf0c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.247+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c198680 0x7fbf0c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf03fff640 1 -- 172.21.15.32:0/2238547915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c198680 msgr2=0x7fbf0c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf03fff640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c198680 0x7fbf0c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf03fff640 1 -- 172.21.15.32:0/2238547915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf0c1007c0 msgr2=0x7fbf0c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf03fff640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf0c1007c0 0x7fbf0c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf03fff640 1 -- 172.21.15.32:0/2238547915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c199280 con 0x7fbf0c107380 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c198680 0x7fbf0c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf0c1007c0 0x7fbf0c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf03fff640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c107380 0x7fbf0c19eb00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbef400c970 tx=0x7fbef400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf08ff9640 1 -- 172.21.15.32:0/2238547915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4014070 con 0x7fbf0c107380 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf08ff9640 1 -- 172.21.15.32:0/2238547915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef4004440 con 0x7fbf0c107380 2024-04-16T05:40:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf08ff9640 1 -- 172.21.15.32:0/2238547915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef400f450 con 0x7fbf0c107380 2024-04-16T05:40:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf1161b640 1 -- 172.21.15.32:0/2238547915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c1a5360 con 0x7fbf0c107380 2024-04-16T05:40:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.248+0000 7fbf1161b640 1 -- 172.21.15.32:0/2238547915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c1a57c0 con 0x7fbf0c107380 2024-04-16T05:40:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.251+0000 7fbf14209740 1 Processor -- start 2024-04-16T05:40:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.251+0000 7fbf08ff9640 1 -- 172.21.15.32:0/2238547915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbef4017070 con 0x7fbf0c107380 2024-04-16T05:40:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.251+0000 7fbf14209740 1 -- start start 2024-04-16T05:40:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.251+0000 7fbf14209740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d4864e60 0x55a3d4865240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.251+0000 7fbf14209740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d486de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.251+0000 7fbf14209740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d486e360 0x55a3d4870750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf14209740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a3d47ca0e0 con 0x55a3d4864e60 2024-04-16T05:40:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf14209740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4767520 con 0x55a3d4865780 2024-04-16T05:40:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf14209740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a3d47c60a0 con 0x55a3d486e360 2024-04-16T05:40:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d4864e60 0x55a3d4865240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d4864e60 0x55a3d4865240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49882/0 (socket says 172.21.15.32:49882) 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/3082520046 learned_addr learned my addr 172.21.15.32:0/3082520046 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf03fff640 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d486e360 0x55a3d4870750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0affd640 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d486de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/3082520046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d486e360 msgr2=0x55a3d4870750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d486e360 0x55a3d4870750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/3082520046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 msgr2=0x55a3d486de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d486de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/3082520046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d478f8d0 con 0x55a3d4864e60 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf08ff9640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbef80a3da0 0x7fbef80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0affd640 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d486de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbef80a3da0 0x7fbef80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d4864e60 0x55a3d4865240 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbf04008d50 tx=0x7fbf0400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad54cc267f22216 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf08ff9640 1 -- 172.21.15.32:0/2238547915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbef40d09d0 con 0x7fbf0c107380 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf017fa640 1 -- 172.21.15.32:0/3082520046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400f830 con 0x55a3d4864e60 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf017fa640 1 -- 172.21.15.32:0/3082520046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0400de20 con 0x55a3d4864e60 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.252+0000 7fbf017fa640 1 -- 172.21.15.32:0/3082520046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400e3b0 con 0x55a3d4864e60 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbef80a3da0 0x7fbef80a6260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbefc02cfd0 tx=0x7fbefc005a20 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 -- 172.21.15.32:0/3082520046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d4864e60 msgr2=0x55a3d4865240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d4864e60 0x55a3d4865240 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbf04008d50 tx=0x7fbf0400d710 comp rx=0 tx=0).stop 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 -- 172.21.15.32:0/3082520046 shutdown_connections 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d486e360 0x55a3d4870750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d486de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 --2- 172.21.15.32:0/3082520046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d4864e60 0x55a3d4865240 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 -- 172.21.15.32:0/3082520046 >> 172.21.15.32:0/3082520046 conn(0x55a3d485f590 msgr2=0x55a3d485f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 -- 172.21.15.32:0/3082520046 shutdown_connections 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.253+0000 7fbf14209740 1 -- 172.21.15.32:0/3082520046 wait complete. 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 1 -- start start 2024-04-16T05:40:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 1 Processor -- start 2024-04-16T05:40:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4864e60 0x55a3d486b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4865780 0x55a3d486b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d486be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4871000 con 0x55a3d486e360 2024-04-16T05:40:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a3d47834d0 con 0x55a3d4864e60 2024-04-16T05:40:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4871180 con 0x55a3d4865780 2024-04-16T05:40:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf14209740 10 client.0 _ll_register_callbacks cb 0x55a3d47682c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d486be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d486be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49892/0 (socket says 172.21.15.32:49892) 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2140712014 learned_addr learned my addr 172.21.15.32:0/2140712014 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf03fff640 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4865780 0x55a3d486b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4864e60 0x55a3d486b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2140712014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4865780 msgr2=0x55a3d486b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4865780 0x55a3d486b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2140712014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4864e60 msgr2=0x55a3d486b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4864e60 0x55a3d486b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.254+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2140712014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d44dc3f0 con 0x55a3d486e360 2024-04-16T05:40:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d486be40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbf04018fd0 tx=0x7fbf04008ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98f7fb611d7d823 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbef27fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbef27fc640 1 -- 172.21.15.32:0/2140712014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400f830 con 0x55a3d486e360 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbef27fc640 1 -- 172.21.15.32:0/2140712014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbf0400f9d0 con 0x55a3d486e360 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 1 -- 172.21.15.32:0/2140712014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf0400de20 con 0x55a3d486e360 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbef27fc640 1 -- 172.21.15.32:0/2140712014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf04031050 con 0x55a3d486e360 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 10 client.16893 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 1 -- 172.21.15.32:0/2140712014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a3d45a67d0 con 0x55a3d486e360 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbef27fc640 1 -- 172.21.15.32:0/2140712014 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0400de20 con 0x55a3d486e360 2024-04-16T05:40:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbef27fc640 1 -- 172.21.15.32:0/2140712014 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbf040213c0 con 0x55a3d486e360 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 10 client.16893 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 20 client.16893 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a3d487e2d0 0x55a3d48806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 4 client.16893 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.255+0000 7fbf14209740 1 -- 172.21.15.32:0/2140712014 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3d47d7b00 con 0x55a3d487e2d0 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.256+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a3d487e2d0 0x55a3d48806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.256+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a3d487e2d0 0x55a3d48806c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=358ef113a0af645f server_cookie=1f21fb0d634adbfd in_seq=0 out_seq=0 2024-04-16T05:40:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.256+0000 7fbef27fc640 10 client.16893 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbef27fc640 1 -- 172.21.15.32:0/2140712014 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3d47d7b00 con 0x55a3d487e2d0 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbef27fc640 10 client.16893 handle_command_reply: tid=0 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 client.16893 shutdown 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 -- 172.21.15.32:0/2140712014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 msgr2=0x55a3d486be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d486be40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbf04018fd0 tx=0x7fbf04008ac0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 -- 172.21.15.32:0/2140712014 shutdown_connections 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d486be40 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a3d487e2d0 0x55a3d48806c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4865780 0x55a3d486b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 --2- 172.21.15.32:0/2140712014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4864e60 0x55a3d486b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 -- 172.21.15.32:0/2140712014 >> 172.21.15.32:0/2140712014 conn(0x55a3d485f590 msgr2=0x55a3d48612e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbef27fc640 0 client.16893 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:24.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 -- 172.21.15.32:0/2140712014 shutdown_connections 2024-04-16T05:40:24.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 1 -- 172.21.15.32:0/2140712014 wait complete. 2024-04-16T05:40:24.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.257+0000 7fbf14209740 20 client.16893 trim_cache size 0 max 16384 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.270+0000 7fbf14209740 1 Processor -- start 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf14209740 1 -- start start 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf14209740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d487a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf14209740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d485a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf14209740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d4885d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf14209740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b1c50 con 0x55a3d486e360 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf14209740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1af520 con 0x55a3d4865780 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf14209740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1ad370 con 0x55a3d4864e60 2024-04-16T05:40:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d4885d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d4885d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49894/0 (socket says 172.21.15.32:49894) 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2567357960 learned_addr learned my addr 172.21.15.32:0/2567357960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d487a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf03fff640 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d485a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2567357960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 msgr2=0x55a3d487a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d487a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2567357960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 msgr2=0x55a3d485a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d485a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2567357960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d485ae60 con 0x55a3d486e360 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf03fff640 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d485a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d487a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.271+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d4885d20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbf04018bc0 tx=0x7fbf04008f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29dd8123f31d6ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf00ff9640 1 -- 172.21.15.32:0/2567357960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04039070 con 0x55a3d486e360 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf00ff9640 1 -- 172.21.15.32:0/2567357960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf04028070 con 0x55a3d486e360 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf00ff9640 1 -- 172.21.15.32:0/2567357960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400de70 con 0x55a3d486e360 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 -- 172.21.15.32:0/2567357960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 msgr2=0x55a3d4885d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d4885d20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbf04018bc0 tx=0x7fbf04008f60 comp rx=0 tx=0).stop 2024-04-16T05:40:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 -- 172.21.15.32:0/2567357960 shutdown_connections 2024-04-16T05:40:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d4885d20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d485a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 --2- 172.21.15.32:0/2567357960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d487a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 -- 172.21.15.32:0/2567357960 >> 172.21.15.32:0/2567357960 conn(0x55a3d485f590 msgr2=0x55a3d486a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 -- 172.21.15.32:0/2567357960 shutdown_connections 2024-04-16T05:40:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.272+0000 7fbf14209740 1 -- 172.21.15.32:0/2567357960 wait complete. 2024-04-16T05:40:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 1 -- start start 2024-04-16T05:40:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 1 Processor -- start 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d4876a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d4871d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d48724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1ad9e0 con 0x55a3d486e360 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1aee50 con 0x55a3d4865780 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1ae0b0 con 0x55a3d4864e60 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf14209740 10 client.0 _ll_register_callbacks cb 0x7fbf04008bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d48724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d48724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49904/0 (socket says 172.21.15.32:49904) 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2671737421 learned_addr learned my addr 172.21.15.32:0/2671737421 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf03fff640 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d4871d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d4876a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2671737421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 msgr2=0x55a3d4876a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d4876a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2671737421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 msgr2=0x55a3d4871d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d4871d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.273+0000 7fbf0b7fe640 1 -- 172.21.15.32:0/2671737421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d48e3590 con 0x55a3d486e360 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf0b7fe640 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d48724d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbf04021950 tx=0x7fbf040213c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d52c05712738bd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbef27fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbef27fc640 1 -- 172.21.15.32:0/2671737421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04042030 con 0x55a3d486e360 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbef27fc640 1 -- 172.21.15.32:0/2671737421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fbf04021b50 con 0x55a3d486e360 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 1 -- 172.21.15.32:0/2671737421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf04028070 con 0x55a3d486e360 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbef27fc640 1 -- 172.21.15.32:0/2671737421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf0404f050 con 0x55a3d486e360 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 10 client.16899 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 1 -- 172.21.15.32:0/2671737421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a3d48e4640 con 0x55a3d486e360 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbef27fc640 1 -- 172.21.15.32:0/2671737421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf04028070 con 0x55a3d486e360 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbef27fc640 1 -- 172.21.15.32:0/2671737421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbf0403e450 con 0x55a3d486e360 2024-04-16T05:40:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 10 client.16899 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 20 client.16899 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a3d48e4bf0 0x55a3d48e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 4 client.16899 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.274+0000 7fbf14209740 1 -- 172.21.15.32:0/2671737421 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3d47d1970 con 0x55a3d48e4bf0 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a3d48e4bf0 0x55a3d48e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbef27fc640 10 client.16899 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf0affd640 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a3d48e4bf0 0x55a3d48e6fe0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e43afdae2905430 server_cookie=7035fc7c0fd0f03 in_seq=0 out_seq=0 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbef27fc640 1 -- 172.21.15.32:0/2671737421 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3d47d1970 con 0x55a3d48e4bf0 2024-04-16T05:40:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbef27fc640 10 client.16899 handle_command_reply: tid=0 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 client.16899 shutdown 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 -- 172.21.15.32:0/2671737421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 msgr2=0x55a3d48724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d48724d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbf04021950 tx=0x7fbf040213c0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 -- 172.21.15.32:0/2671737421 shutdown_connections 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3d486e360 0x55a3d48724d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3d4865780 0x55a3d4871d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55a3d48e4bf0 0x55a3d48e6fe0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 --2- 172.21.15.32:0/2671737421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3d4864e60 0x55a3d4876a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 -- 172.21.15.32:0/2671737421 >> 172.21.15.32:0/2671737421 conn(0x55a3d485f590 msgr2=0x55a3d485dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbef27fc640 0 client.16899 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:24.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 -- 172.21.15.32:0/2671737421 shutdown_connections 2024-04-16T05:40:24.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 1 -- 172.21.15.32:0/2671737421 wait complete. 2024-04-16T05:40:24.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.275+0000 7fbf14209740 20 client.16899 trim_cache size 0 max 16384 2024-04-16T05:40:24.284 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:24.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 -- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbef80a3da0 msgr2=0x7fbef80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbef80a3da0 0x7fbef80a6260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbefc02cfd0 tx=0x7fbefc005a20 comp rx=0 tx=0).stop 2024-04-16T05:40:24.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 -- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c107380 msgr2=0x7fbf0c19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c107380 0x7fbf0c19eb00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbef400c970 tx=0x7fbef400ce30 comp rx=0 tx=0).stop 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 -- 172.21.15.32:0/2238547915 shutdown_connections 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c198680 0x7fbf0c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbef80a3da0 0x7fbef80a6260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf0c107380 0x7fbf0c19eb00 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 --2- 172.21.15.32:0/2238547915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf0c1007c0 0x7fbf0c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 -- 172.21.15.32:0/2238547915 >> 172.21.15.32:0/2238547915 conn(0x7fbf0c077410 msgr2=0x7fbf0c105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 -- 172.21.15.32:0/2238547915 shutdown_connections 2024-04-16T05:40:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:24.276+0000 7fbf1161b640 1 -- 172.21.15.32:0/2238547915 wait complete. 2024-04-16T05:40:24.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:24 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4239075025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:24.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:24 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4239075025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:24.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:24 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4239075025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:25.287 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:40:25.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:40:25.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:40:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 -- 172.21.15.32:0/2720127183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768101140 msgr2=0x7f6768106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 --2- 172.21.15.32:0/2720127183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768101140 0x7f6768106c70 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f675c009900 tx=0x7f675c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 -- 172.21.15.32:0/2720127183 shutdown_connections 2024-04-16T05:40:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 --2- 172.21.15.32:0/2720127183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67681073a0 0x7f676810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 --2- 172.21.15.32:0/2720127183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768101140 0x7f6768106c70 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 --2- 172.21.15.32:0/2720127183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6768100790 0x7f6768100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 -- 172.21.15.32:0/2720127183 >> 172.21.15.32:0/2720127183 conn(0x7f6768077490 msgr2=0x7f67680fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 -- 172.21.15.32:0/2720127183 shutdown_connections 2024-04-16T05:40:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.412+0000 7f676e888640 1 -- 172.21.15.32:0/2720127183 wait complete. 2024-04-16T05:40:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.413+0000 7f676e888640 1 Processor -- start 2024-04-16T05:40:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.413+0000 7f676e888640 1 -- start start 2024-04-16T05:40:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.413+0000 7f676e888640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6768100790 0x7f676819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.413+0000 7f676e888640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67681073a0 0x7f676819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.413+0000 7f676e888640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768198760 0x7f6768198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.413+0000 7f676e888640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67681123c0 con 0x7f6768198760 2024-04-16T05:40:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.413+0000 7f676e888640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6768112240 con 0x7f6768100790 2024-04-16T05:40:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.413+0000 7f676e888640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6768112540 con 0x7f67681073a0 2024-04-16T05:40:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768198760 0x7f6768198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768198760 0x7f6768198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49920/0 (socket says 172.21.15.32:49920) 2024-04-16T05:40:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 -- 172.21.15.32:0/3661776406 learned_addr learned my addr 172.21.15.32:0/3661776406 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f6767fff640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6768100790 0x7f676819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f67677fe640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67681073a0 0x7f676819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 -- 172.21.15.32:0/3661776406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67681073a0 msgr2=0x7f676819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67681073a0 0x7f676819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 -- 172.21.15.32:0/3661776406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6768100790 msgr2=0x7f676819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6768100790 0x7f676819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 -- 172.21.15.32:0/3661776406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6768199360 con 0x7f6768198760 2024-04-16T05:40:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f67677fe640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67681073a0 0x7f676819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f6767fff640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6768100790 0x7f676819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676ce25640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768198760 0x7f6768198be0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6760009fd0 tx=0x7f676000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f67657fa640 1 -- 172.21.15.32:0/3661776406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760017070 con 0x7f6768198760 2024-04-16T05:40:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676e888640 1 -- 172.21.15.32:0/3661776406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67681a5440 con 0x7f6768198760 2024-04-16T05:40:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f67657fa640 1 -- 172.21.15.32:0/3661776406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6760003e50 con 0x7f6768198760 2024-04-16T05:40:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f67657fa640 1 -- 172.21.15.32:0/3661776406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67600106c0 con 0x7f6768198760 2024-04-16T05:40:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.414+0000 7f676e888640 1 -- 172.21.15.32:0/3661776406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67681a57f0 con 0x7f6768198760 2024-04-16T05:40:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f67657fa640 1 -- 172.21.15.32:0/3661776406 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6760003970 con 0x7f6768198760 2024-04-16T05:40:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f6771475740 1 Processor -- start 2024-04-16T05:40:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f6771475740 1 -- start start 2024-04-16T05:40:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f67657fa640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67500a3d20 0x7f67500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f6771475740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b60760 0x55d1b3b60b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f6771475740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b6de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f6771475740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f6771475740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d1b3ac5e90 con 0x55d1b3b61080 2024-04-16T05:40:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f6771475740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d1b3a634c0 con 0x55d1b3b60760 2024-04-16T05:40:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f6771475740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d1b3ac1e50 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f67677fe640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67500a3d20 0x7f67500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.417+0000 7f67657fa640 1 -- 172.21.15.32:0/3661776406 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f67600d0910 con 0x7f6768198760 2024-04-16T05:40:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b6de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b6de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49932/0 (socket says 172.21.15.32:49932) 2024-04-16T05:40:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 -- 172.21.15.32:0/2073772157 learned_addr learned my addr 172.21.15.32:0/2073772157 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f676ce25640 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f6767fff640 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b60760 0x55d1b3b60b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 -- 172.21.15.32:0/2073772157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b6e3c0 msgr2=0x55d1b3b707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 -- 172.21.15.32:0/2073772157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b60760 msgr2=0x55d1b3b60b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b60760 0x55d1b3b60b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 -- 172.21.15.32:0/2073772157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b38a07e0 con 0x55d1b3b61080 2024-04-16T05:40:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f676ce25640 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:40:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67500a3d20 0x7f67500a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f675c036040 tx=0x7f675c02c650 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f67677fe640 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b6de80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6768069f90 tx=0x7f675c045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461dcfdb85fcb678 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f674e7fc640 1 -- 172.21.15.32:0/2073772157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c005050 con 0x55d1b3b61080 2024-04-16T05:40:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f674e7fc640 1 -- 172.21.15.32:0/2073772157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c050070 con 0x55d1b3b61080 2024-04-16T05:40:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.418+0000 7f674e7fc640 1 -- 172.21.15.32:0/2073772157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c0035f0 con 0x55d1b3b61080 2024-04-16T05:40:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 -- 172.21.15.32:0/2073772157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 msgr2=0x55d1b3b6de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b6de80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6768069f90 tx=0x7f675c045000 comp rx=0 tx=0).stop 2024-04-16T05:40:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 -- 172.21.15.32:0/2073772157 shutdown_connections 2024-04-16T05:40:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b6de80 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 --2- 172.21.15.32:0/2073772157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b60760 0x55d1b3b60b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 -- 172.21.15.32:0/2073772157 >> 172.21.15.32:0/2073772157 conn(0x55d1b3b5b1e0 msgr2=0x55d1b3b5b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 -- 172.21.15.32:0/2073772157 shutdown_connections 2024-04-16T05:40:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 -- 172.21.15.32:0/2073772157 wait complete. 2024-04-16T05:40:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.419+0000 7f6771475740 1 -- start start 2024-04-16T05:40:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6771475740 1 Processor -- start 2024-04-16T05:40:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6771475740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6771475740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 0x55d1b3b6b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6771475740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6771475740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d1b3b711a0 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6771475740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d1b3a4e410 con 0x55d1b3b61080 2024-04-16T05:40:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6771475740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d1b3b71320 con 0x55d1b3b60760 2024-04-16T05:40:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6771475740 10 client.0 _ll_register_callbacks cb 0x55d1b3a49b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f676ce25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f676ce25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49936/0 (socket says 172.21.15.32:49936) 2024-04-16T05:40:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f676ce25640 1 -- 172.21.15.32:0/1154090982 learned_addr learned my addr 172.21.15.32:0/1154090982 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f67677fe640 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 0x55d1b3b6b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f6767fff640 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f676ce25640 1 -- 172.21.15.32:0/1154090982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 msgr2=0x55d1b3b6b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.420+0000 7f676ce25640 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f676ce25640 1 -- 172.21.15.32:0/1154090982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 msgr2=0x55d1b3b6b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f676ce25640 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 0x55d1b3b6b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f676ce25640 1 -- 172.21.15.32:0/1154090982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b3900f70 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6767fff640 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f676ce25640 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6be90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f67600cdd80 tx=0x7f676009c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7927336e15625b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f67477fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f67477fe640 1 -- 172.21.15.32:0/1154090982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760020040 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f67477fe640 1 -- 172.21.15.32:0/1154090982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f67600882a0 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 1 -- 172.21.15.32:0/1154090982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675c050070 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f67477fe640 1 -- 172.21.15.32:0/1154090982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67600a51e0 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 10 client.16911 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 1 -- 172.21.15.32:0/1154090982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d1b3798170 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f67477fe640 1 -- 172.21.15.32:0/1154090982 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c050070 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f67477fe640 1 -- 172.21.15.32:0/1154090982 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6760098be0 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 10 client.16911 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 20 client.16911 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d1b3b7e730 0x55d1b3b80b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 4 client.16911 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.421+0000 7f6771475740 1 -- 172.21.15.32:0/1154090982 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b3ad38b0 con 0x55d1b3b7e730 2024-04-16T05:40:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.422+0000 7f6767fff640 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d1b3b7e730 0x55d1b3b80b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.422+0000 7f67477fe640 10 client.16911 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.422+0000 7f6767fff640 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d1b3b7e730 0x55d1b3b80b80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b777687e22ffb973 server_cookie=f37b28a1e76870b7 in_seq=0 out_seq=0 2024-04-16T05:40:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f67477fe640 1 -- 172.21.15.32:0/1154090982 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1b3ad38b0 con 0x55d1b3b7e730 2024-04-16T05:40:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f67477fe640 10 client.16911 handle_command_reply: tid=0 2024-04-16T05:40:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 client.16911 shutdown 2024-04-16T05:40:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 -- 172.21.15.32:0/1154090982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 msgr2=0x55d1b3b6be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6be90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f67600cdd80 tx=0x7f676009c450 comp rx=0 tx=0).stop 2024-04-16T05:40:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 -- 172.21.15.32:0/1154090982 shutdown_connections 2024-04-16T05:40:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6be90 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 0x55d1b3b6b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d1b3b7e730 0x55d1b3b80b80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 --2- 172.21.15.32:0/1154090982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 -- 172.21.15.32:0/1154090982 >> 172.21.15.32:0/1154090982 conn(0x55d1b3b5b1e0 msgr2=0x55d1b3b5cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f67477fe640 0 client.16911 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 -- 172.21.15.32:0/1154090982 shutdown_connections 2024-04-16T05:40:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 1 -- 172.21.15.32:0/1154090982 wait complete. 2024-04-16T05:40:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.423+0000 7f6771475740 20 client.16911 trim_cache size 0 max 16384 2024-04-16T05:40:25.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6771475740 1 Processor -- start 2024-04-16T05:40:25.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6771475740 1 -- start start 2024-04-16T05:40:25.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6771475740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6771475740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 0x55d1b3b7a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6771475740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6771475740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67681afa00 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6771475740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f67681ad2d0 con 0x55d1b3b61080 2024-04-16T05:40:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6771475740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67681ab120 con 0x55d1b3b60760 2024-04-16T05:40:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f676ce25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f676ce25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49942/0 (socket says 172.21.15.32:49942) 2024-04-16T05:40:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f676ce25640 1 -- 172.21.15.32:0/3689415095 learned_addr learned my addr 172.21.15.32:0/3689415095 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f67677fe640 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 0x55d1b3b7a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.441+0000 7f6767fff640 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f676ce25640 1 -- 172.21.15.32:0/3689415095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 msgr2=0x55d1b3b6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f676ce25640 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f676ce25640 1 -- 172.21.15.32:0/3689415095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 msgr2=0x55d1b3b7a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f676ce25640 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 0x55d1b3b7a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f676ce25640 1 -- 172.21.15.32:0/3689415095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b3b5fba0 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f676ce25640 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6a440 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f676000b760 tx=0x7f67600a0aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7153b320977ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f674dffb640 1 -- 172.21.15.32:0/3689415095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760020070 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f674dffb640 1 -- 172.21.15.32:0/3689415095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676007f0d0 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f674dffb640 1 -- 172.21.15.32:0/3689415095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760048280 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f6771475740 1 -- 172.21.15.32:0/3689415095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 msgr2=0x55d1b3b6a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f6771475740 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6a440 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f676000b760 tx=0x7f67600a0aa0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f6771475740 1 -- 172.21.15.32:0/3689415095 shutdown_connections 2024-04-16T05:40:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.442+0000 7f6771475740 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b6a440 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b61080 0x55d1b3b7a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 --2- 172.21.15.32:0/3689415095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 -- 172.21.15.32:0/3689415095 >> 172.21.15.32:0/3689415095 conn(0x55d1b3b5b1e0 msgr2=0x55d1b3b5a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 -- 172.21.15.32:0/3689415095 shutdown_connections 2024-04-16T05:40:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 -- 172.21.15.32:0/3689415095 wait complete. 2024-04-16T05:40:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 -- start start 2024-04-16T05:40:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 Processor -- start 2024-04-16T05:40:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b76600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b76b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67681ab790 con 0x55d1b3b61080 2024-04-16T05:40:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f67681acc00 con 0x55d1b3b6e3c0 2024-04-16T05:40:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67681abe60 con 0x55d1b3b60760 2024-04-16T05:40:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f6771475740 10 client.0 _ll_register_callbacks cb 0x7f676009bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f67677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b76b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f67677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b76b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49946/0 (socket says 172.21.15.32:49946) 2024-04-16T05:40:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.443+0000 7f67677fe640 1 -- 172.21.15.32:0/648278408 learned_addr learned my addr 172.21.15.32:0/648278408 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f6767fff640 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b76600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f676ce25640 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67677fe640 1 -- 172.21.15.32:0/648278408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 msgr2=0x55d1b3b76600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67677fe640 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b76600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67677fe640 1 -- 172.21.15.32:0/648278408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b6e3c0 msgr2=0x55d1b3b735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67677fe640 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67677fe640 1 -- 172.21.15.32:0/648278408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b3b73e00 con 0x55d1b3b61080 2024-04-16T05:40:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67677fe640 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b76b40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f675c004fd0 tx=0x7f675c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad8cf9674e9d872 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67477fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67477fe640 1 -- 172.21.15.32:0/648278408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c0048c0 con 0x55d1b3b61080 2024-04-16T05:40:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67477fe640 1 -- 172.21.15.32:0/648278408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f675c004a60 con 0x55d1b3b61080 2024-04-16T05:40:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f6771475740 1 -- 172.21.15.32:0/648278408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676007f0d0 con 0x55d1b3b61080 2024-04-16T05:40:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f67477fe640 1 -- 172.21.15.32:0/648278408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f675c066050 con 0x55d1b3b61080 2024-04-16T05:40:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f6771475740 10 client.16917 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.444+0000 7f6771475740 1 -- 172.21.15.32:0/648278408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d1b3be57a0 con 0x55d1b3b61080 2024-04-16T05:40:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f67477fe640 1 -- 172.21.15.32:0/648278408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676007f0d0 con 0x55d1b3b61080 2024-04-16T05:40:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f67477fe640 1 -- 172.21.15.32:0/648278408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f675c04a070 con 0x55d1b3b61080 2024-04-16T05:40:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f6771475740 10 client.16917 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f6771475740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f6771475740 20 client.16917 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f6771475740 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d1b3be5ad0 0x55d1b3be7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f6771475740 4 client.16917 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:40:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f6771475740 1 -- 172.21.15.32:0/648278408 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d1b3acd720 con 0x55d1b3be5ad0 2024-04-16T05:40:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f6767fff640 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d1b3be5ad0 0x55d1b3be7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f6767fff640 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d1b3be5ad0 0x55d1b3be7ef0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6583b8fcbfbe5edc server_cookie=fa6c415148bb6911 in_seq=0 out_seq=0 2024-04-16T05:40:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.445+0000 7f67477fe640 10 client.16917 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f67477fe640 1 -- 172.21.15.32:0/648278408 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d1b3acd720 con 0x55d1b3be5ad0 2024-04-16T05:40:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f67477fe640 10 client.16917 handle_command_reply: tid=0 2024-04-16T05:40:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f6771475740 1 client.16917 shutdown 2024-04-16T05:40:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f6771475740 1 -- 172.21.15.32:0/648278408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 msgr2=0x55d1b3b76b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f6771475740 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b76b40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f675c004fd0 tx=0x7f675c000bc0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f6771475740 1 -- 172.21.15.32:0/648278408 shutdown_connections 2024-04-16T05:40:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f6771475740 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1b3b6e3c0 0x55d1b3b735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f6771475740 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d1b3b61080 0x55d1b3b76b40 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.446+0000 7f6771475740 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d1b3be5ad0 0x55d1b3be7ef0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f6771475740 1 --2- 172.21.15.32:0/648278408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1b3b60760 0x55d1b3b76600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f6771475740 1 -- 172.21.15.32:0/648278408 >> 172.21.15.32:0/648278408 conn(0x55d1b3b5b1e0 msgr2=0x55d1b3b6bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f67477fe640 0 client.16917 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:25.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f6771475740 1 -- 172.21.15.32:0/648278408 shutdown_connections 2024-04-16T05:40:25.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f6771475740 1 -- 172.21.15.32:0/648278408 wait complete. 2024-04-16T05:40:25.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f6771475740 20 client.16917 trim_cache size 0 max 16384 2024-04-16T05:40:25.458 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:40:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:40:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "16eebcf7-9708-4ef6-a185-4fd1ec9c70d5", 2024-04-16T05:40:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:40:25.459 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:40:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 -- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67500a3d20 msgr2=0x7f67500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67500a3d20 0x7f67500a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f675c036040 tx=0x7f675c02c650 comp rx=0 tx=0).stop 2024-04-16T05:40:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 -- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768198760 msgr2=0x7f6768198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768198760 0x7f6768198be0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6760009fd0 tx=0x7f676000b260 comp rx=0 tx=0).stop 2024-04-16T05:40:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 -- 172.21.15.32:0/3661776406 shutdown_connections 2024-04-16T05:40:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67500a3d20 0x7f67500a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6768198760 0x7f6768198be0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67681073a0 0x7f676819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 --2- 172.21.15.32:0/3661776406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6768100790 0x7f676819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 -- 172.21.15.32:0/3661776406 >> 172.21.15.32:0/3661776406 conn(0x7f6768077490 msgr2=0x7f6768105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 -- 172.21.15.32:0/3661776406 shutdown_connections 2024-04-16T05:40:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.447+0000 7f676e888640 1 -- 172.21.15.32:0/3661776406 wait complete. 2024-04-16T05:40:25.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:25.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:40:25.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 -- 172.21.15.32:0/3562763093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec06a620 msgr2=0x7f17ec06aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 --2- 172.21.15.32:0/3562763093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec06a620 0x7f17ec06aaa0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009930 tx=0x7f17e002bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 -- 172.21.15.32:0/3562763093 shutdown_connections 2024-04-16T05:40:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 --2- 172.21.15.32:0/3562763093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec06afe0 0x7f17ec10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 --2- 172.21.15.32:0/3562763093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec06a620 0x7f17ec06aaa0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 --2- 172.21.15.32:0/3562763093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17ec107040 0x7f17ec107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 -- 172.21.15.32:0/3562763093 >> 172.21.15.32:0/3562763093 conn(0x7f17ec077ee0 msgr2=0x7f17ec0782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 -- 172.21.15.32:0/3562763093 shutdown_connections 2024-04-16T05:40:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.573+0000 7f17f22f2640 1 -- 172.21.15.32:0/3562763093 wait complete. 2024-04-16T05:40:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f22f2640 1 Processor -- start 2024-04-16T05:40:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f22f2640 1 -- start start 2024-04-16T05:40:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f22f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17ec06a620 0x7f17ec19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f22f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec06afe0 0x7f17ec19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f22f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec107040 0x7f17ec198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f22f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec112410 con 0x7f17ec107040 2024-04-16T05:40:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f22f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec112290 con 0x7f17ec06a620 2024-04-16T05:40:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f22f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec112590 con 0x7f17ec06afe0 2024-04-16T05:40:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f088f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec107040 0x7f17ec198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f088f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec107040 0x7f17ec198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49970/0 (socket says 172.21.15.32:49970) 2024-04-16T05:40:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17f088f640 1 -- 172.21.15.32:0/2699829755 learned_addr learned my addr 172.21.15.32:0/2699829755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec06afe0 0x7f17ec19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.574+0000 7f17ebfff640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17ec06a620 0x7f17ec19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17f088f640 1 -- 172.21.15.32:0/2699829755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec06afe0 msgr2=0x7f17ec19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17f088f640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec06afe0 0x7f17ec19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17f088f640 1 -- 172.21.15.32:0/2699829755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17ec06a620 msgr2=0x7f17ec19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17f088f640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17ec06a620 0x7f17ec19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17f088f640 1 -- 172.21.15.32:0/2699829755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec198f30 con 0x7f17ec107040 2024-04-16T05:40:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17f088f640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec107040 0x7f17ec198670 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f17e400cad0 tx=0x7f17e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17e97fa640 1 -- 172.21.15.32:0/2699829755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e400ae20 con 0x7f17ec107040 2024-04-16T05:40:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17f22f2640 1 -- 172.21.15.32:0/2699829755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ec199220 con 0x7f17ec107040 2024-04-16T05:40:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17e97fa640 1 -- 172.21.15.32:0/2699829755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4005020 con 0x7f17ec107040 2024-04-16T05:40:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17e97fa640 1 -- 172.21.15.32:0/2699829755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e400f6f0 con 0x7f17ec107040 2024-04-16T05:40:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.575+0000 7f17f22f2640 1 -- 172.21.15.32:0/2699829755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ec1a5530 con 0x7f17ec107040 2024-04-16T05:40:25.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.577+0000 7f17e97fa640 1 -- 172.21.15.32:0/2699829755 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f17e40051d0 con 0x7f17ec107040 2024-04-16T05:40:25.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f4fba740 1 Processor -- start 2024-04-16T05:40:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f4fba740 1 -- start start 2024-04-16T05:40:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563277e50 0x555563278230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x555563280e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 0x555563283740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f4fba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5555631dd180 con 0x555563278770 2024-04-16T05:40:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f4fba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55556317a360 con 0x555563281350 2024-04-16T05:40:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f4fba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5555631d9140 con 0x555563277e50 2024-04-16T05:40:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17e97fa640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17d40a3d20 0x7f17d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x555563280e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17eb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563277e50 0x555563278230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17f088f640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17d40a3d20 0x7f17d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x555563280e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49986/0 (socket says 172.21.15.32:49986) 2024-04-16T05:40:25.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17ebfff640 1 -- 172.21.15.32:0/3290540412 learned_addr learned my addr 172.21.15.32:0/3290540412 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/3290540412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 0x555563283740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17e97fa640 1 -- 172.21.15.32:0/2699829755 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f17e40d0f60 con 0x7f17ec107040 2024-04-16T05:40:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17ebfff640 1 -- 172.21.15.32:0/3290540412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563277e50 msgr2=0x555563278230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17ebfff640 1 --2- 172.21.15.32:0/3290540412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563277e50 0x555563278230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17ebfff640 1 -- 172.21.15.32:0/3290540412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 msgr2=0x555563283740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17ebfff640 1 --2- 172.21.15.32:0/3290540412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 0x555563283740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.578+0000 7f17ebfff640 1 -- 172.21.15.32:0/3290540412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555562fb97d0 con 0x555563278770 2024-04-16T05:40:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.579+0000 7f17f088f640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17d40a3d20 0x7f17d40a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f17e40ce420 tx=0x7f17e406b570 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.579+0000 7f17ebfff640 1 --2- 172.21.15.32:0/3290540412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x555563280e10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00cc60 tx=0x7f17dc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffdc9055d3bb79f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.579+0000 7f17d27fc640 1 -- 172.21.15.32:0/3290540412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc014070 con 0x555563278770 2024-04-16T05:40:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.579+0000 7f17d27fc640 1 -- 172.21.15.32:0/3290540412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17dc0047a0 con 0x555563278770 2024-04-16T05:40:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.579+0000 7f17d27fc640 1 -- 172.21.15.32:0/3290540412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc010850 con 0x555563278770 2024-04-16T05:40:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.579+0000 7f17f4fba740 1 -- 172.21.15.32:0/3290540412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 msgr2=0x555563280e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.579+0000 7f17f4fba740 1 --2- 172.21.15.32:0/3290540412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x555563280e10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00cc60 tx=0x7f17dc00e720 comp rx=0 tx=0).stop 2024-04-16T05:40:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 -- 172.21.15.32:0/3290540412 shutdown_connections 2024-04-16T05:40:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 --2- 172.21.15.32:0/3290540412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 0x555563283740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 --2- 172.21.15.32:0/3290540412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x555563280e10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 --2- 172.21.15.32:0/3290540412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563277e50 0x555563278230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 -- 172.21.15.32:0/3290540412 >> 172.21.15.32:0/3290540412 conn(0x555563272580 msgr2=0x555563272960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 -- 172.21.15.32:0/3290540412 shutdown_connections 2024-04-16T05:40:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 -- 172.21.15.32:0/3290540412 wait complete. 2024-04-16T05:40:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 -- start start 2024-04-16T05:40:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 Processor -- start 2024-04-16T05:40:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x55556327e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x55556327e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563281350 0x55556327ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555563284140 con 0x555563278770 2024-04-16T05:40:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5555631962c0 con 0x555563277e50 2024-04-16T05:40:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5555632842c0 con 0x555563281350 2024-04-16T05:40:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17f4fba740 10 client.0 _ll_register_callbacks cb 0x55556317d090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.580+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x55556327e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x55556327e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49992/0 (socket says 172.21.15.32:49992) 2024-04-16T05:40:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17ebfff640 1 -- 172.21.15.32:0/4017850747 learned_addr learned my addr 172.21.15.32:0/4017850747 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563281350 0x55556327ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x55556327e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17ebfff640 1 -- 172.21.15.32:0/4017850747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563281350 msgr2=0x55556327ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17ebfff640 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563281350 0x55556327ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17ebfff640 1 -- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 msgr2=0x55556327e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17ebfff640 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x55556327e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17ebfff640 1 -- 172.21.15.32:0/4017850747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555562eef3f0 con 0x555563278770 2024-04-16T05:40:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x55556327e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17ebfff640 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x55556327e8f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00cc30 tx=0x7f17dc005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac3ce316f2124b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17cb7fe640 1 -- 172.21.15.32:0/4017850747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc014070 con 0x555563278770 2024-04-16T05:40:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17cb7fe640 1 -- 172.21.15.32:0/4017850747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f17dc004440 con 0x555563278770 2024-04-16T05:40:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17f4fba740 1 -- 172.21.15.32:0/4017850747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17dc0047a0 con 0x555563278770 2024-04-16T05:40:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17cb7fe640 1 -- 172.21.15.32:0/4017850747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f17dc02d050 con 0x555563278770 2024-04-16T05:40:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17f4fba740 10 client.16929 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17f4fba740 1 -- 172.21.15.32:0/4017850747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555562eaf0f0 con 0x555563278770 2024-04-16T05:40:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.581+0000 7f17cb7fe640 1 -- 172.21.15.32:0/4017850747 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17dc0047a0 con 0x555563278770 2024-04-16T05:40:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17cb7fe640 1 -- 172.21.15.32:0/4017850747 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f17dc003a40 con 0x555563278770 2024-04-16T05:40:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17f4fba740 10 client.16929 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17f4fba740 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17f4fba740 20 client.16929 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555563291620 0x555563293a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17f4fba740 4 client.16929 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17f4fba740 1 -- 172.21.15.32:0/4017850747 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555631eaba0 con 0x555563291620 2024-04-16T05:40:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555563291620 0x555563293a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17cb7fe640 10 client.16929 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.582+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555563291620 0x555563293a10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84436fc5bdb2b2cd server_cookie=30c02659eb2bd952 in_seq=0 out_seq=0 2024-04-16T05:40:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17cb7fe640 1 -- 172.21.15.32:0/4017850747 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555631eaba0 con 0x555563291620 2024-04-16T05:40:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17cb7fe640 10 client.16929 handle_command_reply: tid=0 2024-04-16T05:40:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 client.16929 shutdown 2024-04-16T05:40:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 -- 172.21.15.32:0/4017850747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 msgr2=0x55556327e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x55556327e8f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00cc30 tx=0x7f17dc005d60 comp rx=0 tx=0).stop 2024-04-16T05:40:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 -- 172.21.15.32:0/4017850747 shutdown_connections 2024-04-16T05:40:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x555563291620 0x555563293a10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563281350 0x55556327ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563278770 0x55556327e8f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4017850747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x55556327e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 -- 172.21.15.32:0/4017850747 >> 172.21.15.32:0/4017850747 conn(0x555563272580 msgr2=0x5555632742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17cb7fe640 0 client.16929 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.583+0000 7f17f4fba740 1 -- 172.21.15.32:0/4017850747 shutdown_connections 2024-04-16T05:40:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.584+0000 7f17f4fba740 1 -- 172.21.15.32:0/4017850747 wait complete. 2024-04-16T05:40:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.584+0000 7f17f4fba740 20 client.16929 trim_cache size 0 max 16384 2024-04-16T05:40:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17f4fba740 1 Processor -- start 2024-04-16T05:40:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17f4fba740 1 -- start start 2024-04-16T05:40:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563277e50 0x555563280ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 0x55556328d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 0x55556327ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17f4fba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1b1c10 con 0x555563277e50 2024-04-16T05:40:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17f4fba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1af4e0 con 0x555563281350 2024-04-16T05:40:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17f4fba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1ad330 con 0x555563278770 2024-04-16T05:40:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563277e50 0x555563280ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563277e50 0x555563280ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49998/0 (socket says 172.21.15.32:49998) 2024-04-16T05:40:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.601+0000 7f17eb7fe640 1 -- 172.21.15.32:0/4194402443 learned_addr learned my addr 172.21.15.32:0/4194402443 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17ebfff640 1 --2- 172.21.15.32:0/4194402443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 0x55556328d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17eb7fe640 1 -- 172.21.15.32:0/4194402443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 msgr2=0x55556328d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/4194402443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 0x55556328d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17eb7fe640 1 -- 172.21.15.32:0/4194402443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 msgr2=0x55556327ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/4194402443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 0x55556327ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17eb7fe640 1 -- 172.21.15.32:0/4194402443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555563284020 con 0x555563277e50 2024-04-16T05:40:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/4194402443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563277e50 0x555563280ae0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009930 tx=0x7f17e002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98c54af71ea1b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17d1ffb640 1 -- 172.21.15.32:0/4194402443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0050070 con 0x555563277e50 2024-04-16T05:40:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17d1ffb640 1 -- 172.21.15.32:0/4194402443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e004bdd0 con 0x555563277e50 2024-04-16T05:40:25.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17d1ffb640 1 -- 172.21.15.32:0/4194402443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e004b340 con 0x555563277e50 2024-04-16T05:40:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.602+0000 7f17f4fba740 1 -- 172.21.15.32:0/4194402443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563277e50 msgr2=0x555563280ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4194402443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563277e50 0x555563280ae0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009930 tx=0x7f17e002bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 -- 172.21.15.32:0/4194402443 shutdown_connections 2024-04-16T05:40:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4194402443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563281350 0x55556327ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4194402443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 0x55556328d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 --2- 172.21.15.32:0/4194402443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563277e50 0x555563280ae0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 -- 172.21.15.32:0/4194402443 >> 172.21.15.32:0/4194402443 conn(0x555563272580 msgr2=0x55556327e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 -- 172.21.15.32:0/4194402443 shutdown_connections 2024-04-16T05:40:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 -- 172.21.15.32:0/4194402443 wait complete. 2024-04-16T05:40:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 -- start start 2024-04-16T05:40:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 Processor -- start 2024-04-16T05:40:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x555563289840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 0x555563286bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563281350 0x555563287110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1ad9a0 con 0x555563281350 2024-04-16T05:40:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1aee10 con 0x555563277e50 2024-04-16T05:40:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1ae070 con 0x555563278770 2024-04-16T05:40:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.603+0000 7f17f4fba740 10 client.0 _ll_register_callbacks cb 0x7f17e002c900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563281350 0x555563287110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563281350 0x555563287110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50010/0 (socket says 172.21.15.32:50010) 2024-04-16T05:40:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 -- 172.21.15.32:0/2821378863 learned_addr learned my addr 172.21.15.32:0/2821378863 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17ebfff640 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 0x555563286bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x555563289840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 -- 172.21.15.32:0/2821378863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 msgr2=0x555563286bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 0x555563286bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 -- 172.21.15.32:0/2821378863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 msgr2=0x555563289840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x555563289840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 -- 172.21.15.32:0/2821378863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555563287850 con 0x555563281350 2024-04-16T05:40:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17eb7fe640 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563281350 0x555563287110 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f17e00049b0 tx=0x7f17e00048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49df89813dfc14a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17cb7fe640 1 -- 172.21.15.32:0/2821378863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0050040 con 0x555563281350 2024-04-16T05:40:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17cb7fe640 1 -- 172.21.15.32:0/2821378863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f17e0004040 con 0x555563281350 2024-04-16T05:40:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17f4fba740 1 -- 172.21.15.32:0/2821378863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e004bdd0 con 0x555563281350 2024-04-16T05:40:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17cb7fe640 1 -- 172.21.15.32:0/2821378863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f17e0004c50 con 0x555563281350 2024-04-16T05:40:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17f4fba740 10 client.16935 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.604+0000 7f17f4fba740 1 -- 172.21.15.32:0/2821378863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555632f7870 con 0x555563281350 2024-04-16T05:40:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17cb7fe640 1 -- 172.21.15.32:0/2821378863 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e004bdd0 con 0x555563281350 2024-04-16T05:40:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17cb7fe640 1 -- 172.21.15.32:0/2821378863 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f17e0056040 con 0x555563281350 2024-04-16T05:40:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17f4fba740 10 client.16935 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17f4fba740 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17f4fba740 20 client.16935 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17f4fba740 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5555632f7f00 0x5555632fa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17f4fba740 4 client.16935 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:40:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17f4fba740 1 -- 172.21.15.32:0/2821378863 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555631e4a10 con 0x5555632f7f00 2024-04-16T05:40:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17ebfff640 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5555632f7f00 0x5555632fa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17cb7fe640 10 client.16935 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.605+0000 7f17ebfff640 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5555632f7f00 0x5555632fa320 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecbcb577ff6ca1a8 server_cookie=7a628b05cda20784 in_seq=0 out_seq=0 2024-04-16T05:40:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17cb7fe640 1 -- 172.21.15.32:0/2821378863 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5555631e4a10 con 0x5555632f7f00 2024-04-16T05:40:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17cb7fe640 10 client.16935 handle_command_reply: tid=0 2024-04-16T05:40:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 client.16935 shutdown 2024-04-16T05:40:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 -- 172.21.15.32:0/2821378863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563281350 msgr2=0x555563287110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563281350 0x555563287110 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f17e00049b0 tx=0x7f17e00048b0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 -- 172.21.15.32:0/2821378863 shutdown_connections 2024-04-16T05:40:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555563281350 0x555563287110 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5555632f7f00 0x5555632fa320 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555563278770 0x555563286bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 --2- 172.21.15.32:0/2821378863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555563277e50 0x555563289840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17cb7fe640 0 client.16935 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:25.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 -- 172.21.15.32:0/2821378863 >> 172.21.15.32:0/2821378863 conn(0x555563272580 msgr2=0x555563271760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 -- 172.21.15.32:0/2821378863 shutdown_connections 2024-04-16T05:40:25.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 1 -- 172.21.15.32:0/2821378863 wait complete. 2024-04-16T05:40:25.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f4fba740 20 client.16935 trim_cache size 0 max 16384 2024-04-16T05:40:25.616 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:40:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:40:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:40:25.617 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:40:25.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f22f2640 1 -- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17d40a3d20 msgr2=0x7f17d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f22f2640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17d40a3d20 0x7f17d40a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f17e40ce420 tx=0x7f17e406b570 comp rx=0 tx=0).stop 2024-04-16T05:40:25.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f22f2640 1 -- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec107040 msgr2=0x7f17ec198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.606+0000 7f17f22f2640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec107040 0x7f17ec198670 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f17e400cad0 tx=0x7f17e400cf90 comp rx=0 tx=0).stop 2024-04-16T05:40:25.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.607+0000 7f17f22f2640 1 -- 172.21.15.32:0/2699829755 shutdown_connections 2024-04-16T05:40:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.607+0000 7f17f22f2640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f17d40a3d20 0x7f17d40a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.607+0000 7f17f22f2640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ec107040 0x7f17ec198670 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.607+0000 7f17f22f2640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec06afe0 0x7f17ec19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.607+0000 7f17f22f2640 1 --2- 172.21.15.32:0/2699829755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17ec06a620 0x7f17ec19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.607+0000 7f17f22f2640 1 -- 172.21.15.32:0/2699829755 >> 172.21.15.32:0/2699829755 conn(0x7f17ec077ee0 msgr2=0x7f17ec0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.607+0000 7f17f22f2640 1 -- 172.21.15.32:0/2699829755 shutdown_connections 2024-04-16T05:40:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.607+0000 7f17f22f2640 1 -- 172.21.15.32:0/2699829755 wait complete. 2024-04-16T05:40:25.620 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:25 smithi032 ceph-mon[32351]: pgmap v167: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:25.620 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:40:25.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:40:25.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.728+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2093645353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd410c630 msgr2=0x7f2cd410ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.728+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2093645353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd410c630 0x7f2cd410ea60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009ca0 tx=0x7f2ccc02c180 comp rx=0 tx=0).stop 2024-04-16T05:40:25.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.728+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2093645353 shutdown_connections 2024-04-16T05:40:25.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.728+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2093645353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd410c630 0x7f2cd410ea60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.728+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2093645353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4103b80 0x7f2cd410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.728+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2093645353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cd4103260 0x7f2cd4103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.728+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2093645353 >> 172.21.15.32:0/2093645353 conn(0x7f2cd4077280 msgr2=0x7f2cd4077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2093645353 shutdown_connections 2024-04-16T05:40:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2093645353 wait complete. 2024-04-16T05:40:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 Processor -- start 2024-04-16T05:40:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 -- start start 2024-04-16T05:40:25.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cd4103260 0x7f2cd419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd4103b80 0x7f2cd419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd410c630 0x7f2cd4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41123a0 con 0x7f2cd4103b80 2024-04-16T05:40:25.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4112220 con 0x7f2cd410c630 2024-04-16T05:40:25.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cdc5bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4112520 con 0x7f2cd4103260 2024-04-16T05:40:25.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.729+0000 7f2cd9b58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd4103b80 0x7f2cd419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cd9b58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd4103b80 0x7f2cd419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50028/0 (socket says 172.21.15.32:50028) 2024-04-16T05:40:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cd9b58640 1 -- 172.21.15.32:0/2157122486 learned_addr learned my addr 172.21.15.32:0/2157122486 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cda359640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cd4103260 0x7f2cd419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cd9b58640 1 -- 172.21.15.32:0/2157122486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cd4103260 msgr2=0x7f2cd419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cd9b58640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cd4103260 0x7f2cd419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cd9b58640 1 -- 172.21.15.32:0/2157122486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd410c630 msgr2=0x7f2cd4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cdab5a640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd410c630 0x7f2cd4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cd9b58640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd410c630 0x7f2cd4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cd9b58640 1 -- 172.21.15.32:0/2157122486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd4199000 con 0x7f2cd4103b80 2024-04-16T05:40:25.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cdab5a640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd410c630 0x7f2cd4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:25.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cd9b58640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd4103b80 0x7f2cd419ebc0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400d950 tx=0x7f2cc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2ccb7fe640 1 -- 172.21.15.32:0/2157122486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4017070 con 0x7f2cd4103b80 2024-04-16T05:40:25.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2ccb7fe640 1 -- 172.21.15.32:0/2157122486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc4009c30 con 0x7f2cd4103b80 2024-04-16T05:40:25.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd41992f0 con 0x7f2cd4103b80 2024-04-16T05:40:25.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2ccb7fe640 1 -- 172.21.15.32:0/2157122486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400f6b0 con 0x7f2cd4103b80 2024-04-16T05:40:25.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.730+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd41a5600 con 0x7f2cd4103b80 2024-04-16T05:40:25.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.731+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd406aea0 con 0x7f2cd4103b80 2024-04-16T05:40:25.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.732+0000 7f2ccb7fe640 1 -- 172.21.15.32:0/2157122486 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2cc40109d0 con 0x7f2cd4103b80 2024-04-16T05:40:25.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.733+0000 7f2ccb7fe640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2cc00a3d20 0x7f2cc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:25.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.733+0000 7f2cda359640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2cc00a3d20 0x7f2cc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:25.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.733+0000 7f2ccb7fe640 1 -- 172.21.15.32:0/2157122486 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2cc40d0800 con 0x7f2cd4103b80 2024-04-16T05:40:25.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.734+0000 7f2cda359640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2cc00a3d20 0x7f2cc00a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005fd0 tx=0x7f2cd0005e20 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:25.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.738+0000 7f2ccb7fe640 1 -- 172.21.15.32:0/2157122486 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cc4009de0 con 0x7f2cd4103b80 2024-04-16T05:40:25.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:25 smithi134 ceph-mon[36153]: pgmap v167: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:25.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:25 smithi082 ceph-mon[36195]: pgmap v167: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.967+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cd4107720 con 0x7f2cd4103b80 2024-04-16T05:40:25.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.967+0000 7f2ccb7fe640 1 -- 172.21.15.32:0/2157122486 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f2cc409cb00 con 0x7f2cd4103b80 2024-04-16T05:40:25.966 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:40:25.966 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:40:25.967 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:40:25.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2cc00a3d20 msgr2=0x7f2cc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2cc00a3d20 0x7f2cc00a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005fd0 tx=0x7f2cd0005e20 comp rx=0 tx=0).stop 2024-04-16T05:40:25.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd4103b80 msgr2=0x7f2cd419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd4103b80 0x7f2cd419ebc0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400d950 tx=0x7f2cc400de10 comp rx=0 tx=0).stop 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 shutdown_connections 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd410c630 0x7f2cd4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2cc00a3d20 0x7f2cc00a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd4103b80 0x7f2cd419ebc0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.969+0000 7f2cdc5bd640 1 --2- 172.21.15.32:0/2157122486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cd4103260 0x7f2cd419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.970+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 >> 172.21.15.32:0/2157122486 conn(0x7f2cd4077280 msgr2=0x7f2cd40ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.970+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 shutdown_connections 2024-04-16T05:40:25.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:25.970+0000 7f2cdc5bd640 1 -- 172.21.15.32:0/2157122486 wait complete. 2024-04-16T05:40:25.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:40:26.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3592800981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8101ae0 msgr2=0x7fdbd8105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3592800981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8101ae0 0x7fdbd8105b20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009900 tx=0x7fdbc002bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3592800981 shutdown_connections 2024-04-16T05:40:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3592800981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd8106200 0x7fdbd810ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3592800981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8101ae0 0x7fdbd8105b20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3592800981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd8101130 0x7fdbd8101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3592800981 >> 172.21.15.32:0/3592800981 conn(0x7fdbd80780b0 msgr2=0x7fdbd80784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3592800981 shutdown_connections 2024-04-16T05:40:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.088+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3592800981 wait complete. 2024-04-16T05:40:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.089+0000 7fdbdd0f8640 1 Processor -- start 2024-04-16T05:40:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.089+0000 7fdbdd0f8640 1 -- start start 2024-04-16T05:40:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.089+0000 7fdbdd0f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd8101130 0x7fdbd810e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbdd0f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8106200 0x7fdbd810eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbdd0f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd810f1e0 0x7fdbd81a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbdd0f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8112390 con 0x7fdbd8106200 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbdd0f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8112210 con 0x7fdbd8101130 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbdd0f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8112510 con 0x7fdbd810f1e0 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8106200 0x7fdbd810eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8106200 0x7fdbd810eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50040/0 (socket says 172.21.15.32:50040) 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 -- 172.21.15.32:0/3359352663 learned_addr learned my addr 172.21.15.32:0/3359352663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd8101130 0x7fdbd810e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd759e640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd810f1e0 0x7fdbd81a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 -- 172.21.15.32:0/3359352663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd810f1e0 msgr2=0x7fdbd81a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd810f1e0 0x7fdbd81a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 -- 172.21.15.32:0/3359352663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd8101130 msgr2=0x7fdbd810e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd8101130 0x7fdbd810e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 -- 172.21.15.32:0/3359352663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd81a7040 con 0x7fdbd8106200 2024-04-16T05:40:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd759e640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd810f1e0 0x7fdbd81a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd8101130 0x7fdbd810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbd659c640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8106200 0x7fdbd810eca0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0000c00 tx=0x7fdbc002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbcffff640 1 -- 172.21.15.32:0/3359352663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0041070 con 0x7fdbd8106200 2024-04-16T05:40:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3359352663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd81a72d0 con 0x7fdbd8106200 2024-04-16T05:40:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbcffff640 1 -- 172.21.15.32:0/3359352663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc0004450 con 0x7fdbd8106200 2024-04-16T05:40:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbcffff640 1 -- 172.21.15.32:0/3359352663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0034440 con 0x7fdbd8106200 2024-04-16T05:40:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.090+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3359352663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd81a77c0 con 0x7fdbd8106200 2024-04-16T05:40:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbcffff640 1 -- 172.21.15.32:0/3359352663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdbc0044070 con 0x7fdbd8106200 2024-04-16T05:40:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbdfdb7740 1 Processor -- start 2024-04-16T05:40:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbdfdb7740 1 -- start start 2024-04-16T05:40:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbb9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb9780 0x56084cbc1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbc4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56084cb1e0e0 con 0x56084cbb9780 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56084cabb520 con 0x56084cbc2360 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56084cb1a0a0 con 0x56084cbb8e60 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbcffff640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fdbc40a3d50 0x7fdbc40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbd6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb9780 0x56084cbc1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbd6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb9780 0x56084cbc1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50046/0 (socket says 172.21.15.32:50046) 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.093+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/1068266179 learned_addr learned my addr 172.21.15.32:0/1068266179 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd759e640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fdbc40a3d50 0x7fdbc40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbcffff640 1 -- 172.21.15.32:0/3359352663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fdbc00f2770 con 0x7fdbd8106200 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd759e640 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbb9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd659c640 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbc4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/1068266179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 msgr2=0x56084cbb9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbb9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/1068266179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 msgr2=0x56084cbc4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbc4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/1068266179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084cae38d0 con 0x56084cbb9780 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd759e640 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbb9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:26.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd659c640 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbc4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb9780 0x56084cbc1e20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8007f70 tx=0x7fdbc800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44af5c5996f3cb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbd759e640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fdbc40a3d50 0x7fdbc40a6210 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000dd90 tx=0x7fdbd0005db0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbcd7fa640 1 -- 172.21.15.32:0/1068266179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8014070 con 0x56084cbb9780 2024-04-16T05:40:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbcd7fa640 1 -- 172.21.15.32:0/1068266179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc8004d20 con 0x56084cbb9780 2024-04-16T05:40:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.094+0000 7fdbcd7fa640 1 -- 172.21.15.32:0/1068266179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80054f0 con 0x56084cbb9780 2024-04-16T05:40:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/1068266179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb9780 msgr2=0x56084cbc1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb9780 0x56084cbc1e20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8007f70 tx=0x7fdbc800e7e0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/1068266179 shutdown_connections 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbc4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb9780 0x56084cbc1e20 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/1068266179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbb9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/1068266179 >> 172.21.15.32:0/1068266179 conn(0x56084cbb3590 msgr2=0x56084cbb3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/1068266179 shutdown_connections 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/1068266179 wait complete. 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 -- start start 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.095+0000 7fdbdfdb7740 1 Processor -- start 2024-04-16T05:40:26.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbbf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 0x56084cbbf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbbfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56084cbc5000 con 0x56084cbb8e60 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56084cad74d0 con 0x56084cbc2360 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56084cbc5180 con 0x56084cbb9780 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbdfdb7740 10 client.0 _ll_register_callbacks cb 0x56084cabc2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbbf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbbf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50062/0 (socket says 172.21.15.32:50062) 2024-04-16T05:40:26.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/3474084633 learned_addr learned my addr 172.21.15.32:0/3474084633 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd759e640 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbbfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd659c640 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 0x56084cbbf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/3474084633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 msgr2=0x56084cbbf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 0x56084cbbf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/3474084633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 msgr2=0x56084cbbfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbbfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/3474084633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084c8303f0 con 0x56084cbb8e60 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbbf360 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800ece0 tx=0x7fdbc8007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d51c48fb1652e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/3474084633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8014070 con 0x56084cbb8e60 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/3474084633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fdbc80053d0 con 0x56084cbb8e60 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.096+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/3474084633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc8004d20 con 0x56084cbb8e60 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/3474084633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdbc802f050 con 0x56084cbb8e60 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbdfdb7740 10 client.16953 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/3474084633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56084c8fa7d0 con 0x56084cbb8e60 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/3474084633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc8004d20 con 0x56084cbb8e60 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/3474084633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdbc8018a80 con 0x56084cbb8e60 2024-04-16T05:40:26.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbdfdb7740 10 client.16953 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbdfdb7740 10 client.16953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbdfdb7740 20 client.16953 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56084cbd2230 0x56084cbd4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbdfdb7740 4 client.16953 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/3474084633 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084cb2bb00 con 0x56084cbd2230 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.097+0000 7fdbd659c640 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56084cbd2230 0x56084cbd4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbbe7fc640 10 client.16953 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbd659c640 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56084cbd2230 0x56084cbd4620 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6470e5984e20d535 server_cookie=f943029857b69b66 in_seq=0 out_seq=0 2024-04-16T05:40:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/3474084633 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56084cb2bb00 con 0x56084cbd2230 2024-04-16T05:40:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbbe7fc640 10 client.16953 handle_command_reply: tid=0 2024-04-16T05:40:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 client.16953 shutdown 2024-04-16T05:40:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/3474084633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 msgr2=0x56084cbbf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbbf360 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800ece0 tx=0x7fdbc8007cb0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/3474084633 shutdown_connections 2024-04-16T05:40:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56084cbd2230 0x56084cbd4620 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbbfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbbe7fc640 0 client.16953 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 0x56084cbbf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/3474084633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbbf360 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.098+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/3474084633 >> 172.21.15.32:0/3474084633 conn(0x56084cbb3590 msgr2=0x56084cbb52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.099+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/3474084633 shutdown_connections 2024-04-16T05:40:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.099+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/3474084633 wait complete. 2024-04-16T05:40:26.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.099+0000 7fdbdfdb7740 20 client.16953 trim_cache size 0 max 16384 2024-04-16T05:40:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.112+0000 7fdbdfdb7740 1 Processor -- start 2024-04-16T05:40:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.112+0000 7fdbdfdb7740 1 -- start start 2024-04-16T05:40:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.112+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbc1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.112+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 0x56084cbce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.112+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbb25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.112+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81b3d40 con 0x56084cbb8e60 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.112+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81b1610 con 0x56084cbc2360 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.112+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81af460 con 0x56084cbb9780 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbc1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbc1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50074/0 (socket says 172.21.15.32:50074) 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/2398092706 learned_addr learned my addr 172.21.15.32:0/2398092706 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd659c640 1 --2- 172.21.15.32:0/2398092706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 0x56084cbce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/2398092706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 msgr2=0x56084cbce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/2398092706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 0x56084cbce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/2398092706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 msgr2=0x56084cbb25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/2398092706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbb25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 -- 172.21.15.32:0/2398092706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084cbaee60 con 0x56084cbb8e60 2024-04-16T05:40:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/2398092706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbc1a90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005c00 tx=0x7fdbc800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcbb41aba0cf58e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbccff9640 1 -- 172.21.15.32:0/2398092706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8014040 con 0x56084cbb8e60 2024-04-16T05:40:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbccff9640 1 -- 172.21.15.32:0/2398092706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc8026070 con 0x56084cbb8e60 2024-04-16T05:40:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.113+0000 7fdbccff9640 1 -- 172.21.15.32:0/2398092706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8021c20 con 0x56084cbb8e60 2024-04-16T05:40:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2398092706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 msgr2=0x56084cbc1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2398092706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbc1a90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005c00 tx=0x7fdbc800e7e0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2398092706 shutdown_connections 2024-04-16T05:40:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2398092706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbc2360 0x56084cbb25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2398092706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb9780 0x56084cbce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2398092706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbb8e60 0x56084cbc1a90 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2398092706 >> 172.21.15.32:0/2398092706 conn(0x56084cbb3590 msgr2=0x56084cbb1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2398092706 shutdown_connections 2024-04-16T05:40:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2398092706 wait complete. 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 -- start start 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.114+0000 7fdbdfdb7740 1 Processor -- start 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbb9780 0x56084cbca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbdfdb7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbc2360 0x56084cbc5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81afad0 con 0x56084cbc2360 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81b0f40 con 0x56084cbb9780 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbdfdb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81b01a0 con 0x56084cbb8e60 2024-04-16T05:40:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbdfdb7740 10 client.0 _ll_register_callbacks cb 0x7fdbc8007dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbc2360 0x56084cbc5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbc2360 0x56084cbc5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50084/0 (socket says 172.21.15.32:50084) 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 -- 172.21.15.32:0/2560261331 learned_addr learned my addr 172.21.15.32:0/2560261331 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd659c640 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbb9780 0x56084cbca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 -- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 msgr2=0x56084cbca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 -- 172.21.15.32:0/2560261331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbb9780 msgr2=0x56084cbca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbb9780 0x56084cbca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 -- 172.21.15.32:0/2560261331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084cbc6170 con 0x56084cbc2360 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbd759e640 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbc2360 0x56084cbc5940 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0012780 tx=0x7fdbd000d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40564915405827a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/2560261331 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd002a060 con 0x56084cbc2360 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/2560261331 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fdbd001c8c0 con 0x56084cbc2360 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.115+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2560261331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc8026070 con 0x56084cbc2360 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/2560261331 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdbd0031050 con 0x56084cbc2360 2024-04-16T05:40:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbdfdb7740 10 client.16959 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2560261331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56084cc380b0 con 0x56084cbc2360 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/2560261331 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc8026070 con 0x56084cbc2360 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/2560261331 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdbd001a3c0 con 0x56084cbc2360 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbdfdb7740 10 client.16959 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbdfdb7740 10 client.16959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbdfdb7740 20 client.16959 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56084cc38860 0x56084cc3ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbdfdb7740 4 client.16959 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2560261331 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56084cb25970 con 0x56084cc38860 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.116+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56084cc38860 0x56084cc3ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbd6d9d640 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56084cc38860 0x56084cc3ac80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39d6bb859a08855d server_cookie=2ad9504b16e845ee in_seq=0 out_seq=0 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbbe7fc640 10 client.16959 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbbe7fc640 1 -- 172.21.15.32:0/2560261331 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56084cb25970 con 0x56084cc38860 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbbe7fc640 10 client.16959 handle_command_reply: tid=0 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 client.16959 shutdown 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2560261331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbc2360 msgr2=0x56084cbc5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbc2360 0x56084cbc5940 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0012780 tx=0x7fdbd000d5b0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2560261331 shutdown_connections 2024-04-16T05:40:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x56084cc38860 0x56084cc3ac80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56084cbc2360 0x56084cbc5940 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56084cbb9780 0x56084cbca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbbe7fc640 0 client.16959 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 --2- 172.21.15.32:0/2560261331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56084cbb8e60 0x56084cbca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2560261331 >> 172.21.15.32:0/2560261331 conn(0x56084cbb3590 msgr2=0x56084cbb2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2560261331 shutdown_connections 2024-04-16T05:40:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 1 -- 172.21.15.32:0/2560261331 wait complete. 2024-04-16T05:40:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.117+0000 7fdbdfdb7740 20 client.16959 trim_cache size 0 max 16384 2024-04-16T05:40:26.126 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fdbc40a3d50 msgr2=0x7fdbc40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fdbc40a3d50 0x7fdbc40a6210 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000dd90 tx=0x7fdbd0005db0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8106200 msgr2=0x7fdbd810eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8106200 0x7fdbd810eca0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0000c00 tx=0x7fdbc002cc70 comp rx=0 tx=0).stop 2024-04-16T05:40:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3359352663 shutdown_connections 2024-04-16T05:40:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd810f1e0 0x7fdbd81a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fdbc40a3d50 0x7fdbc40a6210 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbd8106200 0x7fdbd810eca0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 --2- 172.21.15.32:0/3359352663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd8101130 0x7fdbd810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3359352663 >> 172.21.15.32:0/3359352663 conn(0x7fdbd80780b0 msgr2=0x7fdbd80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3359352663 shutdown_connections 2024-04-16T05:40:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.118+0000 7fdbdd0f8640 1 -- 172.21.15.32:0/3359352663 wait complete. 2024-04-16T05:40:26.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:40:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 -- 172.21.15.32:0/2474309322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768101130 msgr2=0x7f8768101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 --2- 172.21.15.32:0/2474309322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768101130 0x7f8768101510 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8758009900 tx=0x7f875802bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 -- 172.21.15.32:0/2474309322 shutdown_connections 2024-04-16T05:40:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 --2- 172.21.15.32:0/2474309322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768106200 0x7f876810ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 --2- 172.21.15.32:0/2474309322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101ae0 0x7f8768105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 --2- 172.21.15.32:0/2474309322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768101130 0x7f8768101510 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 -- 172.21.15.32:0/2474309322 >> 172.21.15.32:0/2474309322 conn(0x7f87680780b0 msgr2=0x7f87680784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 -- 172.21.15.32:0/2474309322 shutdown_connections 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.237+0000 7f876dcb1640 1 -- 172.21.15.32:0/2474309322 wait complete. 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f876dcb1640 1 Processor -- start 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f876dcb1640 1 -- start start 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f876dcb1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101130 0x7f876819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f876dcb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768106200 0x7f876819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f876dcb1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768198630 0x7f8768198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f876dcb1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87681123d0 con 0x7f8768198630 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f876dcb1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8768112250 con 0x7f8768106200 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f876dcb1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8768112550 con 0x7f8768101130 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f8767fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768198630 0x7f8768198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f8767fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768198630 0x7f8768198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50100/0 (socket says 172.21.15.32:50100) 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f8767fff640 1 -- 172.21.15.32:0/366676320 learned_addr learned my addr 172.21.15.32:0/366676320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f87677fe640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101130 0x7f876819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.238+0000 7f875ffff640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768106200 0x7f876819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f8767fff640 1 -- 172.21.15.32:0/366676320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101130 msgr2=0x7f876819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f8767fff640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101130 0x7f876819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f8767fff640 1 -- 172.21.15.32:0/366676320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768106200 msgr2=0x7f876819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f8767fff640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768106200 0x7f876819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f8767fff640 1 -- 172.21.15.32:0/366676320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8768199230 con 0x7f8768198630 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f87677fe640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101130 0x7f876819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f875ffff640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768106200 0x7f876819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f8767fff640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768198630 0x7f8768198ab0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f876000cce0 tx=0x7f876000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f87657fa640 1 -- 172.21.15.32:0/366676320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876000f450 con 0x7f8768198630 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f87657fa640 1 -- 172.21.15.32:0/366676320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f876000fad0 con 0x7f8768198630 2024-04-16T05:40:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f876dcb1640 1 -- 172.21.15.32:0/366676320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87681a5310 con 0x7f8768198630 2024-04-16T05:40:26.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f87657fa640 1 -- 172.21.15.32:0/366676320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760018c10 con 0x7f8768198630 2024-04-16T05:40:26.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.239+0000 7f876dcb1640 1 -- 172.21.15.32:0/366676320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87681a5710 con 0x7f8768198630 2024-04-16T05:40:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.241+0000 7f87657fa640 1 -- 172.21.15.32:0/366676320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f876000f5f0 con 0x7f8768198630 2024-04-16T05:40:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.241+0000 7f8770971740 1 Processor -- start 2024-04-16T05:40:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.241+0000 7f8770971740 1 -- start start 2024-04-16T05:40:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f8770971740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119df0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f8770971740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119df8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f8770971740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 0x55b119dfb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f8770971740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b119d550e0 con 0x55b119df0780 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f8770971740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b119cf2520 con 0x55b119defe60 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f8770971740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b119d510a0 con 0x55b119df9360 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f87677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119df8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f87677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119df8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50116/0 (socket says 172.21.15.32:50116) 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f87677fe640 1 -- 172.21.15.32:0/3745823902 learned_addr learned my addr 172.21.15.32:0/3745823902 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f875ffff640 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119df0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f87657fa640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f87540a3da0 0x7f87540a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f875ffff640 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 0x55b119dfb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f8767fff640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f87540a3da0 0x7f87540a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f875ffff640 1 -- 172.21.15.32:0/3745823902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 msgr2=0x55b119dfb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f875ffff640 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 0x55b119dfb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f875ffff640 1 -- 172.21.15.32:0/3745823902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 msgr2=0x55b119df8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f875ffff640 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119df8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f875ffff640 1 -- 172.21.15.32:0/3745823902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b119d1a8d0 con 0x55b119defe60 2024-04-16T05:40:26.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f87657fa640 1 -- 172.21.15.32:0/366676320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f87600d11d0 con 0x7f8768198630 2024-04-16T05:40:26.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.242+0000 7f875ffff640 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119df0240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8768199e80 tx=0x7f875000b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e708f91e603a444 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8767fff640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f87540a3da0 0x7f87540a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f876009cad0 tx=0x7f876006f8c0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f875dffb640 1 -- 172.21.15.32:0/3745823902 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750018030 con 0x55b119defe60 2024-04-16T05:40:26.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f875dffb640 1 -- 172.21.15.32:0/3745823902 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8750012930 con 0x55b119defe60 2024-04-16T05:40:26.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f875dffb640 1 -- 172.21.15.32:0/3745823902 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500117d0 con 0x55b119defe60 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 -- 172.21.15.32:0/3745823902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 msgr2=0x55b119df0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119df0240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8768199e80 tx=0x7f875000b5a0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 -- 172.21.15.32:0/3745823902 shutdown_connections 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 0x55b119dfb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119df8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 --2- 172.21.15.32:0/3745823902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119df0240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 -- 172.21.15.32:0/3745823902 >> 172.21.15.32:0/3745823902 conn(0x55b119dea590 msgr2=0x55b119dea970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 -- 172.21.15.32:0/3745823902 shutdown_connections 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.243+0000 7f8770971740 1 -- 172.21.15.32:0/3745823902 wait complete. 2024-04-16T05:40:26.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 1 -- start start 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 1 Processor -- start 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119defe60 0x55b119df6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df0780 0x55b119df6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 0x55b119df6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b119dfc060 con 0x55b119defe60 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b119d0e4d0 con 0x55b119df0780 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b119dfc1e0 con 0x55b119df9360 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f8770971740 10 client.0 _ll_register_callbacks cb 0x55b119cf32c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:26.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f875ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119defe60 0x55b119df6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f875ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119defe60 0x55b119df6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50126/0 (socket says 172.21.15.32:50126) 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.244+0000 7f875ffff640 1 -- 172.21.15.32:0/192737959 learned_addr learned my addr 172.21.15.32:0/192737959 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f87677fe640 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df0780 0x55b119df6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f875ffff640 1 -- 172.21.15.32:0/192737959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 msgr2=0x55b119df6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f875ffff640 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 0x55b119df6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f875ffff640 1 -- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df0780 msgr2=0x55b119df6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f875ffff640 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df0780 0x55b119df6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f875ffff640 1 -- 172.21.15.32:0/192737959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b119a673f0 con 0x55b119defe60 2024-04-16T05:40:26.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f87677fe640 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df0780 0x55b119df6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f875ffff640 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119defe60 0x55b119df6330 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8750005f30 tx=0x7f8750005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc679743785d190d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8746ffd640 1 -- 172.21.15.32:0/192737959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750018040 con 0x55b119defe60 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8746ffd640 1 -- 172.21.15.32:0/192737959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8750005040 con 0x55b119defe60 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8770971740 1 -- 172.21.15.32:0/192737959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8750012930 con 0x55b119defe60 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8746ffd640 1 -- 172.21.15.32:0/192737959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8750030050 con 0x55b119defe60 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8770971740 10 client.16971 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8770971740 1 -- 172.21.15.32:0/192737959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b119b317d0 con 0x55b119defe60 2024-04-16T05:40:26.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8746ffd640 1 -- 172.21.15.32:0/192737959 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8750012930 con 0x55b119defe60 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8746ffd640 1 -- 172.21.15.32:0/192737959 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f875000f040 con 0x55b119defe60 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8770971740 10 client.16971 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8770971740 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8770971740 20 client.16971 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.245+0000 7f8770971740 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b119e091d0 0x55b119e0b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.246+0000 7f8770971740 4 client.16971 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.246+0000 7f8770971740 1 -- 172.21.15.32:0/192737959 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b119d62b00 con 0x55b119e091d0 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.246+0000 7f87677fe640 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b119e091d0 0x55b119e0b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.246+0000 7f8746ffd640 10 client.16971 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:26.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.246+0000 7f87677fe640 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b119e091d0 0x55b119e0b5c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3d84c826bb04f03 server_cookie=1cfbcee8bbab69bc in_seq=0 out_seq=0 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8746ffd640 1 -- 172.21.15.32:0/192737959 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b119d62b00 con 0x55b119e091d0 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8746ffd640 10 client.16971 handle_command_reply: tid=0 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 client.16971 shutdown 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 -- 172.21.15.32:0/192737959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119defe60 msgr2=0x55b119df6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119defe60 0x55b119df6330 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8750005f30 tx=0x7f8750005f60 comp rx=0 tx=0).stop 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 -- 172.21.15.32:0/192737959 shutdown_connections 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b119e091d0 0x55b119e0b5c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df9360 0x55b119df6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df0780 0x55b119df6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8746ffd640 0 client.16971 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:26.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 --2- 172.21.15.32:0/192737959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119defe60 0x55b119df6330 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 -- 172.21.15.32:0/192737959 >> 172.21.15.32:0/192737959 conn(0x55b119dea590 msgr2=0x55b119dec2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 -- 172.21.15.32:0/192737959 shutdown_connections 2024-04-16T05:40:26.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 1 -- 172.21.15.32:0/192737959 wait complete. 2024-04-16T05:40:26.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.247+0000 7f8770971740 20 client.16971 trim_cache size 0 max 16384 2024-04-16T05:40:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.260+0000 7f8770971740 1 Processor -- start 2024-04-16T05:40:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.260+0000 7f8770971740 1 -- start start 2024-04-16T05:40:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f8770971740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119defe60 0x55b119df8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f8770971740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119e05030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f8770971740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df9360 0x55b119e10c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f8770971740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87681b1b90 con 0x55b119df0780 2024-04-16T05:40:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f8770971740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87681af460 con 0x55b119df9360 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f8770971740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87681ad2b0 con 0x55b119defe60 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119e05030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119e05030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50134/0 (socket says 172.21.15.32:50134) 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 -- 172.21.15.32:0/1359824365 learned_addr learned my addr 172.21.15.32:0/1359824365 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f875ffff640 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119defe60 0x55b119df8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f875ffff640 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df9360 0x55b119e10c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 -- 172.21.15.32:0/1359824365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119defe60 msgr2=0x55b119df8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119defe60 0x55b119df8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 -- 172.21.15.32:0/1359824365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df9360 msgr2=0x55b119e10c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df9360 0x55b119e10c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 -- 172.21.15.32:0/1359824365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b119deb160 con 0x55b119df0780 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f87677fe640 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119e05030 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8758009680 tx=0x7f875802ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55296d8ed141108 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.261+0000 7f875d7fa640 1 -- 172.21.15.32:0/1359824365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875804a070 con 0x55b119df0780 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f875d7fa640 1 -- 172.21.15.32:0/1359824365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8758005520 con 0x55b119df0780 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f875d7fa640 1 -- 172.21.15.32:0/1359824365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580346a0 con 0x55b119df0780 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 -- 172.21.15.32:0/1359824365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 msgr2=0x55b119e05030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119e05030 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8758009680 tx=0x7f875802ca30 comp rx=0 tx=0).stop 2024-04-16T05:40:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 -- 172.21.15.32:0/1359824365 shutdown_connections 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119df9360 0x55b119e10c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df0780 0x55b119e05030 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 --2- 172.21.15.32:0/1359824365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119defe60 0x55b119df8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 -- 172.21.15.32:0/1359824365 >> 172.21.15.32:0/1359824365 conn(0x55b119dea590 msgr2=0x55b119dea970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 -- 172.21.15.32:0/1359824365 shutdown_connections 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 -- 172.21.15.32:0/1359824365 wait complete. 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.262+0000 7f8770971740 1 -- start start 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8770971740 1 Processor -- start 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8770971740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119e01a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8770971740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df0780 0x55b119dfcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8770971740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df9360 0x55b119dfd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8770971740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87681ad920 con 0x55b119df9360 2024-04-16T05:40:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8770971740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87681aed90 con 0x55b119defe60 2024-04-16T05:40:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8770971740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87681adff0 con 0x55b119df0780 2024-04-16T05:40:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8770971740 10 client.0 _ll_register_callbacks cb 0x7f8758009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df9360 0x55b119dfd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df9360 0x55b119dfd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50150/0 (socket says 172.21.15.32:50150) 2024-04-16T05:40:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 -- 172.21.15.32:0/1521504075 learned_addr learned my addr 172.21.15.32:0/1521504075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f87677fe640 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df0780 0x55b119dfcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119e01a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 -- 172.21.15.32:0/1521504075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df0780 msgr2=0x55b119dfcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df0780 0x55b119dfcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 -- 172.21.15.32:0/1521504075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 msgr2=0x55b119e01a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119e01a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 -- 172.21.15.32:0/1521504075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b119dfdbe0 con 0x55b119df9360 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f875ffff640 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df9360 0x55b119dfd3b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8750005d00 tx=0x7f8750005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3560cd35f940b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.263+0000 7f8746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8746ffd640 1 -- 172.21.15.32:0/1521504075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750018040 con 0x55b119df9360 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8746ffd640 1 -- 172.21.15.32:0/1521504075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8750011d60 con 0x55b119df9360 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 1 -- 172.21.15.32:0/1521504075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8758005520 con 0x55b119df9360 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8746ffd640 1 -- 172.21.15.32:0/1521504075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8750043050 con 0x55b119df9360 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 10 client.16977 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 1 -- 172.21.15.32:0/1521504075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b119e6f560 con 0x55b119df9360 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8746ffd640 1 -- 172.21.15.32:0/1521504075 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8758005520 con 0x55b119df9360 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8746ffd640 1 -- 172.21.15.32:0/1521504075 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8750005000 con 0x55b119df9360 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 10 client.16977 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 10 client.16977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:40:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 20 client.16977 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b119e6fc80 0x55b119e720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 4 client.16977 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.264+0000 7f8770971740 1 -- 172.21.15.32:0/1521504075 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b119d5c970 con 0x55b119e6fc80 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f87677fe640 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b119e6fc80 0x55b119e720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8746ffd640 10 client.16977 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f87677fe640 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b119e6fc80 0x55b119e720a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fce866e687d987af server_cookie=cf48383b5faa18cf in_seq=0 out_seq=0 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8746ffd640 1 -- 172.21.15.32:0/1521504075 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b119d5c970 con 0x55b119e6fc80 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8746ffd640 10 client.16977 handle_command_reply: tid=0 2024-04-16T05:40:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 client.16977 shutdown 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 -- 172.21.15.32:0/1521504075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df9360 msgr2=0x55b119dfd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df9360 0x55b119dfd3b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8750005d00 tx=0x7f8750005f60 comp rx=0 tx=0).stop 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 -- 172.21.15.32:0/1521504075 shutdown_connections 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55b119e6fc80 0x55b119e720a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b119df9360 0x55b119dfd3b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b119df0780 0x55b119dfcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 --2- 172.21.15.32:0/1521504075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b119defe60 0x55b119e01a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8746ffd640 0 client.16977 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 -- 172.21.15.32:0/1521504075 >> 172.21.15.32:0/1521504075 conn(0x55b119dea590 msgr2=0x55b119deb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 -- 172.21.15.32:0/1521504075 shutdown_connections 2024-04-16T05:40:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 1 -- 172.21.15.32:0/1521504075 wait complete. 2024-04-16T05:40:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.265+0000 7f8770971740 20 client.16977 trim_cache size 0 max 16384 2024-04-16T05:40:26.275 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 -- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f87540a3da0 msgr2=0x7f87540a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f87540a3da0 0x7f87540a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f876009cad0 tx=0x7f876006f8c0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 -- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768198630 msgr2=0x7f8768198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768198630 0x7f8768198ab0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f876000cce0 tx=0x7f876000a620 comp rx=0 tx=0).stop 2024-04-16T05:40:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 -- 172.21.15.32:0/366676320 shutdown_connections 2024-04-16T05:40:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f87540a3da0 0x7f87540a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8768198630 0x7f8768198ab0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768106200 0x7f876819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 --2- 172.21.15.32:0/366676320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101130 0x7f876819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 -- 172.21.15.32:0/366676320 >> 172.21.15.32:0/366676320 conn(0x7f87680780b0 msgr2=0x7f87680fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 -- 172.21.15.32:0/366676320 shutdown_connections 2024-04-16T05:40:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.266+0000 7f876dcb1640 1 -- 172.21.15.32:0/366676320 wait complete. 2024-04-16T05:40:26.279 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:40:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 -- 172.21.15.32:0/2163754453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868102250 msgr2=0x7f6868102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 --2- 172.21.15.32:0/2163754453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868102250 0x7f6868102630 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6858009900 tx=0x7f685802bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 -- 172.21.15.32:0/2163754453 shutdown_connections 2024-04-16T05:40:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 --2- 172.21.15.32:0/2163754453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6868107320 0x7f686810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 --2- 172.21.15.32:0/2163754453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868102c00 0x7f6868106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 --2- 172.21.15.32:0/2163754453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868102250 0x7f6868102630 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 -- 172.21.15.32:0/2163754453 >> 172.21.15.32:0/2163754453 conn(0x7f6868079490 msgr2=0x7f68680798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 -- 172.21.15.32:0/2163754453 shutdown_connections 2024-04-16T05:40:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.387+0000 7f686cceb640 1 -- 172.21.15.32:0/2163754453 wait complete. 2024-04-16T05:40:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f686cceb640 1 Processor -- start 2024-04-16T05:40:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f686cceb640 1 -- start start 2024-04-16T05:40:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f686cceb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6868102250 0x7f686819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f686cceb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868107320 0x7f686819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f686cceb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68681986a0 0x7f6868198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f686cceb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68681123e0 con 0x7f6868107320 2024-04-16T05:40:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f686cceb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6868112260 con 0x7f68681986a0 2024-04-16T05:40:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f686cceb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6868112560 con 0x7f6868102250 2024-04-16T05:40:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f685dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868107320 0x7f686819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f685dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868107320 0x7f686819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50166/0 (socket says 172.21.15.32:50166) 2024-04-16T05:40:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.388+0000 7f685dd9b640 1 -- 172.21.15.32:0/2740023885 learned_addr learned my addr 172.21.15.32:0/2740023885 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f686659c640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6868102250 0x7f686819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f6866d9d640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68681986a0 0x7f6868198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685dd9b640 1 -- 172.21.15.32:0/2740023885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6868102250 msgr2=0x7f686819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685dd9b640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6868102250 0x7f686819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685dd9b640 1 -- 172.21.15.32:0/2740023885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68681986a0 msgr2=0x7f6868198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685dd9b640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68681986a0 0x7f6868198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685dd9b640 1 -- 172.21.15.32:0/2740023885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68681992a0 con 0x7f6868107320 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f686659c640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6868102250 0x7f686819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685dd9b640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868107320 0x7f686819eb20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6850005fd0 tx=0x7f685000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685ffff640 1 -- 172.21.15.32:0/2740023885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850017070 con 0x7f6868107320 2024-04-16T05:40:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f686cceb640 1 -- 172.21.15.32:0/2740023885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68681a5380 con 0x7f6868107320 2024-04-16T05:40:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685ffff640 1 -- 172.21.15.32:0/2740023885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6850009c30 con 0x7f6868107320 2024-04-16T05:40:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f685ffff640 1 -- 172.21.15.32:0/2740023885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850010460 con 0x7f6868107320 2024-04-16T05:40:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.389+0000 7f686cceb640 1 -- 172.21.15.32:0/2740023885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68681a5780 con 0x7f6868107320 2024-04-16T05:40:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.391+0000 7f685ffff640 1 -- 172.21.15.32:0/2740023885 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6850003e70 con 0x7f6868107320 2024-04-16T05:40:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686f9b3740 1 Processor -- start 2024-04-16T05:40:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686f9b3740 1 -- start start 2024-04-16T05:40:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 0x5586f6d0ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d1d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d1fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686f9b3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5586f6c750e0 con 0x5586f6d1d770 2024-04-16T05:40:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686f9b3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5586f6c12520 con 0x5586f6d104b0 2024-04-16T05:40:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686f9b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586f6c710a0 con 0x5586f6d0fb90 2024-04-16T05:40:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f685ffff640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68540a3d20 0x7f68540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f685dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d1fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f6866d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 0x5586f6d0ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f685dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d1fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50174/0 (socket says 172.21.15.32:50174) 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f685dd9b640 1 -- 172.21.15.32:0/1713369525 learned_addr learned my addr 172.21.15.32:0/1713369525 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f6866d9d640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68540a3d20 0x7f68540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f686659c640 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d1d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.392+0000 7f685ffff640 1 -- 172.21.15.32:0/2740023885 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f68500d0830 con 0x7f6868107320 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685dd9b640 1 -- 172.21.15.32:0/1713369525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 msgr2=0x5586f6d0ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685dd9b640 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 0x5586f6d0ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685dd9b640 1 -- 172.21.15.32:0/1713369525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 msgr2=0x5586f6d1d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685dd9b640 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d1d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685dd9b640 1 -- 172.21.15.32:0/1713369525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f6c3a8d0 con 0x5586f6d1d770 2024-04-16T05:40:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f686659c640 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d1d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:40:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f6866d9d640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68540a3d20 0x7f68540a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6860009fd0 tx=0x7f6860009e00 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685dd9b640 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d1fb60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f685009c130 tx=0x7f68500987d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a994b8981f8855d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685cd99640 1 -- 172.21.15.32:0/1713369525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850009de0 con 0x5586f6d1d770 2024-04-16T05:40:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685cd99640 1 -- 172.21.15.32:0/1713369525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f685009c430 con 0x5586f6d1d770 2024-04-16T05:40:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.393+0000 7f685cd99640 1 -- 172.21.15.32:0/1713369525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500d0430 con 0x5586f6d1d770 2024-04-16T05:40:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 -- 172.21.15.32:0/1713369525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 msgr2=0x5586f6d1fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d1fb60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f685009c130 tx=0x7f68500987d0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 -- 172.21.15.32:0/1713369525 shutdown_connections 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d1fb60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d1d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 --2- 172.21.15.32:0/1713369525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 0x5586f6d0ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 -- 172.21.15.32:0/1713369525 >> 172.21.15.32:0/1713369525 conn(0x5586f6d0a590 msgr2=0x5586f6d0a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 -- 172.21.15.32:0/1713369525 shutdown_connections 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 -- 172.21.15.32:0/1713369525 wait complete. 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 -- start start 2024-04-16T05:40:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.394+0000 7f686f9b3740 1 Processor -- start 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d0fb90 0x5586f6d1a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d104b0 0x5586f6d1ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d1d770 0x5586f6d1b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686f9b3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5586f6d204d0 con 0x5586f6d104b0 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686f9b3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5586f6c2e4d0 con 0x5586f6d0fb90 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686f9b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586f6d20650 con 0x5586f6d1d770 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686f9b3740 10 client.0 _ll_register_callbacks cb 0x5586f6c132c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d104b0 0x5586f6d1ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d104b0 0x5586f6d1ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50184/0 (socket says 172.21.15.32:50184) 2024-04-16T05:40:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 -- 172.21.15.32:0/3002829670 learned_addr learned my addr 172.21.15.32:0/3002829670 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f6866d9d640 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d1d770 0x5586f6d1b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686659c640 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d0fb90 0x5586f6d1a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 -- 172.21.15.32:0/3002829670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d1d770 msgr2=0x5586f6d1b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d1d770 0x5586f6d1b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 -- 172.21.15.32:0/3002829670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d0fb90 msgr2=0x5586f6d1a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d0fb90 0x5586f6d1a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 -- 172.21.15.32:0/3002829670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f69873f0 con 0x5586f6d104b0 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f685dd9b640 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d104b0 0x5586f6d1ad10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6850009610 tx=0x7f6850098cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b2572ed02cba32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f684dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f684dffb640 1 -- 172.21.15.32:0/3002829670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500ce060 con 0x5586f6d104b0 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f684dffb640 1 -- 172.21.15.32:0/3002829670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f68500a4c10 con 0x5586f6d104b0 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.395+0000 7f686f9b3740 1 -- 172.21.15.32:0/3002829670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f685009c430 con 0x5586f6d104b0 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f684dffb640 1 -- 172.21.15.32:0/3002829670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68500a50a0 con 0x5586f6d104b0 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686f9b3740 10 client.16989 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686f9b3740 1 -- 172.21.15.32:0/3002829670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586f6a517d0 con 0x5586f6d104b0 2024-04-16T05:40:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f684dffb640 1 -- 172.21.15.32:0/3002829670 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f685009c430 con 0x5586f6d104b0 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f684dffb640 1 -- 172.21.15.32:0/3002829670 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68500bb200 con 0x5586f6d104b0 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686f9b3740 10 client.16989 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686f9b3740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686f9b3740 20 client.16989 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5586f6d2d5f0 0x5586f6d2f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686f9b3740 4 client.16989 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686f9b3740 1 -- 172.21.15.32:0/3002829670 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586f6c82b00 con 0x5586f6d2d5f0 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.396+0000 7f686659c640 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5586f6d2d5f0 0x5586f6d2f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.397+0000 7f686659c640 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5586f6d2d5f0 0x5586f6d2f9e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916d1639dbea7b7b server_cookie=a27e969d3fac9440 in_seq=0 out_seq=0 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.397+0000 7f684dffb640 10 client.16989 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.397+0000 7f684dffb640 1 -- 172.21.15.32:0/3002829670 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586f6c82b00 con 0x5586f6d2d5f0 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.397+0000 7f684dffb640 10 client.16989 handle_command_reply: tid=0 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.397+0000 7f686f9b3740 1 client.16989 shutdown 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 -- 172.21.15.32:0/3002829670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d104b0 msgr2=0x5586f6d1ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d104b0 0x5586f6d1ad10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6850009610 tx=0x7f6850098cd0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 -- 172.21.15.32:0/3002829670 shutdown_connections 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5586f6d2d5f0 0x5586f6d2f9e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d1d770 0x5586f6d1b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d104b0 0x5586f6d1ad10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f684dffb640 0 client.16989 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3002829670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d0fb90 0x5586f6d1a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 -- 172.21.15.32:0/3002829670 >> 172.21.15.32:0/3002829670 conn(0x5586f6d0a590 msgr2=0x5586f6d0c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 -- 172.21.15.32:0/3002829670 shutdown_connections 2024-04-16T05:40:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 1 -- 172.21.15.32:0/3002829670 wait complete. 2024-04-16T05:40:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.398+0000 7f686f9b3740 20 client.16989 trim_cache size 0 max 16384 2024-04-16T05:40:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.411+0000 7f686f9b3740 1 Processor -- start 2024-04-16T05:40:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.411+0000 7f686f9b3740 1 -- start start 2024-04-16T05:40:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.411+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 0x5586f6d0c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.411+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d32a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.411+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d32f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.411+0000 7f686f9b3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68681af9e0 con 0x5586f6d1d770 2024-04-16T05:40:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.411+0000 7f686f9b3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f68681ad2b0 con 0x5586f6d104b0 2024-04-16T05:40:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.411+0000 7f686f9b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68681ab100 con 0x5586f6d0fb90 2024-04-16T05:40:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d32f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d32f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50198/0 (socket says 172.21.15.32:50198) 2024-04-16T05:40:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 -- 172.21.15.32:0/17003875 learned_addr learned my addr 172.21.15.32:0/17003875 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f686659c640 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 0x5586f6d0c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f685dd9b640 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d32a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 -- 172.21.15.32:0/17003875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 msgr2=0x5586f6d0c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 0x5586f6d0c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 -- 172.21.15.32:0/17003875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 msgr2=0x5586f6d32a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d32a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 -- 172.21.15.32:0/17003875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f6d0b160 con 0x5586f6d1d770 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f685dd9b640 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d32a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f6866d9d640 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d32f40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f686000fa80 tx=0x7f6860009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20f12e52f096907 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f684ffff640 1 -- 172.21.15.32:0/17003875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860019040 con 0x5586f6d1d770 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f684ffff640 1 -- 172.21.15.32:0/17003875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6860004030 con 0x5586f6d1d770 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.412+0000 7f684ffff640 1 -- 172.21.15.32:0/17003875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860005350 con 0x5586f6d1d770 2024-04-16T05:40:26.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- 172.21.15.32:0/17003875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 msgr2=0x5586f6d32f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d32f40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f686000fa80 tx=0x7f6860009500 comp rx=0 tx=0).stop 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- 172.21.15.32:0/17003875 shutdown_connections 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d1d770 0x5586f6d32f40 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d104b0 0x5586f6d32a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 --2- 172.21.15.32:0/17003875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d0fb90 0x5586f6d0c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- 172.21.15.32:0/17003875 >> 172.21.15.32:0/17003875 conn(0x5586f6d0a590 msgr2=0x5586f6d0b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- 172.21.15.32:0/17003875 shutdown_connections 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- 172.21.15.32:0/17003875 wait complete. 2024-04-16T05:40:26.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- start start 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 Processor -- start 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d0fb90 0x5586f6d2fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d104b0 0x5586f6d301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d1d770 0x5586f6d8b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68681ab770 con 0x5586f6d0fb90 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f68681acbe0 con 0x5586f6d1d770 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68681abe40 con 0x5586f6d104b0 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686f9b3740 10 client.0 _ll_register_callbacks cb 0x7f68600093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.413+0000 7f686659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d0fb90 0x5586f6d2fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d0fb90 0x5586f6d2fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50212/0 (socket says 172.21.15.32:50212) 2024-04-16T05:40:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686659c640 1 -- 172.21.15.32:0/3634259065 learned_addr learned my addr 172.21.15.32:0/3634259065 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f6866d9d640 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d1d770 0x5586f6d8b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f685dd9b640 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d104b0 0x5586f6d301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686659c640 1 -- 172.21.15.32:0/3634259065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d104b0 msgr2=0x5586f6d301f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686659c640 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d104b0 0x5586f6d301f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686659c640 1 -- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d1d770 msgr2=0x5586f6d8b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686659c640 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d1d770 0x5586f6d8b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686659c640 1 -- 172.21.15.32:0/3634259065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f6d8bdf0 con 0x5586f6d0fb90 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f6866d9d640 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d1d770 0x5586f6d8b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686659c640 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d0fb90 0x5586f6d2fcb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f685802cfd0 tx=0x7f685802cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c13df6ba936cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f684dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f684dffb640 1 -- 172.21.15.32:0/3634259065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858049070 con 0x5586f6d0fb90 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f684dffb640 1 -- 172.21.15.32:0/3634259065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6858004040 con 0x5586f6d0fb90 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686f9b3740 1 -- 172.21.15.32:0/3634259065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6860004030 con 0x5586f6d0fb90 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f684dffb640 1 -- 172.21.15.32:0/3634259065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6858055050 con 0x5586f6d0fb90 2024-04-16T05:40:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686f9b3740 10 client.16995 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.414+0000 7f686f9b3740 1 -- 172.21.15.32:0/3634259065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586f6d93130 con 0x5586f6d0fb90 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f684dffb640 1 -- 172.21.15.32:0/3634259065 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6860004030 con 0x5586f6d0fb90 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f684dffb640 1 -- 172.21.15.32:0/3634259065 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6858045250 con 0x5586f6d0fb90 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f686f9b3740 10 client.16995 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f686f9b3740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f686f9b3740 20 client.16995 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5586f6d93890 0x5586f6d95cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f686f9b3740 4 client.16995 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f686f9b3740 1 -- 172.21.15.32:0/3634259065 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586f6c7c970 con 0x5586f6d93890 2024-04-16T05:40:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f685dd9b640 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5586f6d93890 0x5586f6d95cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f684dffb640 10 client.16995 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f685dd9b640 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5586f6d93890 0x5586f6d95cb0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30149b53f17196e6 server_cookie=b158f22e47a6900c in_seq=0 out_seq=0 2024-04-16T05:40:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f684dffb640 1 -- 172.21.15.32:0/3634259065 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586f6c7c970 con 0x5586f6d93890 2024-04-16T05:40:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f684dffb640 10 client.16995 handle_command_reply: tid=0 2024-04-16T05:40:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.415+0000 7f686f9b3740 1 client.16995 shutdown 2024-04-16T05:40:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 -- 172.21.15.32:0/3634259065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d0fb90 msgr2=0x5586f6d2fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d0fb90 0x5586f6d2fcb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f685802cfd0 tx=0x7f685802cbd0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 -- 172.21.15.32:0/3634259065 shutdown_connections 2024-04-16T05:40:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f6d1d770 0x5586f6d8b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5586f6d93890 0x5586f6d95cb0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f6d104b0 0x5586f6d301f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f684dffb640 0 client.16995 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 --2- 172.21.15.32:0/3634259065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586f6d0fb90 0x5586f6d2fcb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 -- 172.21.15.32:0/3634259065 >> 172.21.15.32:0/3634259065 conn(0x5586f6d0a590 msgr2=0x5586f6d09730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 -- 172.21.15.32:0/3634259065 shutdown_connections 2024-04-16T05:40:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 1 -- 172.21.15.32:0/3634259065 wait complete. 2024-04-16T05:40:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686f9b3740 20 client.16995 trim_cache size 0 max 16384 2024-04-16T05:40:26.427 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:26.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 -- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68540a3d20 msgr2=0x7f68540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68540a3d20 0x7f68540a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6860009fd0 tx=0x7f6860009e00 comp rx=0 tx=0).stop 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 -- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868107320 msgr2=0x7f686819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868107320 0x7f686819eb20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6850005fd0 tx=0x7f685000dde0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 -- 172.21.15.32:0/2740023885 shutdown_connections 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68681986a0 0x7f6868198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f68540a3d20 0x7f68540a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6868107320 0x7f686819eb20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 --2- 172.21.15.32:0/2740023885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6868102250 0x7f686819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 -- 172.21.15.32:0/2740023885 >> 172.21.15.32:0/2740023885 conn(0x7f6868079490 msgr2=0x7f68680ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.416+0000 7f686cceb640 1 -- 172.21.15.32:0/2740023885 shutdown_connections 2024-04-16T05:40:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:26.417+0000 7f686cceb640 1 -- 172.21.15.32:0/2740023885 wait complete. 2024-04-16T05:40:26.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:26 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:40:26.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:26 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:26.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:26 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:40:26.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:26 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:26.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:26 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:40:26.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:26 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2157122486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:26.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:26 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:40:26.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:26 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:26.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:26 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:40:26.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:26 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:26.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:26 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:40:26.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:26 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2157122486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:26.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:26 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:40:26.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:26 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:26.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:26 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:40:26.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:26 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:26.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:26 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:40:26.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:26 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2157122486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:27.431 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:40:27.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:40:27.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:40:27.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 -- 172.21.15.32:0/3300037805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c10ea10 msgr2=0x7f8b2c110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 --2- 172.21.15.32:0/3300037805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c10ea10 0x7f8b2c110e00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009ca0 tx=0x7f8b2402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 -- 172.21.15.32:0/3300037805 shutdown_connections 2024-04-16T05:40:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 --2- 172.21.15.32:0/3300037805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c10ea10 0x7f8b2c110e00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 --2- 172.21.15.32:0/3300037805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b2c06afa0 0x7f8b2c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 --2- 172.21.15.32:0/3300037805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c06a680 0x7f8b2c06aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 -- 172.21.15.32:0/3300037805 >> 172.21.15.32:0/3300037805 conn(0x7f8b2c077f60 msgr2=0x7f8b2c078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 -- 172.21.15.32:0/3300037805 shutdown_connections 2024-04-16T05:40:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.555+0000 7f8b33488640 1 -- 172.21.15.32:0/3300037805 wait complete. 2024-04-16T05:40:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b33488640 1 Processor -- start 2024-04-16T05:40:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b33488640 1 -- start start 2024-04-16T05:40:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b33488640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b2c06a680 0x7f8b2c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b33488640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c06afa0 0x7f8b2c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b33488640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c10ea10 0x7f8b2c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b33488640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c114590 con 0x7f8b2c06afa0 2024-04-16T05:40:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b33488640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c114410 con 0x7f8b2c10ea10 2024-04-16T05:40:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b33488640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c114710 con 0x7f8b2c06a680 2024-04-16T05:40:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b30a23640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c06afa0 0x7f8b2c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b30a23640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c06afa0 0x7f8b2c1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50230/0 (socket says 172.21.15.32:50230) 2024-04-16T05:40:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.556+0000 7f8b30a23640 1 -- 172.21.15.32:0/840964452 learned_addr learned my addr 172.21.15.32:0/840964452 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b31224640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b2c06a680 0x7f8b2c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b31a25640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c10ea10 0x7f8b2c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b30a23640 1 -- 172.21.15.32:0/840964452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b2c06a680 msgr2=0x7f8b2c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b30a23640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b2c06a680 0x7f8b2c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b30a23640 1 -- 172.21.15.32:0/840964452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c10ea10 msgr2=0x7f8b2c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b30a23640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c10ea10 0x7f8b2c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b30a23640 1 -- 172.21.15.32:0/840964452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c19aff0 con 0x7f8b2c06afa0 2024-04-16T05:40:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b31a25640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c10ea10 0x7f8b2c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b30a23640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c06afa0 0x7f8b2c1a0c00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00c920 tx=0x7f8b1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b227fc640 1 -- 172.21.15.32:0/840964452 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c007c70 con 0x7f8b2c06afa0 2024-04-16T05:40:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b227fc640 1 -- 172.21.15.32:0/840964452 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1c007e10 con 0x7f8b2c06afa0 2024-04-16T05:40:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b33488640 1 -- 172.21.15.32:0/840964452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c19b2e0 con 0x7f8b2c06afa0 2024-04-16T05:40:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b227fc640 1 -- 172.21.15.32:0/840964452 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c018890 con 0x7f8b2c06afa0 2024-04-16T05:40:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.557+0000 7f8b33488640 1 -- 172.21.15.32:0/840964452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1a7580 con 0x7f8b2c06afa0 2024-04-16T05:40:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.559+0000 7f8b227fc640 1 -- 172.21.15.32:0/840964452 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b1c018a70 con 0x7f8b2c06afa0 2024-04-16T05:40:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b36074740 1 Processor -- start 2024-04-16T05:40:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b36074740 1 -- start start 2024-04-16T05:40:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b36074740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b36074740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b36074740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b36074740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5599c8140e90 con 0x5599c81dbab0 2024-04-16T05:40:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b36074740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5599c80de4c0 con 0x5599c81dc3d0 2024-04-16T05:40:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b36074740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5599c813ce50 con 0x5599c81e4fb0 2024-04-16T05:40:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b227fc640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b180a3d20 0x7f8b180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.560+0000 7f8b31a25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81dbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50246/0 (socket says 172.21.15.32:50246) 2024-04-16T05:40:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 -- 172.21.15.32:0/3001118316 learned_addr learned my addr 172.21.15.32:0/3001118316 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b180a3d20 0x7f8b180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b227fc640 1 -- 172.21.15.32:0/840964452 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b1c0d0c70 con 0x7f8b2c06afa0 2024-04-16T05:40:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31224640 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b30a23640 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 -- 172.21.15.32:0/3001118316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 msgr2=0x5599c81e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 -- 172.21.15.32:0/3001118316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 msgr2=0x5599c81e4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81e4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 -- 172.21.15.32:0/3001118316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c7f1b7e0 con 0x5599c81dbab0 2024-04-16T05:40:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31224640 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:40:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b180a3d20 0x7f8b180a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009a60 tx=0x7f8b24005fa0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b31a25640 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81dbe90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009ca0 tx=0x7f8b24046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a3c3664c54fe97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b177fe640 1 -- 172.21.15.32:0/3001118316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24051070 con 0x5599c81dbab0 2024-04-16T05:40:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b177fe640 1 -- 172.21.15.32:0/3001118316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b24003e50 con 0x5599c81dbab0 2024-04-16T05:40:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.561+0000 7f8b177fe640 1 -- 172.21.15.32:0/3001118316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24039070 con 0x5599c81dbab0 2024-04-16T05:40:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 -- 172.21.15.32:0/3001118316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 msgr2=0x5599c81dbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81dbe90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009ca0 tx=0x7f8b24046000 comp rx=0 tx=0).stop 2024-04-16T05:40:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 -- 172.21.15.32:0/3001118316 shutdown_connections 2024-04-16T05:40:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 --2- 172.21.15.32:0/3001118316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81dbe90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 -- 172.21.15.32:0/3001118316 >> 172.21.15.32:0/3001118316 conn(0x5599c81d61e0 msgr2=0x5599c81d65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 -- 172.21.15.32:0/3001118316 shutdown_connections 2024-04-16T05:40:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.562+0000 7f8b36074740 1 -- 172.21.15.32:0/3001118316 wait complete. 2024-04-16T05:40:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 1 -- start start 2024-04-16T05:40:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 1 Processor -- start 2024-04-16T05:40:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81dc3d0 0x5599c81e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81e4fb0 0x5599c81e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5599c81e7d40 con 0x5599c81e4fb0 2024-04-16T05:40:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5599c80c9410 con 0x5599c81dbab0 2024-04-16T05:40:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5599c81e7ec0 con 0x5599c81dc3d0 2024-04-16T05:40:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b36074740 10 client.0 _ll_register_callbacks cb 0x5599c80c4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31a25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81e4fb0 0x5599c81e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31a25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81e4fb0 0x5599c81e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50250/0 (socket says 172.21.15.32:50250) 2024-04-16T05:40:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31a25640 1 -- 172.21.15.32:0/3902747218 learned_addr learned my addr 172.21.15.32:0/3902747218 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b30a23640 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81dc3d0 0x5599c81e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31224640 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31a25640 1 -- 172.21.15.32:0/3902747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81dc3d0 msgr2=0x5599c81e25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31a25640 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81dc3d0 0x5599c81e25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31a25640 1 -- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 msgr2=0x5599c81e2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31a25640 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.563+0000 7f8b31a25640 1 -- 172.21.15.32:0/3902747218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c7f7bf70 con 0x5599c81e4fb0 2024-04-16T05:40:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b31224640 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b31a25640 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81e4fb0 0x5599c81e2b00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8b24032fb0 tx=0x7f8b2402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aeefcee74b0977 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b14ff9640 1 -- 172.21.15.32:0/3902747218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2405a060 con 0x5599c81e4fb0 2024-04-16T05:40:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b14ff9640 1 -- 172.21.15.32:0/3902747218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b24055450 con 0x5599c81e4fb0 2024-04-16T05:40:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 1 -- 172.21.15.32:0/3902747218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b24003e50 con 0x5599c81e4fb0 2024-04-16T05:40:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b14ff9640 1 -- 172.21.15.32:0/3902747218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b24061050 con 0x5599c81e4fb0 2024-04-16T05:40:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 10 client.17007 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 1 -- 172.21.15.32:0/3902747218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599c7e13170 con 0x5599c81e4fb0 2024-04-16T05:40:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b14ff9640 1 -- 172.21.15.32:0/3902747218 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b24003e50 con 0x5599c81e4fb0 2024-04-16T05:40:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b14ff9640 1 -- 172.21.15.32:0/3902747218 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8b2404b070 con 0x5599c81e4fb0 2024-04-16T05:40:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 10 client.17007 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 20 client.17007 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:27.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5599c81f51f0 0x5599c81f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 4 client.17007 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.564+0000 7f8b36074740 1 -- 172.21.15.32:0/3902747218 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599c814e8b0 con 0x5599c81f51f0 2024-04-16T05:40:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.565+0000 7f8b31224640 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5599c81f51f0 0x5599c81f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.565+0000 7f8b31224640 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5599c81f51f0 0x5599c81f75e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238af0965f46f51d server_cookie=6f39ed184b13cceb in_seq=0 out_seq=0 2024-04-16T05:40:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.565+0000 7f8b14ff9640 10 client.17007 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b14ff9640 1 -- 172.21.15.32:0/3902747218 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599c814e8b0 con 0x5599c81f51f0 2024-04-16T05:40:27.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b14ff9640 10 client.17007 handle_command_reply: tid=0 2024-04-16T05:40:27.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 client.17007 shutdown 2024-04-16T05:40:27.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 -- 172.21.15.32:0/3902747218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81e4fb0 msgr2=0x5599c81e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81e4fb0 0x5599c81e2b00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8b24032fb0 tx=0x7f8b2402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 -- 172.21.15.32:0/3902747218 shutdown_connections 2024-04-16T05:40:27.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81e4fb0 0x5599c81e2b00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5599c81f51f0 0x5599c81f75e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81dc3d0 0x5599c81e25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 --2- 172.21.15.32:0/3902747218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b14ff9640 0 client.17007 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:27.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 -- 172.21.15.32:0/3902747218 >> 172.21.15.32:0/3902747218 conn(0x5599c81d61e0 msgr2=0x5599c81d7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 -- 172.21.15.32:0/3902747218 shutdown_connections 2024-04-16T05:40:27.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 1 -- 172.21.15.32:0/3902747218 wait complete. 2024-04-16T05:40:27.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.566+0000 7f8b36074740 20 client.17007 trim_cache size 0 max 16384 2024-04-16T05:40:27.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b36074740 1 Processor -- start 2024-04-16T05:40:27.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b36074740 1 -- start start 2024-04-16T05:40:27.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b36074740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b36074740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dc3d0 0x5599c81e2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b36074740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b36074740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1b3cf0 con 0x5599c81dc3d0 2024-04-16T05:40:27.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b36074740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1b15c0 con 0x5599c81dbab0 2024-04-16T05:40:27.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b36074740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1af410 con 0x5599c81e4fb0 2024-04-16T05:40:27.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b30a23640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dc3d0 0x5599c81e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.584+0000 7f8b30a23640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dc3d0 0x5599c81e2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50260/0 (socket says 172.21.15.32:50260) 2024-04-16T05:40:27.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b30a23640 1 -- 172.21.15.32:0/27862369 learned_addr learned my addr 172.21.15.32:0/27862369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b31224640 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b31a25640 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b30a23640 1 -- 172.21.15.32:0/27862369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 msgr2=0x5599c81fa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b30a23640 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81fa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b30a23640 1 -- 172.21.15.32:0/27862369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 msgr2=0x5599c81e47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b30a23640 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b30a23640 1 -- 172.21.15.32:0/27862369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c81dc1f0 con 0x5599c81dc3d0 2024-04-16T05:40:27.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b30a23640 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dc3d0 0x5599c81e2750 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0ce080 tx=0x7f8b1c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6876d51001627a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b16ffd640 1 -- 172.21.15.32:0/27862369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c09cad0 con 0x5599c81dc3d0 2024-04-16T05:40:27.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b16ffd640 1 -- 172.21.15.32:0/27862369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1c09cd10 con 0x5599c81dc3d0 2024-04-16T05:40:27.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.585+0000 7f8b16ffd640 1 -- 172.21.15.32:0/27862369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0870b0 con 0x5599c81dc3d0 2024-04-16T05:40:27.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 -- 172.21.15.32:0/27862369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dc3d0 msgr2=0x5599c81e2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dc3d0 0x5599c81e2750 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0ce080 tx=0x7f8b1c087040 comp rx=0 tx=0).stop 2024-04-16T05:40:27.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 -- 172.21.15.32:0/27862369 shutdown_connections 2024-04-16T05:40:27.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dc3d0 0x5599c81e2750 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 --2- 172.21.15.32:0/27862369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dbab0 0x5599c81e47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 -- 172.21.15.32:0/27862369 >> 172.21.15.32:0/27862369 conn(0x5599c81d61e0 msgr2=0x5599c81e2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 -- 172.21.15.32:0/27862369 shutdown_connections 2024-04-16T05:40:27.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 -- 172.21.15.32:0/27862369 wait complete. 2024-04-16T05:40:27.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 -- start start 2024-04-16T05:40:27.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.586+0000 7f8b36074740 1 Processor -- start 2024-04-16T05:40:27.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b36074740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b36074740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b36074740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b36074740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1afa80 con 0x5599c81dbab0 2024-04-16T05:40:27.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b36074740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1b0ef0 con 0x5599c81dc3d0 2024-04-16T05:40:27.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b36074740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1b0150 con 0x5599c81e4fb0 2024-04-16T05:40:27.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b36074740 10 client.0 _ll_register_callbacks cb 0x7f8b2404f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:27.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81f7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50272/0 (socket says 172.21.15.32:50272) 2024-04-16T05:40:27.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 -- 172.21.15.32:0/1705426854 learned_addr learned my addr 172.21.15.32:0/1705426854 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31a25640 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b30a23640 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 -- 172.21.15.32:0/1705426854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 msgr2=0x5599c81f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 -- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 msgr2=0x5599c81f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 -- 172.21.15.32:0/1705426854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c825a3d0 con 0x5599c81dbab0 2024-04-16T05:40:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b30a23640 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b31224640 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81f7fe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x5599c81e8010 tx=0x7f8b2800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10291089af89c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b14ff9640 1 -- 172.21.15.32:0/1705426854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2801d380 con 0x5599c81dbab0 2024-04-16T05:40:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b36074740 1 -- 172.21.15.32:0/1705426854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b1c09cd10 con 0x5599c81dbab0 2024-04-16T05:40:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.587+0000 7f8b14ff9640 1 -- 172.21.15.32:0/1705426854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8b2801d520 con 0x5599c81dbab0 2024-04-16T05:40:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b14ff9640 1 -- 172.21.15.32:0/1705426854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b2802f050 con 0x5599c81dbab0 2024-04-16T05:40:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b36074740 10 client.17013 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b36074740 1 -- 172.21.15.32:0/1705426854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599c825b720 con 0x5599c81dbab0 2024-04-16T05:40:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b14ff9640 1 -- 172.21.15.32:0/1705426854 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1c09cd10 con 0x5599c81dbab0 2024-04-16T05:40:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b14ff9640 1 -- 172.21.15.32:0/1705426854 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8b28011440 con 0x5599c81dbab0 2024-04-16T05:40:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b36074740 10 client.17013 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b36074740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b36074740 20 client.17013 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b36074740 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5599c825be80 0x5599c825e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b36074740 4 client.17013 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:40:27.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b36074740 1 -- 172.21.15.32:0/1705426854 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5599c8148720 con 0x5599c825be80 2024-04-16T05:40:27.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.588+0000 7f8b30a23640 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5599c825be80 0x5599c825e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.589+0000 7f8b30a23640 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5599c825be80 0x5599c825e2a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=730b916f2cc59556 server_cookie=25c02cba4070c5c9 in_seq=0 out_seq=0 2024-04-16T05:40:27.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.589+0000 7f8b14ff9640 10 client.17013 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:27.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b14ff9640 1 -- 172.21.15.32:0/1705426854 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5599c8148720 con 0x5599c825be80 2024-04-16T05:40:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b14ff9640 10 client.17013 handle_command_reply: tid=0 2024-04-16T05:40:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 client.17013 shutdown 2024-04-16T05:40:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 -- 172.21.15.32:0/1705426854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 msgr2=0x5599c81f7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81f7fe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x5599c81e8010 tx=0x7f8b2800bd90 comp rx=0 tx=0).stop 2024-04-16T05:40:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 -- 172.21.15.32:0/1705426854 shutdown_connections 2024-04-16T05:40:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5599c825be80 0x5599c825e2a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599c81e4fb0 0x5599c81f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c81dc3d0 0x5599c81f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b14ff9640 0 client.17013 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 --2- 172.21.15.32:0/1705426854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5599c81dbab0 0x5599c81f7fe0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 -- 172.21.15.32:0/1705426854 >> 172.21.15.32:0/1705426854 conn(0x5599c81d61e0 msgr2=0x5599c81f09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 -- 172.21.15.32:0/1705426854 shutdown_connections 2024-04-16T05:40:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 1 -- 172.21.15.32:0/1705426854 wait complete. 2024-04-16T05:40:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.590+0000 7f8b36074740 20 client.17013 trim_cache size 0 max 16384 2024-04-16T05:40:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:40:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:40:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "db622bc2-b59d-40a8-9f1a-3927dfab4f8e", 2024-04-16T05:40:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:40:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:40:27.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 -- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b180a3d20 msgr2=0x7f8b180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b180a3d20 0x7f8b180a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009a60 tx=0x7f8b24005fa0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 -- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c06afa0 msgr2=0x7f8b2c1a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c06afa0 0x7f8b2c1a0c00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00c920 tx=0x7f8b1c00cde0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 -- 172.21.15.32:0/840964452 shutdown_connections 2024-04-16T05:40:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c10ea10 0x7f8b2c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8b180a3d20 0x7f8b180a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2c06afa0 0x7f8b2c1a0c00 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 --2- 172.21.15.32:0/840964452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b2c06a680 0x7f8b2c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 -- 172.21.15.32:0/840964452 >> 172.21.15.32:0/840964452 conn(0x7f8b2c077f60 msgr2=0x7f8b2c10fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 -- 172.21.15.32:0/840964452 shutdown_connections 2024-04-16T05:40:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.591+0000 7f8b33488640 1 -- 172.21.15.32:0/840964452 wait complete. 2024-04-16T05:40:27.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:27.609 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:40:27.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.718+0000 7f0fbef9d640 1 -- 172.21.15.32:0/4051718085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb81019b0 msgr2=0x7f0fb81059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.718+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/4051718085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb81019b0 0x7f0fb81059f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009900 tx=0x7f0fa802bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:27.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.718+0000 7f0fbef9d640 1 -- 172.21.15.32:0/4051718085 shutdown_connections 2024-04-16T05:40:27.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.718+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/4051718085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fb81060d0 0x7f0fb810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.718+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/4051718085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb81019b0 0x7f0fb81059f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/4051718085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fb8101000 0x7f0fb81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 -- 172.21.15.32:0/4051718085 >> 172.21.15.32:0/4051718085 conn(0x7f0fb80fc880 msgr2=0x7f0fb80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 -- 172.21.15.32:0/4051718085 shutdown_connections 2024-04-16T05:40:27.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 -- 172.21.15.32:0/4051718085 wait complete. 2024-04-16T05:40:27.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 Processor -- start 2024-04-16T05:40:27.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 -- start start 2024-04-16T05:40:27.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fb8101000 0x7f0fb819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fb81060d0 0x7f0fb819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb8198660 0x7f0fb8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81122a0 con 0x7f0fb8198660 2024-04-16T05:40:27.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8112120 con 0x7f0fb8101000 2024-04-16T05:40:27.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbef9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8112420 con 0x7f0fb81060d0 2024-04-16T05:40:27.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.719+0000 7f0fbe79c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb8198660 0x7f0fb8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbe79c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb8198660 0x7f0fb8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50284/0 (socket says 172.21.15.32:50284) 2024-04-16T05:40:27.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbe79c640 1 -- 172.21.15.32:0/3850549978 learned_addr learned my addr 172.21.15.32:0/3850549978 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fb8101000 0x7f0fb819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fb81060d0 0x7f0fb819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbe79c640 1 -- 172.21.15.32:0/3850549978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fb81060d0 msgr2=0x7f0fb819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbe79c640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fb81060d0 0x7f0fb819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbe79c640 1 -- 172.21.15.32:0/3850549978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fb8101000 msgr2=0x7f0fb819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbe79c640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fb8101000 0x7f0fb819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbe79c640 1 -- 172.21.15.32:0/3850549978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb8199260 con 0x7f0fb8198660 2024-04-16T05:40:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fb8101000 0x7f0fb819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fb81060d0 0x7f0fb819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbe79c640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb8198660 0x7f0fb8198ae0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c970 tx=0x7f0fb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0faeffd640 1 -- 172.21.15.32:0/3850549978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0007bb0 con 0x7f0fb8198660 2024-04-16T05:40:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0faeffd640 1 -- 172.21.15.32:0/3850549978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb0007d50 con 0x7f0fb8198660 2024-04-16T05:40:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbef9d640 1 -- 172.21.15.32:0/3850549978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb81a5340 con 0x7f0fb8198660 2024-04-16T05:40:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0faeffd640 1 -- 172.21.15.32:0/3850549978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000f450 con 0x7f0fb8198660 2024-04-16T05:40:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.720+0000 7f0fbef9d640 1 -- 172.21.15.32:0/3850549978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb81a5740 con 0x7f0fb8198660 2024-04-16T05:40:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.722+0000 7f0faeffd640 1 -- 172.21.15.32:0/3850549978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0fb0004f60 con 0x7f0fb8198660 2024-04-16T05:40:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fc2dee740 1 Processor -- start 2024-04-16T05:40:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fc2dee740 1 -- start start 2024-04-16T05:40:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79140430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140970 0x559e79149010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 0x559e7914b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559e790a5180 con 0x559e79140050 2024-04-16T05:40:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559e79042360 con 0x559e79140970 2024-04-16T05:40:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e790a1140 con 0x559e79149550 2024-04-16T05:40:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0faeffd640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0fa40a3da0 0x7f0fa40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fbd79a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fbe79c640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0fa40a3da0 0x7f0fa40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fbd79a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79140430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50296/0 (socket says 172.21.15.32:50296) 2024-04-16T05:40:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.723+0000 7f0fbd79a640 1 -- 172.21.15.32:0/651253910 learned_addr learned my addr 172.21.15.32:0/651253910 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 0x559e7914b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140970 0x559e79149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0faeffd640 1 -- 172.21.15.32:0/3850549978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f0fb00d0880 con 0x7f0fb8198660 2024-04-16T05:40:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbd79a640 1 -- 172.21.15.32:0/651253910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 msgr2=0x559e7914b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 0x559e7914b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbd79a640 1 -- 172.21.15.32:0/651253910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140970 msgr2=0x559e79149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140970 0x559e79149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbd79a640 1 -- 172.21.15.32:0/651253910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e78e817d0 con 0x559e79140050 2024-04-16T05:40:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140970 0x559e79149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79140430 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8005ec0 tx=0x7f0fa8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92632a530f87183 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fbe79c640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0fa40a3da0 0x7f0fa40a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0098e50 tx=0x7f0fb00987a0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fa3fff640 1 -- 172.21.15.32:0/651253910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8004020 con 0x559e79140050 2024-04-16T05:40:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fa3fff640 1 -- 172.21.15.32:0/651253910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa8005520 con 0x559e79140050 2024-04-16T05:40:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.724+0000 7f0fa3fff640 1 -- 172.21.15.32:0/651253910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8004c80 con 0x559e79140050 2024-04-16T05:40:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 -- 172.21.15.32:0/651253910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 msgr2=0x559e79140430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79140430 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8005ec0 tx=0x7f0fa8005b50 comp rx=0 tx=0).stop 2024-04-16T05:40:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 -- 172.21.15.32:0/651253910 shutdown_connections 2024-04-16T05:40:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 0x559e7914b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140970 0x559e79149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/651253910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79140430 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 -- 172.21.15.32:0/651253910 >> 172.21.15.32:0/651253910 conn(0x559e7913a780 msgr2=0x559e7913ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 -- 172.21.15.32:0/651253910 shutdown_connections 2024-04-16T05:40:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 -- 172.21.15.32:0/651253910 wait complete. 2024-04-16T05:40:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 -- start start 2024-04-16T05:40:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.725+0000 7f0fc2dee740 1 Processor -- start 2024-04-16T05:40:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 0x559e791465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140970 0x559e79146af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 0x559e79147030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559e7914c280 con 0x559e79140970 2024-04-16T05:40:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559e7905e2c0 con 0x559e79140050 2024-04-16T05:40:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e7914c400 con 0x559e79149550 2024-04-16T05:40:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fc2dee740 10 client.0 _ll_register_callbacks cb 0x559e79045090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140970 0x559e79146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140970 0x559e79146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50304/0 (socket says 172.21.15.32:50304) 2024-04-16T05:40:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 -- 172.21.15.32:0/1590683777 learned_addr learned my addr 172.21.15.32:0/1590683777 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 0x559e791465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 0x559e79147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 -- 172.21.15.32:0/1590683777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 msgr2=0x559e79147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 0x559e79147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 -- 172.21.15.32:0/1590683777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 msgr2=0x559e791465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 0x559e791465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 -- 172.21.15.32:0/1590683777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e78db73f0 con 0x559e79140970 2024-04-16T05:40:27.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140970 0x559e79146af0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4012720 tx=0x7f0fb4012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb3f455180dc5f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fa17fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:27.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fa17fa640 1 -- 172.21.15.32:0/1590683777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb401f060 con 0x559e79140970 2024-04-16T05:40:27.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.726+0000 7f0fa17fa640 1 -- 172.21.15.32:0/1590683777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f0fb4010d60 con 0x559e79140970 2024-04-16T05:40:27.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 1 -- 172.21.15.32:0/1590683777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fa8005520 con 0x559e79140970 2024-04-16T05:40:27.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fa17fa640 1 -- 172.21.15.32:0/1590683777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0fb4026050 con 0x559e79140970 2024-04-16T05:40:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 10 client.17025 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 1 -- 172.21.15.32:0/1590683777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e78d770f0 con 0x559e79140970 2024-04-16T05:40:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fa17fa640 1 -- 172.21.15.32:0/1590683777 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa8005520 con 0x559e79140970 2024-04-16T05:40:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fa17fa640 1 -- 172.21.15.32:0/1590683777 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0fb4019030 con 0x559e79140970 2024-04-16T05:40:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 10 client.17025 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 20 client.17025 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559e791598a0 0x559e7915bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 4 client.17025 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:27.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fc2dee740 1 -- 172.21.15.32:0/1590683777 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e790b2ba0 con 0x559e791598a0 2024-04-16T05:40:27.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.727+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559e791598a0 0x559e7915bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.728+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559e791598a0 0x559e7915bcd0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24dd6babf653bad server_cookie=d426dbc2b5b3b979 in_seq=0 out_seq=0 2024-04-16T05:40:27.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.728+0000 7f0fa17fa640 10 client.17025 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:27.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.728+0000 7f0fa17fa640 1 -- 172.21.15.32:0/1590683777 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e790b2ba0 con 0x559e791598a0 2024-04-16T05:40:27.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.728+0000 7f0fa17fa640 10 client.17025 handle_command_reply: tid=0 2024-04-16T05:40:27.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.728+0000 7f0fc2dee740 1 client.17025 shutdown 2024-04-16T05:40:27.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 -- 172.21.15.32:0/1590683777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140970 msgr2=0x559e79146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140970 0x559e79146af0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4012720 tx=0x7f0fb4012be0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 -- 172.21.15.32:0/1590683777 shutdown_connections 2024-04-16T05:40:27.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559e791598a0 0x559e7915bcd0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79149550 0x559e79147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140970 0x559e79146af0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/1590683777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 0x559e791465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fa17fa640 0 client.17025 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:27.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 -- 172.21.15.32:0/1590683777 >> 172.21.15.32:0/1590683777 conn(0x559e7913a780 msgr2=0x559e7913c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 -- 172.21.15.32:0/1590683777 shutdown_connections 2024-04-16T05:40:27.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 1 -- 172.21.15.32:0/1590683777 wait complete. 2024-04-16T05:40:27.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.729+0000 7f0fc2dee740 20 client.17025 trim_cache size 0 max 16384 2024-04-16T05:40:27.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:27 smithi032 ceph-mon[32351]: pgmap v168: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fc2dee740 1 Processor -- start 2024-04-16T05:40:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fc2dee740 1 -- start start 2024-04-16T05:40:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 0x559e79155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79149550 0x559e7913c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81b1a10 con 0x559e79140050 2024-04-16T05:40:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81af2e0 con 0x559e79149550 2024-04-16T05:40:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81ad130 con 0x559e79140970 2024-04-16T05:40:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fbd79a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fbd79a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79148ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50320/0 (socket says 172.21.15.32:50320) 2024-04-16T05:40:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fbd79a640 1 -- 172.21.15.32:0/2163865075 learned_addr learned my addr 172.21.15.32:0/2163865075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.747+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/2163865075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 0x559e79155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fbd79a640 1 -- 172.21.15.32:0/2163865075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 msgr2=0x559e79155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/2163865075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 0x559e79155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fbd79a640 1 -- 172.21.15.32:0/2163865075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79149550 msgr2=0x559e7913c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/2163865075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79149550 0x559e7913c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fbd79a640 1 -- 172.21.15.32:0/2163865075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e79142180 con 0x559e79140050 2024-04-16T05:40:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/2163865075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79148ce0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000c00 tx=0x7f0fa802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbc6b69f770a99f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fa37fe640 1 -- 172.21.15.32:0/2163865075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8059070 con 0x559e79140050 2024-04-16T05:40:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fa37fe640 1 -- 172.21.15.32:0/2163865075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa80546f0 con 0x559e79140050 2024-04-16T05:40:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.748+0000 7f0fa37fe640 1 -- 172.21.15.32:0/2163865075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8054070 con 0x559e79140050 2024-04-16T05:40:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- 172.21.15.32:0/2163865075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 msgr2=0x559e79148ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/2163865075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79148ce0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000c00 tx=0x7f0fa802bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- 172.21.15.32:0/2163865075 shutdown_connections 2024-04-16T05:40:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/2163865075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79149550 0x559e7913c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/2163865075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 0x559e79155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/2163865075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79140050 0x559e79148ce0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- 172.21.15.32:0/2163865075 >> 172.21.15.32:0/2163865075 conn(0x559e7913a780 msgr2=0x559e7913bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- 172.21.15.32:0/2163865075 shutdown_connections 2024-04-16T05:40:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- 172.21.15.32:0/2163865075 wait complete. 2024-04-16T05:40:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- start start 2024-04-16T05:40:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 Processor -- start 2024-04-16T05:40:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 0x559e7914f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 0x559e7914f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79149550 0x559e791bee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81ad7a0 con 0x559e79149550 2024-04-16T05:40:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81aec10 con 0x559e79140050 2024-04-16T05:40:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81ade70 con 0x559e79140970 2024-04-16T05:40:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.749+0000 7f0fc2dee740 10 client.0 _ll_register_callbacks cb 0x7f0fb4009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79149550 0x559e791bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79149550 0x559e791bee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50334/0 (socket says 172.21.15.32:50334) 2024-04-16T05:40:27.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 -- 172.21.15.32:0/3587895625 learned_addr learned my addr 172.21.15.32:0/3587895625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:27.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 0x559e7914f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 0x559e7914f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 -- 172.21.15.32:0/3587895625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 msgr2=0x559e7914f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 0x559e7914f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 -- 172.21.15.32:0/3587895625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 msgr2=0x559e7914f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:27.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 0x559e7914f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 -- 172.21.15.32:0/3587895625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e791bf3a0 con 0x559e79149550 2024-04-16T05:40:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fa17fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fbd79a640 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79149550 0x559e791bee60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0fa802c820 tx=0x7f0fa8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd07f8c038d4a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fa17fa640 1 -- 172.21.15.32:0/3587895625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8059030 con 0x559e79149550 2024-04-16T05:40:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fa17fa640 1 -- 172.21.15.32:0/3587895625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f0fa80042e0 con 0x559e79149550 2024-04-16T05:40:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fc2dee740 1 -- 172.21.15.32:0/3587895625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fa80546f0 con 0x559e79149550 2024-04-16T05:40:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fa17fa640 1 -- 172.21.15.32:0/3587895625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0fa806b050 con 0x559e79149550 2024-04-16T05:40:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fc2dee740 10 client.17031 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fc2dee740 1 -- 172.21.15.32:0/3587895625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e791c0550 con 0x559e79149550 2024-04-16T05:40:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.750+0000 7f0fa17fa640 1 -- 172.21.15.32:0/3587895625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa80546f0 con 0x559e79149550 2024-04-16T05:40:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fa17fa640 1 -- 172.21.15.32:0/3587895625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0fa8003ab0 con 0x559e79149550 2024-04-16T05:40:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fc2dee740 10 client.17031 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fc2dee740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fc2dee740 20 client.17031 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559e791c0720 0x559e791c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fc2dee740 4 client.17031 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:40:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fc2dee740 1 -- 172.21.15.32:0/3587895625 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e790aca10 con 0x559e791c0720 2024-04-16T05:40:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559e791c0720 0x559e791c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fbdf9b640 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559e791c0720 0x559e791c2b10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642caadcd059e83c server_cookie=ace03f6439453d6c in_seq=0 out_seq=0 2024-04-16T05:40:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.751+0000 7f0fa17fa640 10 client.17031 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fa17fa640 1 -- 172.21.15.32:0/3587895625 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559e790aca10 con 0x559e791c0720 2024-04-16T05:40:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fa17fa640 10 client.17031 handle_command_reply: tid=0 2024-04-16T05:40:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 client.17031 shutdown 2024-04-16T05:40:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 -- 172.21.15.32:0/3587895625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79149550 msgr2=0x559e791bee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79149550 0x559e791bee60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0fa802c820 tx=0x7f0fa8005c90 comp rx=0 tx=0).stop 2024-04-16T05:40:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 -- 172.21.15.32:0/3587895625 shutdown_connections 2024-04-16T05:40:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e79149550 0x559e791bee60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x559e791c0720 0x559e791c2b10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e79140970 0x559e7914f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 --2- 172.21.15.32:0/3587895625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e79140050 0x559e7914f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 -- 172.21.15.32:0/3587895625 >> 172.21.15.32:0/3587895625 conn(0x559e7913a780 msgr2=0x559e7913c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fa17fa640 0 client.17031 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 -- 172.21.15.32:0/3587895625 shutdown_connections 2024-04-16T05:40:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 1 -- 172.21.15.32:0/3587895625 wait complete. 2024-04-16T05:40:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fc2dee740 20 client.17031 trim_cache size 0 max 16384 2024-04-16T05:40:27.764 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:40:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:40:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:40:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "db622bc2-b59d-40a8-9f1a-3927dfab4f8e": { 2024-04-16T05:40:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:40:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "db622bc2-b59d-40a8-9f1a-3927dfab4f8e", 2024-04-16T05:40:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:40:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:40:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:40:27.765 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:40:27.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fbef9d640 1 -- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0fa40a3da0 msgr2=0x7f0fa40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.752+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0fa40a3da0 0x7f0fa40a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0098e50 tx=0x7f0fb00987a0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 -- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb8198660 msgr2=0x7f0fb8198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb8198660 0x7f0fb8198ae0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c970 tx=0x7f0fb000ce30 comp rx=0 tx=0).stop 2024-04-16T05:40:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 -- 172.21.15.32:0/3850549978 shutdown_connections 2024-04-16T05:40:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0fa40a3da0 0x7f0fa40a6260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb8198660 0x7f0fb8198ae0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fb81060d0 0x7f0fb819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 --2- 172.21.15.32:0/3850549978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fb8101000 0x7f0fb819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 -- 172.21.15.32:0/3850549978 >> 172.21.15.32:0/3850549978 conn(0x7f0fb80fc880 msgr2=0x7f0fb80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 -- 172.21.15.32:0/3850549978 shutdown_connections 2024-04-16T05:40:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:27.753+0000 7f0fbef9d640 1 -- 172.21.15.32:0/3850549978 wait complete. 2024-04-16T05:40:27.768 INFO:tasks.cephfs.filesystem:scrub status for tag:db622bc2-b59d-40a8-9f1a-3927dfab4f8e - {'path': '/', 'tag': 'db622bc2-b59d-40a8-9f1a-3927dfab4f8e', 'options': 'recursive,force'} 2024-04-16T05:40:27.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:27 smithi134 ceph-mon[36153]: pgmap v168: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:27.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:27 smithi082 ceph-mon[36195]: pgmap v168: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:28.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:28 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:40:28.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:28 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:28.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:28 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:40:28.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:28 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:28 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:40:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:28 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:28 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:40:28.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:28 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:28 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:40:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:28 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:28 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:40:28.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:28 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:40:29.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:29 smithi134 ceph-mon[36153]: pgmap v169: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:40:29.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:29 smithi082 ceph-mon[36195]: pgmap v169: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:40:29.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:29 smithi032 ceph-mon[32351]: pgmap v169: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:40:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:31 smithi134 ceph-mon[36153]: pgmap v170: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:31.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:31 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:40:31.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:31 smithi082 ceph-mon[36195]: pgmap v170: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:31.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:31 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:40:31.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:31 smithi032 ceph-mon[32351]: pgmap v170: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:31.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:31 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:40:33.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:33 smithi134 ceph-mon[36153]: pgmap v171: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-16T05:40:33.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:33 smithi082 ceph-mon[36195]: pgmap v171: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-16T05:40:33.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:33 smithi032 ceph-mon[32351]: pgmap v171: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-16T05:40:35.660 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:35 smithi134 ceph-mon[36153]: pgmap v172: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-16T05:40:35.900 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:35 smithi082 ceph-mon[36195]: pgmap v172: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-16T05:40:35.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:35 smithi032 ceph-mon[32351]: pgmap v172: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-16T05:40:37.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:37 smithi134 ceph-mon[36153]: pgmap v173: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-16T05:40:37.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:37 smithi082 ceph-mon[36195]: pgmap v173: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-16T05:40:37.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:37 smithi032 ceph-mon[32351]: pgmap v173: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-04-16T05:40:39.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:39 smithi032 ceph-mon[32351]: pgmap v174: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-16T05:40:39.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:39 smithi134 ceph-mon[36153]: pgmap v174: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-16T05:40:39.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:39 smithi082 ceph-mon[36195]: pgmap v174: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-16T05:40:41.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:41 smithi134 ceph-mon[36153]: pgmap v175: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:41.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:41 smithi082 ceph-mon[36195]: pgmap v175: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:41.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:41 smithi032 ceph-mon[32351]: pgmap v175: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:40:44.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:43 smithi134 ceph-mon[36153]: pgmap v176: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:40:44.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:43 smithi082 ceph-mon[36195]: pgmap v176: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:40:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:43 smithi032 ceph-mon[32351]: pgmap v176: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:40:46.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:45 smithi134 ceph-mon[36153]: pgmap v177: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:46.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:45 smithi082 ceph-mon[36195]: pgmap v177: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:46.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:45 smithi032 ceph-mon[32351]: pgmap v177: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:48.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:47 smithi134 ceph-mon[36153]: pgmap v178: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:48.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:47 smithi082 ceph-mon[36195]: pgmap v178: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:48.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:47 smithi032 ceph-mon[32351]: pgmap v178: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:49 smithi134 ceph-mon[36153]: pgmap v179: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:50.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:49 smithi082 ceph-mon[36195]: pgmap v179: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:50.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:49 smithi032 ceph-mon[32351]: pgmap v179: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:52.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:51 smithi134 ceph-mon[36153]: pgmap v180: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:52.175 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:51 smithi082 ceph-mon[36195]: pgmap v180: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:52.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:51 smithi032 ceph-mon[32351]: pgmap v180: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:53 smithi134 ceph-mon[36153]: pgmap v181: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:54.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:53 smithi082 ceph-mon[36195]: pgmap v181: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:54.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:53 smithi032 ceph-mon[32351]: pgmap v181: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:40:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:55 smithi134 ceph-mon[36153]: pgmap v182: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:56.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:55 smithi082 ceph-mon[36195]: pgmap v182: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:56.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:55 smithi032 ceph-mon[32351]: pgmap v182: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:57.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:57.769 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:40:57.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 -- 172.21.15.32:0/291975690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c4101bf0 msgr2=0x7f65c410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/291975690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c4101bf0 0x7f65c410dd30 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009cd0 tx=0x7f65bc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 -- 172.21.15.32:0/291975690 shutdown_connections 2024-04-16T05:40:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/291975690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c4101bf0 0x7f65c410dd30 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/291975690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c4101250 0x7f65c41016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/291975690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c41070e0 0x7f65c41074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 -- 172.21.15.32:0/291975690 >> 172.21.15.32:0/291975690 conn(0x7f65c4076d00 msgr2=0x7f65c4077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 -- 172.21.15.32:0/291975690 shutdown_connections 2024-04-16T05:40:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.891+0000 7f65c8d7c640 1 -- 172.21.15.32:0/291975690 wait complete. 2024-04-16T05:40:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c8d7c640 1 Processor -- start 2024-04-16T05:40:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c8d7c640 1 -- start start 2024-04-16T05:40:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c8d7c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c4101250 0x7f65c410e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c8d7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c4101bf0 0x7f65c410ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c8d7c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c41070e0 0x7f65c410f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c8d7c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4112460 con 0x7f65c41070e0 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c8d7c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41122e0 con 0x7f65c4101250 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c8d7c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41125e0 con 0x7f65c4101bf0 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c41070e0 0x7f65c410f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c41070e0 0x7f65c410f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55022/0 (socket says 172.21.15.32:55022) 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.892+0000 7f65c2d9d640 1 -- 172.21.15.32:0/1058060809 learned_addr learned my addr 172.21.15.32:0/1058060809 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c4101bf0 0x7f65c410ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c259c640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c4101250 0x7f65c410e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c2d9d640 1 -- 172.21.15.32:0/1058060809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c4101bf0 msgr2=0x7f65c410ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c2d9d640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c4101bf0 0x7f65c410ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c2d9d640 1 -- 172.21.15.32:0/1058060809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c4101250 msgr2=0x7f65c410e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c2d9d640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c4101250 0x7f65c410e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c2d9d640 1 -- 172.21.15.32:0/1058060809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c41a6d60 con 0x7f65c41070e0 2024-04-16T05:40:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c259c640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c4101250 0x7f65c410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c4101bf0 0x7f65c410ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c2d9d640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c41070e0 0x7f65c410f2d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0348f0 tx=0x7f65bc032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65b37fe640 1 -- 172.21.15.32:0/1058060809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc041020 con 0x7f65c41070e0 2024-04-16T05:40:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65b37fe640 1 -- 172.21.15.32:0/1058060809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65bc034cd0 con 0x7f65c41070e0 2024-04-16T05:40:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c8d7c640 1 -- 172.21.15.32:0/1058060809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c41a6ff0 con 0x7f65c41070e0 2024-04-16T05:40:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65b37fe640 1 -- 172.21.15.32:0/1058060809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc039040 con 0x7f65c41070e0 2024-04-16T05:40:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.893+0000 7f65c8d7c640 1 -- 172.21.15.32:0/1058060809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41a74b0 con 0x7f65c41070e0 2024-04-16T05:40:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.896+0000 7f65cb5fd740 1 Processor -- start 2024-04-16T05:40:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.896+0000 7f65b37fe640 1 -- 172.21.15.32:0/1058060809 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65bc044020 con 0x7f65c41070e0 2024-04-16T05:40:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65cb5fd740 1 -- start start 2024-04-16T05:40:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f5e50 0x5615a03f6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a03fee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a0401740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5615a035b180 con 0x5615a03f5e50 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5615a02f8360 con 0x5615a03f6770 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5615a0357140 con 0x5615a03ff350 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f5e50 0x5615a03f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f5e50 0x5615a03f6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55030/0 (socket says 172.21.15.32:55030) 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 -- 172.21.15.32:0/137035247 learned_addr learned my addr 172.21.15.32:0/137035247 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c259c640 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a03fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a0401740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 -- 172.21.15.32:0/137035247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 msgr2=0x5615a0401740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65b37fe640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f65ac0a3d50 0x7f65ac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a0401740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 -- 172.21.15.32:0/137035247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 msgr2=0x5615a03fee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a03fee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.897+0000 7f65c1d9b640 1 -- 172.21.15.32:0/137035247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615a01377d0 con 0x5615a03f5e50 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.898+0000 7f65c2d9d640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f65ac0a3d50 0x7f65ac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.898+0000 7f65b37fe640 1 -- 172.21.15.32:0/1058060809 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f65bc0f2940 con 0x7f65c41070e0 2024-04-16T05:40:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.898+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f5e50 0x5615a03f6230 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f65b800be80 tx=0x7f65b800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd27a42bf6a0bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.898+0000 7f65b0ff9640 1 -- 172.21.15.32:0/137035247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8019030 con 0x5615a03f5e50 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.898+0000 7f65b0ff9640 1 -- 172.21.15.32:0/137035247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b8011a20 con 0x5615a03f5e50 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.898+0000 7f65b0ff9640 1 -- 172.21.15.32:0/137035247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b801d920 con 0x5615a03f5e50 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 -- 172.21.15.32:0/137035247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f5e50 msgr2=0x5615a03f6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65c2d9d640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f65ac0a3d50 0x7f65ac0a6210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009430 tx=0x7f65bc02e040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f5e50 0x5615a03f6230 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f65b800be80 tx=0x7f65b800b730 comp rx=0 tx=0).stop 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 -- 172.21.15.32:0/137035247 shutdown_connections 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a0401740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a03fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/137035247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f5e50 0x5615a03f6230 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 -- 172.21.15.32:0/137035247 >> 172.21.15.32:0/137035247 conn(0x5615a03f0580 msgr2=0x5615a03f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 -- 172.21.15.32:0/137035247 shutdown_connections 2024-04-16T05:40:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 -- 172.21.15.32:0/137035247 wait complete. 2024-04-16T05:40:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 -- start start 2024-04-16T05:40:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 Processor -- start 2024-04-16T05:40:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.899+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f5e50 0x5615a03fc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a03fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5615a0402140 con 0x5615a03f6770 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5615a03142c0 con 0x5615a03f5e50 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5615a04022c0 con 0x5615a03ff350 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 10 client.0 _ll_register_callbacks cb 0x5615a02fb090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55046/0 (socket says 172.21.15.32:55046) 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 -- 172.21.15.32:0/870713877 learned_addr learned my addr 172.21.15.32:0/870713877 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a03fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f5e50 0x5615a03fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 -- 172.21.15.32:0/870713877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 msgr2=0x5615a03fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a03fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 -- 172.21.15.32:0/870713877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f5e50 msgr2=0x5615a03fc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f5e50 0x5615a03fc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 -- 172.21.15.32:0/870713877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615a006d3f0 con 0x5615a03f6770 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a03fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65c259c640 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fc8a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f65b400bf30 tx=0x7f65b40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4216e97d541bbf85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65a9ffb640 1 -- 172.21.15.32:0/870713877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40035f0 con 0x5615a03f6770 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65a9ffb640 1 -- 172.21.15.32:0/870713877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f65b4003790 con 0x5615a03f6770 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 1 -- 172.21.15.32:0/870713877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b8011a20 con 0x5615a03f6770 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65a9ffb640 1 -- 172.21.15.32:0/870713877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65b4003990 con 0x5615a03f6770 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 10 client.17043 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.900+0000 7f65cb5fd740 1 -- 172.21.15.32:0/870713877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5615a002d0f0 con 0x5615a03f6770 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65a9ffb640 1 -- 172.21.15.32:0/870713877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b8011a20 con 0x5615a03f6770 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65a9ffb640 1 -- 172.21.15.32:0/870713877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f65b4019070 con 0x5615a03f6770 2024-04-16T05:40:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65cb5fd740 10 client.17043 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65cb5fd740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65cb5fd740 20 client.17043 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5615a040f740 0x5615a0411bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65cb5fd740 4 client.17043 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65cb5fd740 1 -- 172.21.15.32:0/870713877 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615a0368ba0 con 0x5615a040f740 2024-04-16T05:40:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.901+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5615a040f740 0x5615a0411bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65a9ffb640 10 client.17043 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5615a040f740 0x5615a0411bc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db3f8cb8e5763a0 server_cookie=8aa7e0bf7c8b4fa0 in_seq=0 out_seq=0 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65a9ffb640 1 -- 172.21.15.32:0/870713877 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615a0368ba0 con 0x5615a040f740 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65a9ffb640 10 client.17043 handle_command_reply: tid=0 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 client.17043 shutdown 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 -- 172.21.15.32:0/870713877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 msgr2=0x5615a03fc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fc8a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f65b400bf30 tx=0x7f65b40128e0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 -- 172.21.15.32:0/870713877 shutdown_connections 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5615a040f740 0x5615a0411bc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03ff350 0x5615a03fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fc8a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/870713877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f5e50 0x5615a03fc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65a9ffb640 0 client.17043 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.902+0000 7f65cb5fd740 1 -- 172.21.15.32:0/870713877 >> 172.21.15.32:0/870713877 conn(0x5615a03f0580 msgr2=0x5615a03f22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.903+0000 7f65cb5fd740 1 -- 172.21.15.32:0/870713877 shutdown_connections 2024-04-16T05:40:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.903+0000 7f65cb5fd740 1 -- 172.21.15.32:0/870713877 wait complete. 2024-04-16T05:40:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.903+0000 7f65cb5fd740 20 client.17043 trim_cache size 0 max 16384 2024-04-16T05:40:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65cb5fd740 1 Processor -- start 2024-04-16T05:40:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65cb5fd740 1 -- start start 2024-04-16T05:40:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 0x5615a03fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03ff350 0x5615a040b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b3e70 con 0x5615a03f6770 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b1740 con 0x5615a03ff350 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41af590 con 0x5615a03f5e50 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.920+0000 7f65c259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55062/0 (socket says 172.21.15.32:55062) 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c259c640 1 -- 172.21.15.32:0/788100699 learned_addr learned my addr 172.21.15.32:0/788100699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03ff350 0x5615a040b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 0x5615a03fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c259c640 1 -- 172.21.15.32:0/788100699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 msgr2=0x5615a03fea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c259c640 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 0x5615a03fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c259c640 1 -- 172.21.15.32:0/788100699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03ff350 msgr2=0x5615a040b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c259c640 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03ff350 0x5615a040b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c259c640 1 -- 172.21.15.32:0/788100699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615a03fa8b0 con 0x5615a03f6770 2024-04-16T05:40:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03ff350 0x5615a040b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65c259c640 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fb2a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f65b4010e80 tx=0x7f65b4005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102f1e544a24542e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65abfff640 1 -- 172.21.15.32:0/788100699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b402d020 con 0x5615a03f6770 2024-04-16T05:40:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65abfff640 1 -- 172.21.15.32:0/788100699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4004480 con 0x5615a03f6770 2024-04-16T05:40:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.921+0000 7f65abfff640 1 -- 172.21.15.32:0/788100699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40106f0 con 0x5615a03f6770 2024-04-16T05:40:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- 172.21.15.32:0/788100699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 msgr2=0x5615a03fb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fb2a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f65b4010e80 tx=0x7f65b4005870 comp rx=0 tx=0).stop 2024-04-16T05:40:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- 172.21.15.32:0/788100699 shutdown_connections 2024-04-16T05:40:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03ff350 0x5615a040b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03f6770 0x5615a03fb2a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/788100699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 0x5615a03fea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- 172.21.15.32:0/788100699 >> 172.21.15.32:0/788100699 conn(0x5615a03f0580 msgr2=0x5615a03ef760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- 172.21.15.32:0/788100699 shutdown_connections 2024-04-16T05:40:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- 172.21.15.32:0/788100699 wait complete. 2024-04-16T05:40:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- start start 2024-04-16T05:40:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 Processor -- start 2024-04-16T05:40:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 0x5615a0407330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a0407870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03ff350 0x5615a04057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41afc00 con 0x5615a03ff350 2024-04-16T05:40:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b1070 con 0x5615a03f6770 2024-04-16T05:40:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b02d0 con 0x5615a03f5e50 2024-04-16T05:40:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65cb5fd740 10 client.0 _ll_register_callbacks cb 0x7f65b400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65c1d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03ff350 0x5615a04057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65c1d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03ff350 0x5615a04057d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55070/0 (socket says 172.21.15.32:55070) 2024-04-16T05:40:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.922+0000 7f65c1d9b640 1 -- 172.21.15.32:0/169254484 learned_addr learned my addr 172.21.15.32:0/169254484 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 0x5615a0407330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c259c640 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a0407870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c1d9b640 1 -- 172.21.15.32:0/169254484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 msgr2=0x5615a0407330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 0x5615a0407330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c1d9b640 1 -- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 msgr2=0x5615a0407870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a0407870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c1d9b640 1 -- 172.21.15.32:0/169254484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615a0405fb0 con 0x5615a03ff350 2024-04-16T05:40:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c259c640 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a0407870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65c1d9b640 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03ff350 0x5615a04057d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f65b801d4d0 tx=0x7f65b8020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e38a97abb39d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65a9ffb640 1 -- 172.21.15.32:0/169254484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8028040 con 0x5615a03ff350 2024-04-16T05:40:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65a9ffb640 1 -- 172.21.15.32:0/169254484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f65b8013040 con 0x5615a03ff350 2024-04-16T05:40:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65cb5fd740 1 -- 172.21.15.32:0/169254484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b4004480 con 0x5615a03ff350 2024-04-16T05:40:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65a9ffb640 1 -- 172.21.15.32:0/169254484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65b8019040 con 0x5615a03ff350 2024-04-16T05:40:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65cb5fd740 10 client.17049 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65cb5fd740 1 -- 172.21.15.32:0/169254484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5615a0476780 con 0x5615a03ff350 2024-04-16T05:40:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.923+0000 7f65a9ffb640 1 -- 172.21.15.32:0/169254484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4004480 con 0x5615a03ff350 2024-04-16T05:40:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65a9ffb640 1 -- 172.21.15.32:0/169254484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f65b8011660 con 0x5615a03ff350 2024-04-16T05:40:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65cb5fd740 10 client.17049 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65cb5fd740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65cb5fd740 20 client.17049 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5615a0476d60 0x5615a0479180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65cb5fd740 4 client.17049 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:40:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65cb5fd740 1 -- 172.21.15.32:0/169254484 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615a0362a10 con 0x5615a0476d60 2024-04-16T05:40:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65c259c640 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5615a0476d60 0x5615a0479180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65a9ffb640 10 client.17049 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.924+0000 7f65c259c640 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5615a0476d60 0x5615a0479180 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4f6201441adfaa server_cookie=d57cd8ed191b66fb in_seq=0 out_seq=0 2024-04-16T05:40:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65a9ffb640 1 -- 172.21.15.32:0/169254484 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5615a0362a10 con 0x5615a0476d60 2024-04-16T05:40:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65a9ffb640 10 client.17049 handle_command_reply: tid=0 2024-04-16T05:40:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 client.17049 shutdown 2024-04-16T05:40:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 -- 172.21.15.32:0/169254484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03ff350 msgr2=0x5615a04057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03ff350 0x5615a04057d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f65b801d4d0 tx=0x7f65b8020040 comp rx=0 tx=0).stop 2024-04-16T05:40:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 -- 172.21.15.32:0/169254484 shutdown_connections 2024-04-16T05:40:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615a03ff350 0x5615a04057d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615a03f6770 0x5615a0407870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5615a0476d60 0x5615a0479180 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 --2- 172.21.15.32:0/169254484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615a03f5e50 0x5615a0407330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65a9ffb640 0 client.17049 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 -- 172.21.15.32:0/169254484 >> 172.21.15.32:0/169254484 conn(0x5615a03f0580 msgr2=0x5615a03fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 -- 172.21.15.32:0/169254484 shutdown_connections 2024-04-16T05:40:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 1 -- 172.21.15.32:0/169254484 wait complete. 2024-04-16T05:40:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65cb5fd740 20 client.17049 trim_cache size 0 max 16384 2024-04-16T05:40:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:40:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:40:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:40:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:40:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65c8d7c640 1 -- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f65ac0a3d50 msgr2=0x7f65ac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f65ac0a3d50 0x7f65ac0a6210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009430 tx=0x7f65bc02e040 comp rx=0 tx=0).stop 2024-04-16T05:40:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65c8d7c640 1 -- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c41070e0 msgr2=0x7f65c410f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c41070e0 0x7f65c410f2d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0348f0 tx=0x7f65bc032ec0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.925+0000 7f65c8d7c640 1 -- 172.21.15.32:0/1058060809 shutdown_connections 2024-04-16T05:40:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.926+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f65ac0a3d50 0x7f65ac0a6210 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.926+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c41070e0 0x7f65c410f2d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.926+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c4101bf0 0x7f65c410ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.926+0000 7f65c8d7c640 1 --2- 172.21.15.32:0/1058060809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c4101250 0x7f65c410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:57.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.926+0000 7f65c8d7c640 1 -- 172.21.15.32:0/1058060809 >> 172.21.15.32:0/1058060809 conn(0x7f65c4076d00 msgr2=0x7f65c40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:57.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.926+0000 7f65c8d7c640 1 -- 172.21.15.32:0/1058060809 shutdown_connections 2024-04-16T05:40:57.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:57.926+0000 7f65c8d7c640 1 -- 172.21.15.32:0/1058060809 wait complete. 2024-04-16T05:40:57.941 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:40:57.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:40:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 -- 172.21.15.32:0/1614230150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158102240 msgr2=0x7f0158102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 --2- 172.21.15.32:0/1614230150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158102240 0x7f0158102620 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f0148009900 tx=0x7f014802bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 -- 172.21.15.32:0/1614230150 shutdown_connections 2024-04-16T05:40:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 --2- 172.21.15.32:0/1614230150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158107310 0x7f015810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 --2- 172.21.15.32:0/1614230150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158102bf0 0x7f0158106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 --2- 172.21.15.32:0/1614230150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158102240 0x7f0158102620 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 -- 172.21.15.32:0/1614230150 >> 172.21.15.32:0/1614230150 conn(0x7f01580793c0 msgr2=0x7f01580797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 -- 172.21.15.32:0/1614230150 shutdown_connections 2024-04-16T05:40:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.049+0000 7f015d24a640 1 -- 172.21.15.32:0/1614230150 wait complete. 2024-04-16T05:40:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f015d24a640 1 Processor -- start 2024-04-16T05:40:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f015d24a640 1 -- start start 2024-04-16T05:40:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f015d24a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158102240 0x7f015819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f015d24a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158107310 0x7f015819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f015d24a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158198740 0x7f0158198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f015d24a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01581123c0 con 0x7f0158107310 2024-04-16T05:40:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f015d24a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0158112240 con 0x7f0158198740 2024-04-16T05:40:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f015d24a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0158112540 con 0x7f0158102240 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f014e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158107310 0x7f015819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f014e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158107310 0x7f015819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55090/0 (socket says 172.21.15.32:55090) 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.050+0000 7f014e59c640 1 -- 172.21.15.32:0/67932635 learned_addr learned my addr 172.21.15.32:0/67932635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f015759e640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158198740 0x7f0158198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f0156d9d640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158102240 0x7f015819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f014e59c640 1 -- 172.21.15.32:0/67932635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158102240 msgr2=0x7f015819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f014e59c640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158102240 0x7f015819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f014e59c640 1 -- 172.21.15.32:0/67932635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158198740 msgr2=0x7f0158198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f014e59c640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158198740 0x7f0158198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f014e59c640 1 -- 172.21.15.32:0/67932635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0158199340 con 0x7f0158107310 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f014e59c640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158107310 0x7f015819ebc0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f014000d740 tx=0x7f014000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f0154d99640 1 -- 172.21.15.32:0/67932635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140017070 con 0x7f0158107310 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f0154d99640 1 -- 172.21.15.32:0/67932635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f014000de50 con 0x7f0158107310 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01581a5420 con 0x7f0158107310 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f0154d99640 1 -- 172.21.15.32:0/67932635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140004e00 con 0x7f0158107310 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.051+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01581a5820 con 0x7f0158107310 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.052+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0158103710 con 0x7f0158107310 2024-04-16T05:40:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.053+0000 7f0154d99640 1 -- 172.21.15.32:0/67932635 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0140004fa0 con 0x7f0158107310 2024-04-16T05:40:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.054+0000 7f0154d99640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f01440a3da0 0x7f01440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.054+0000 7f0156d9d640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f01440a3da0 0x7f01440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.054+0000 7f0154d99640 1 -- 172.21.15.32:0/67932635 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f01400d04b0 con 0x7f0158107310 2024-04-16T05:40:58.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.054+0000 7f0156d9d640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f01440a3da0 0x7f01440a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0148006010 tx=0x7f0148000bc0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.059+0000 7f0154d99640 1 -- 172.21.15.32:0/67932635 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f014009c7b0 con 0x7f0158107310 2024-04-16T05:40:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:57 smithi134 ceph-mon[36153]: pgmap v183: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:58.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:57 smithi082 ceph-mon[36195]: pgmap v183: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:58.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:57 smithi032 ceph-mon[32351]: pgmap v183: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:40:58.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.287+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0158069270 con 0x7f0158107310 2024-04-16T05:40:58.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.288+0000 7f0154d99640 1 -- 172.21.15.32:0/67932635 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f014009c0b0 con 0x7f0158107310 2024-04-16T05:40:58.287 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:40:58.287 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:40:58.288 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:40:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f01440a3da0 msgr2=0x7f01440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f01440a3da0 0x7f01440a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0148006010 tx=0x7f0148000bc0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158107310 msgr2=0x7f015819ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158107310 0x7f015819ebc0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f014000d740 tx=0x7f014000dc00 comp rx=0 tx=0).stop 2024-04-16T05:40:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 shutdown_connections 2024-04-16T05:40:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158198740 0x7f0158198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f01440a3da0 0x7f01440a6260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0158107310 0x7f015819ebc0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 --2- 172.21.15.32:0/67932635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158102240 0x7f015819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 >> 172.21.15.32:0/67932635 conn(0x7f01580793c0 msgr2=0x7f0158100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 shutdown_connections 2024-04-16T05:40:58.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.290+0000 7f015d24a640 1 -- 172.21.15.32:0/67932635 wait complete. 2024-04-16T05:40:58.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:40:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 -- 172.21.15.32:0/885837171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c806a990 msgr2=0x7f83c806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 --2- 172.21.15.32:0/885837171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c806a990 0x7f83c806ad70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009900 tx=0x7f83b802bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 -- 172.21.15.32:0/885837171 shutdown_connections 2024-04-16T05:40:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 --2- 172.21.15.32:0/885837171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c81073c0 0x7f83c810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 --2- 172.21.15.32:0/885837171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806b340 0x7f83c8106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 --2- 172.21.15.32:0/885837171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c806a990 0x7f83c806ad70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 -- 172.21.15.32:0/885837171 >> 172.21.15.32:0/885837171 conn(0x7f83c8077280 msgr2=0x7f83c8077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 -- 172.21.15.32:0/885837171 shutdown_connections 2024-04-16T05:40:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.408+0000 7f83cda2a640 1 -- 172.21.15.32:0/885837171 wait complete. 2024-04-16T05:40:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83cda2a640 1 Processor -- start 2024-04-16T05:40:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83cda2a640 1 -- start start 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83cda2a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806a990 0x7f83c819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83cda2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c81073c0 0x7f83c819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83cda2a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c8198640 0x7f83c8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83cda2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8112420 con 0x7f83c81073c0 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83cda2a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81122a0 con 0x7f83c806a990 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83cda2a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81125a0 con 0x7f83c8198640 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.409+0000 7f83c67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c81073c0 0x7f83c819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c81073c0 0x7f83c819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55104/0 (socket says 172.21.15.32:55104) 2024-04-16T05:40:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c67fc640 1 -- 172.21.15.32:0/537271260 learned_addr learned my addr 172.21.15.32:0/537271260 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c77fe640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c8198640 0x7f83c8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806a990 0x7f83c819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c67fc640 1 -- 172.21.15.32:0/537271260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c8198640 msgr2=0x7f83c8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c67fc640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c8198640 0x7f83c8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c67fc640 1 -- 172.21.15.32:0/537271260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806a990 msgr2=0x7f83c819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c67fc640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806a990 0x7f83c819e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c67fc640 1 -- 172.21.15.32:0/537271260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c8199240 con 0x7f83c81073c0 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83c67fc640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c81073c0 0x7f83c819eac0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f83b000c8c0 tx=0x7f83b000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83cca28640 1 -- 172.21.15.32:0/537271260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0007bb0 con 0x7f83c81073c0 2024-04-16T05:40:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83cca28640 1 -- 172.21.15.32:0/537271260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b0007d50 con 0x7f83c81073c0 2024-04-16T05:40:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83cda2a640 1 -- 172.21.15.32:0/537271260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c81a5320 con 0x7f83c81073c0 2024-04-16T05:40:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83cca28640 1 -- 172.21.15.32:0/537271260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0018830 con 0x7f83c81073c0 2024-04-16T05:40:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.410+0000 7f83cda2a640 1 -- 172.21.15.32:0/537271260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c81a5720 con 0x7f83c81073c0 2024-04-16T05:40:58.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.412+0000 7f83cca28640 1 -- 172.21.15.32:0/537271260 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f83b0018a10 con 0x7f83c81073c0 2024-04-16T05:40:58.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.412+0000 7f83d0615740 1 Processor -- start 2024-04-16T05:40:58.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83d0615740 1 -- start start 2024-04-16T05:40:58.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83d0615740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbcc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83d0615740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83d0615740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83d0615740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5632ccb310e0 con 0x5632ccbd5360 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83d0615740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5632ccace520 con 0x5632ccbcc780 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83d0615740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5632ccb2d0a0 con 0x5632ccbcbe60 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbcc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83cca28640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f83b40a3d20 0x7f83b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbcc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:51410/0 (socket says 172.21.15.32:51410) 2024-04-16T05:40:58.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 -- 172.21.15.32:0/1927952132 learned_addr learned my addr 172.21.15.32:0/1927952132 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55110/0 (socket says 172.21.15.32:55110) 2024-04-16T05:40:58.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/1927952132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f83b40a3d20 0x7f83b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83cca28640 1 -- 172.21.15.32:0/537271260 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f83b00d0d70 con 0x7f83c81073c0 2024-04-16T05:40:58.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 -- 172.21.15.32:0/1927952132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 msgr2=0x5632ccbd4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 --2- 172.21.15.32:0/1927952132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 -- 172.21.15.32:0/1927952132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 msgr2=0x5632ccbd7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 --2- 172.21.15.32:0/1927952132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.413+0000 7f83c77fe640 1 -- 172.21.15.32:0/1927952132 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ccaf68d0 con 0x5632ccbcbe60 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.414+0000 7f83c77fe640 1 --2- 172.21.15.32:0/1927952132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbcc240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83c00149c0 tx=0x7f83c0014e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c5c12d8f0432e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.414+0000 7f83c77fe640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f83b40a3d20 0x7f83b40a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f83c001c000 tx=0x7f83c001e000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.414+0000 7f83bdffb640 1 -- 172.21.15.32:0/1927952132 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0014050 con 0x5632ccbcbe60 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.414+0000 7f83bdffb640 1 -- 172.21.15.32:0/1927952132 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c000cd90 con 0x5632ccbcbe60 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.414+0000 7f83bdffb640 1 -- 172.21.15.32:0/1927952132 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0013a10 con 0x5632ccbcbe60 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- 172.21.15.32:0/1927952132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 msgr2=0x5632ccbcc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 --2- 172.21.15.32:0/1927952132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbcc240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83c00149c0 tx=0x7f83c0014e80 comp rx=0 tx=0).stop 2024-04-16T05:40:58.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- 172.21.15.32:0/1927952132 shutdown_connections 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 --2- 172.21.15.32:0/1927952132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 --2- 172.21.15.32:0/1927952132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 --2- 172.21.15.32:0/1927952132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbcc240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- 172.21.15.32:0/1927952132 >> 172.21.15.32:0/1927952132 conn(0x5632ccbc6590 msgr2=0x5632ccbc6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- 172.21.15.32:0/1927952132 shutdown_connections 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- 172.21.15.32:0/1927952132 wait complete. 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- start start 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 Processor -- start 2024-04-16T05:40:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbd23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5632ccbd8000 con 0x5632ccbd5360 2024-04-16T05:40:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5632ccaea4d0 con 0x5632ccbcc780 2024-04-16T05:40:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5632ccbd8180 con 0x5632ccbcbe60 2024-04-16T05:40:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.415+0000 7f83d0615740 10 client.0 _ll_register_callbacks cb 0x5632ccacf2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55112/0 (socket says 172.21.15.32:55112) 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 -- 172.21.15.32:0/1353430316 learned_addr learned my addr 172.21.15.32:0/1353430316 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbd23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c67fc640 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 -- 172.21.15.32:0/1353430316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 msgr2=0x5632ccbd23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbd23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 -- 172.21.15.32:0/1353430316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 msgr2=0x5632ccbd28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 -- 172.21.15.32:0/1353430316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632cc8433f0 con 0x5632ccbd5360 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83c77fe640 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd2e20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f83c000cf80 tx=0x7f83c00049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1b848deac95881 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83a6ffd640 1 -- 172.21.15.32:0/1353430316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0014250 con 0x5632ccbd5360 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83a6ffd640 1 -- 172.21.15.32:0/1353430316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f83c000c8b0 con 0x5632ccbd5360 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83d0615740 1 -- 172.21.15.32:0/1353430316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c000cd90 con 0x5632ccbd5360 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83a6ffd640 1 -- 172.21.15.32:0/1353430316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83c0037050 con 0x5632ccbd5360 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83d0615740 10 client.17067 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83d0615740 1 -- 172.21.15.32:0/1353430316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632cc90d7d0 con 0x5632ccbd5360 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.416+0000 7f83a6ffd640 1 -- 172.21.15.32:0/1353430316 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c000cd90 con 0x5632ccbd5360 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83a6ffd640 1 -- 172.21.15.32:0/1353430316 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f83c000f040 con 0x5632ccbd5360 2024-04-16T05:40:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83d0615740 10 client.17067 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83d0615740 10 client.17067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83d0615740 20 client.17067 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83d0615740 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5632ccbe9700 0x5632ccbebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83d0615740 4 client.17067 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83d0615740 1 -- 172.21.15.32:0/1353430316 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632ccb3eb00 con 0x5632ccbe9700 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5632ccbe9700 0x5632ccbebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5632ccbe9700 0x5632ccbebaf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88cc3b7b8d7bbc10 server_cookie=2131df061dc93f in_seq=0 out_seq=0 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.417+0000 7f83a6ffd640 10 client.17067 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83a6ffd640 1 -- 172.21.15.32:0/1353430316 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632ccb3eb00 con 0x5632ccbe9700 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83a6ffd640 10 client.17067 handle_command_reply: tid=0 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 client.17067 shutdown 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 -- 172.21.15.32:0/1353430316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 msgr2=0x5632ccbd2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd2e20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f83c000cf80 tx=0x7f83c00049e0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 -- 172.21.15.32:0/1353430316 shutdown_connections 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5632ccbe9700 0x5632ccbebaf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbd2e20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 --2- 172.21.15.32:0/1353430316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbd23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83a6ffd640 0 client.17067 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.418+0000 7f83d0615740 1 -- 172.21.15.32:0/1353430316 >> 172.21.15.32:0/1353430316 conn(0x5632ccbc6590 msgr2=0x5632ccbc82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.419+0000 7f83d0615740 1 -- 172.21.15.32:0/1353430316 shutdown_connections 2024-04-16T05:40:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.419+0000 7f83d0615740 1 -- 172.21.15.32:0/1353430316 wait complete. 2024-04-16T05:40:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.419+0000 7f83d0615740 20 client.17067 trim_cache size 0 max 16384 2024-04-16T05:40:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83d0615740 1 Processor -- start 2024-04-16T05:40:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83d0615740 1 -- start start 2024-04-16T05:40:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83d0615740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbe5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83d0615740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83d0615740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbf11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83d0615740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81af990 con 0x5632ccbd5360 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83d0615740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81ad260 con 0x5632ccbcc780 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83d0615740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81ab0b0 con 0x5632ccbcbe60 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbf11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbf11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55128/0 (socket says 172.21.15.32:55128) 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.432+0000 7f83c77fe640 1 -- 172.21.15.32:0/2553911543 learned_addr learned my addr 172.21.15.32:0/2553911543 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbe5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83c67fc640 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83c77fe640 1 -- 172.21.15.32:0/2553911543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 msgr2=0x5632ccbe5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83c77fe640 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbe5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83c77fe640 1 -- 172.21.15.32:0/2553911543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 msgr2=0x5632ccbd2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83c77fe640 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83c77fe640 1 -- 172.21.15.32:0/2553911543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ccbc7160 con 0x5632ccbd5360 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83c77fe640 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbf11e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f83c0014ec0 tx=0x7f83c00093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d60648cd16970e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83bd7fa640 1 -- 172.21.15.32:0/2553911543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0009860 con 0x5632ccbd5360 2024-04-16T05:40:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83bd7fa640 1 -- 172.21.15.32:0/2553911543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c002e070 con 0x5632ccbd5360 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.433+0000 7f83bd7fa640 1 -- 172.21.15.32:0/2553911543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c000cd00 con 0x5632ccbd5360 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- 172.21.15.32:0/2553911543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 msgr2=0x5632ccbf11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbf11e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f83c0014ec0 tx=0x7f83c00093c0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- 172.21.15.32:0/2553911543 shutdown_connections 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbd5360 0x5632ccbf11e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcc780 0x5632ccbd2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 --2- 172.21.15.32:0/2553911543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbcbe60 0x5632ccbe5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- 172.21.15.32:0/2553911543 >> 172.21.15.32:0/2553911543 conn(0x5632ccbc6590 msgr2=0x5632ccbc8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- 172.21.15.32:0/2553911543 shutdown_connections 2024-04-16T05:40:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- 172.21.15.32:0/2553911543 wait complete. 2024-04-16T05:40:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- start start 2024-04-16T05:40:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 Processor -- start 2024-04-16T05:40:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcbe60 0x5632ccbe1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbcc780 0x5632ccbdcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbd5360 0x5632ccbdd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81ab720 con 0x5632ccbcc780 2024-04-16T05:40:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81acb90 con 0x5632ccbcbe60 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81abdf0 con 0x5632ccbd5360 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.434+0000 7f83d0615740 10 client.0 _ll_register_callbacks cb 0x7f83b009c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbcc780 0x5632ccbdcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbcc780 0x5632ccbdcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55136/0 (socket says 172.21.15.32:55136) 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 -- 172.21.15.32:0/2496572149 learned_addr learned my addr 172.21.15.32:0/2496572149 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c77fe640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbd5360 0x5632ccbdd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcbe60 0x5632ccbe1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 -- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbd5360 msgr2=0x5632ccbdd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbd5360 0x5632ccbdd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 -- 172.21.15.32:0/2496572149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcbe60 msgr2=0x5632ccbe1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcbe60 0x5632ccbe1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 -- 172.21.15.32:0/2496572149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ccbddb00 con 0x5632ccbcc780 2024-04-16T05:40:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c77fe640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbd5360 0x5632ccbdd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcbe60 0x5632ccbe1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83c67fc640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbcc780 0x5632ccbdcf50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f83b009c670 tx=0x7f83b001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2517e8acf4fa725b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83a6ffd640 1 -- 172.21.15.32:0/2496572149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b001f300 con 0x5632ccbcc780 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83a6ffd640 1 -- 172.21.15.32:0/2496572149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f83b00ce3a0 con 0x5632ccbcc780 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83d0615740 1 -- 172.21.15.32:0/2496572149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c002e070 con 0x5632ccbcc780 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83a6ffd640 1 -- 172.21.15.32:0/2496572149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83b00ce5a0 con 0x5632ccbcc780 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83d0615740 10 client.17073 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83d0615740 1 -- 172.21.15.32:0/2496572149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632ccc4f8e0 con 0x5632ccbcc780 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.435+0000 7f83a6ffd640 1 -- 172.21.15.32:0/2496572149 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c002e070 con 0x5632ccbcc780 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83a6ffd640 1 -- 172.21.15.32:0/2496572149 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f83b00a5150 con 0x5632ccbcc780 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83d0615740 10 client.17073 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83d0615740 10 client.17073 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:40:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83d0615740 20 client.17073 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83d0615740 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5632ccc500e0 0x5632ccc52500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83d0615740 4 client.17073 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83d0615740 1 -- 172.21.15.32:0/2496572149 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632ccb38970 con 0x5632ccc500e0 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5632ccc500e0 0x5632ccc52500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83c6ffd640 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5632ccc500e0 0x5632ccc52500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c3e3bfcc2b08f3a server_cookie=6083057a71aeae81 in_seq=0 out_seq=0 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.436+0000 7f83a6ffd640 10 client.17073 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83a6ffd640 1 -- 172.21.15.32:0/2496572149 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632ccb38970 con 0x5632ccc500e0 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83a6ffd640 10 client.17073 handle_command_reply: tid=0 2024-04-16T05:40:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 client.17073 shutdown 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 -- 172.21.15.32:0/2496572149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbcc780 msgr2=0x5632ccbdcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbcc780 0x5632ccbdcf50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f83b009c670 tx=0x7f83b001c000 comp rx=0 tx=0).stop 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 -- 172.21.15.32:0/2496572149 shutdown_connections 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5632ccc500e0 0x5632ccc52500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632ccbd5360 0x5632ccbdd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5632ccbcc780 0x5632ccbdcf50 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 --2- 172.21.15.32:0/2496572149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5632ccbcbe60 0x5632ccbe1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83a6ffd640 0 client.17073 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:40:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 -- 172.21.15.32:0/2496572149 >> 172.21.15.32:0/2496572149 conn(0x5632ccbc6590 msgr2=0x5632ccbc4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 -- 172.21.15.32:0/2496572149 shutdown_connections 2024-04-16T05:40:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 1 -- 172.21.15.32:0/2496572149 wait complete. 2024-04-16T05:40:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83d0615740 20 client.17073 trim_cache size 0 max 16384 2024-04-16T05:40:58.446 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83cda2a640 1 -- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f83b40a3d20 msgr2=0x7f83b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83cda2a640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f83b40a3d20 0x7f83b40a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f83c001c000 tx=0x7f83c001e000 comp rx=0 tx=0).stop 2024-04-16T05:40:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83cda2a640 1 -- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c81073c0 msgr2=0x7f83c819eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.437+0000 7f83cda2a640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c81073c0 0x7f83c819eac0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f83b000c8c0 tx=0x7f83b000cd80 comp rx=0 tx=0).stop 2024-04-16T05:40:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.438+0000 7f83cda2a640 1 -- 172.21.15.32:0/537271260 shutdown_connections 2024-04-16T05:40:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.438+0000 7f83cda2a640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c8198640 0x7f83c8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.438+0000 7f83cda2a640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f83b40a3d20 0x7f83b40a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.438+0000 7f83cda2a640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c81073c0 0x7f83c819eac0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.438+0000 7f83cda2a640 1 --2- 172.21.15.32:0/537271260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806a990 0x7f83c819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.438+0000 7f83cda2a640 1 -- 172.21.15.32:0/537271260 >> 172.21.15.32:0/537271260 conn(0x7f83c8077280 msgr2=0x7f83c80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.438+0000 7f83cda2a640 1 -- 172.21.15.32:0/537271260 shutdown_connections 2024-04-16T05:40:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.438+0000 7f83cda2a640 1 -- 172.21.15.32:0/537271260 wait complete. 2024-04-16T05:40:58.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:40:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 -- 172.21.15.32:0/3600295002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101ae0 msgr2=0x7f4ff4105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/3600295002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101ae0 0x7f4ff4105b20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009900 tx=0x7f4fe802bc30 comp rx=0 tx=0).stop 2024-04-16T05:40:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 -- 172.21.15.32:0/3600295002 shutdown_connections 2024-04-16T05:40:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/3600295002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff4106200 0x7f4ff410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/3600295002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101ae0 0x7f4ff4105b20 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/3600295002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ff4101130 0x7f4ff4101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 -- 172.21.15.32:0/3600295002 >> 172.21.15.32:0/3600295002 conn(0x7f4ff40780b0 msgr2=0x7f4ff40784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 -- 172.21.15.32:0/3600295002 shutdown_connections 2024-04-16T05:40:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 -- 172.21.15.32:0/3600295002 wait complete. 2024-04-16T05:40:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 Processor -- start 2024-04-16T05:40:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.558+0000 7f4ffb42c640 1 -- start start 2024-04-16T05:40:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ffb42c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101130 0x7f4ff4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ffb42c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff4106200 0x7f4ff4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ffb42c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ff419a0d0 0x7f4ff41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ffb42c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff4111270 con 0x7f4ff4101130 2024-04-16T05:40:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ffb42c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41110f0 con 0x7f4ff419a0d0 2024-04-16T05:40:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ffb42c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41113f0 con 0x7f4ff4106200 2024-04-16T05:40:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101130 0x7f4ff4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101130 0x7f4ff4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60658/0 (socket says 172.21.15.32:60658) 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 -- 172.21.15.32:0/4162972694 learned_addr learned my addr 172.21.15.32:0/4162972694 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff4106200 0x7f4ff4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff99c9640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ff419a0d0 0x7f4ff41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 -- 172.21.15.32:0/4162972694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff4106200 msgr2=0x7f4ff4199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff4106200 0x7f4ff4199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 -- 172.21.15.32:0/4162972694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ff419a0d0 msgr2=0x7f4ff41a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ff419a0d0 0x7f4ff41a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 -- 172.21.15.32:0/4162972694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff41a4e50 con 0x7f4ff4101130 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff99c9640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ff419a0d0 0x7f4ff41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101130 0x7f4ff4199650 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c920 tx=0x7f4fe400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff27fc640 1 -- 172.21.15.32:0/4162972694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4007c70 con 0x7f4ff4101130 2024-04-16T05:40:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff27fc640 1 -- 172.21.15.32:0/4162972694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4007e10 con 0x7f4ff4101130 2024-04-16T05:40:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ffb42c640 1 -- 172.21.15.32:0/4162972694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff41a5140 con 0x7f4ff4101130 2024-04-16T05:40:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ff27fc640 1 -- 172.21.15.32:0/4162972694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4018890 con 0x7f4ff4101130 2024-04-16T05:40:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.559+0000 7f4ffb42c640 1 -- 172.21.15.32:0/4162972694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff41a5660 con 0x7f4ff4101130 2024-04-16T05:40:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.561+0000 7f4ff27fc640 1 -- 172.21.15.32:0/4162972694 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4fe4018a70 con 0x7f4ff4101130 2024-04-16T05:40:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ffe00e740 1 Processor -- start 2024-04-16T05:40:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ffe00e740 1 -- start start 2024-04-16T05:40:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ff27fc640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4fdc0a3d20 0x7f4fdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a500230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a502b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56328a4580e0 con 0x56328a4f34b0 2024-04-16T05:40:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56328a3f5520 con 0x56328a500770 2024-04-16T05:40:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56328a4540a0 con 0x56328a4f2b90 2024-04-16T05:40:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ff27fc640 1 -- 172.21.15.32:0/4162972694 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4fe40d0da0 con 0x7f4ff4101130 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.562+0000 7f4ff89c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a500230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff89c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a500230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60668/0 (socket says 172.21.15.32:60668) 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff89c7640 1 -- 172.21.15.32:0/756078973 learned_addr learned my addr 172.21.15.32:0/756078973 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff99c9640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4fdc0a3d20 0x7f4fdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff99c9640 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a502b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff89c7640 1 -- 172.21.15.32:0/756078973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 msgr2=0x56328a4f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4f2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff89c7640 1 -- 172.21.15.32:0/756078973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 msgr2=0x56328a502b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a502b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff89c7640 1 -- 172.21.15.32:0/756078973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328a41d8d0 con 0x56328a4f34b0 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff99c9640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4fdc0a3d20 0x7f4fdc0a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005c40 tx=0x7f4fec009980 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a500230 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4ff406b7a0 tx=0x7f4fe802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e5caf55f4a60c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4fdb7fe640 1 -- 172.21.15.32:0/756078973 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe802cb00 con 0x56328a4f34b0 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4fdb7fe640 1 -- 172.21.15.32:0/756078973 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe802cca0 con 0x56328a4f34b0 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.563+0000 7f4fdb7fe640 1 -- 172.21.15.32:0/756078973 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80048e0 con 0x56328a4f34b0 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 -- 172.21.15.32:0/756078973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 msgr2=0x56328a500230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a500230 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4ff406b7a0 tx=0x7f4fe802c650 comp rx=0 tx=0).stop 2024-04-16T05:40:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 -- 172.21.15.32:0/756078973 shutdown_connections 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a502b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a500230 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/756078973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 -- 172.21.15.32:0/756078973 >> 172.21.15.32:0/756078973 conn(0x56328a4ed590 msgr2=0x56328a4ed970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 -- 172.21.15.32:0/756078973 shutdown_connections 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 -- 172.21.15.32:0/756078973 wait complete. 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 -- start start 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.564+0000 7f4ffe00e740 1 Processor -- start 2024-04-16T05:40:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a4fe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56328a5034d0 con 0x56328a4f34b0 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56328a4114d0 con 0x56328a500770 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56328a503650 con 0x56328a4f2b90 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ffe00e740 10 client.0 _ll_register_callbacks cb 0x56328a3f62c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60674/0 (socket says 172.21.15.32:60674) 2024-04-16T05:40:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 -- 172.21.15.32:0/425431394 learned_addr learned my addr 172.21.15.32:0/425431394 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff99c9640 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a4fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 -- 172.21.15.32:0/425431394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 msgr2=0x56328a4fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4fd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 -- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 msgr2=0x56328a4fe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a4fe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 -- 172.21.15.32:0/425431394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328a16a3f0 con 0x56328a4f34b0 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff99c9640 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a4fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:58.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fdcd0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4fe802c170 tx=0x7f4fe8005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f2ba90c1a59f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.565+0000 7f4fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/425431394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8003a00 con 0x56328a4f34b0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/425431394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4fe8003ba0 con 0x56328a4f34b0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 1 -- 172.21.15.32:0/425431394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe802cca0 con 0x56328a4f34b0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/425431394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4fe8057050 con 0x56328a4f34b0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 10 client.17085 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 1 -- 172.21.15.32:0/425431394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56328a2347d0 con 0x56328a4f34b0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/425431394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe802cca0 con 0x56328a4f34b0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/425431394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4fe803d420 con 0x56328a4f34b0 2024-04-16T05:40:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 10 client.17085 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 10 client.17085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 20 client.17085 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56328a510580 0x56328a512970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 4 client.17085 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ffe00e740 1 -- 172.21.15.32:0/425431394 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56328a465b00 con 0x56328a510580 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.566+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56328a510580 0x56328a512970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.567+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56328a510580 0x56328a512970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c4efb1dd956353 server_cookie=fcbe1fff5958d3be in_seq=0 out_seq=0 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.567+0000 7f4fd8ff9640 10 client.17085 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/425431394 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56328a465b00 con 0x56328a510580 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4fd8ff9640 10 client.17085 handle_command_reply: tid=0 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 client.17085 shutdown 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 -- 172.21.15.32:0/425431394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 msgr2=0x56328a4fdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fdcd0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4fe802c170 tx=0x7f4fe8005540 comp rx=0 tx=0).stop 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 -- 172.21.15.32:0/425431394 shutdown_connections 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56328a510580 0x56328a512970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a4fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4fd8ff9640 0 client.17085 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fdcd0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/425431394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 -- 172.21.15.32:0/425431394 >> 172.21.15.32:0/425431394 conn(0x56328a4ed590 msgr2=0x56328a4ef2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 -- 172.21.15.32:0/425431394 shutdown_connections 2024-04-16T05:40:58.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 1 -- 172.21.15.32:0/425431394 wait complete. 2024-04-16T05:40:58.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.568+0000 7f4ffe00e740 20 client.17085 trim_cache size 0 max 16384 2024-04-16T05:40:58.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.581+0000 7f4ffe00e740 1 Processor -- start 2024-04-16T05:40:58.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.581+0000 7f4ffe00e740 1 -- start start 2024-04-16T05:40:58.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.581+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f2b90 0x56328a4ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.581+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f34b0 0x56328a50c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.581+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a518020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.581+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41af7c0 con 0x56328a4f2b90 2024-04-16T05:40:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.581+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41ad090 con 0x56328a500770 2024-04-16T05:40:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.581+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41aaee0 con 0x56328a4f34b0 2024-04-16T05:40:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f2b90 0x56328a4ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f2b90 0x56328a4ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60682/0 (socket says 172.21.15.32:60682) 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 -- 172.21.15.32:0/1626129424 learned_addr learned my addr 172.21.15.32:0/1626129424 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/1626129424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f34b0 0x56328a50c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 -- 172.21.15.32:0/1626129424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f34b0 msgr2=0x56328a50c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/1626129424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f34b0 0x56328a50c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 -- 172.21.15.32:0/1626129424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 msgr2=0x56328a518020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/1626129424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a518020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 -- 172.21.15.32:0/1626129424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328a4f32e0 con 0x56328a4f2b90 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/1626129424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f2b90 0x56328a4ffec0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40ce1b0 tx=0x7f4fe400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf81f2597ac4203 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4fdaffd640 1 -- 172.21.15.32:0/1626129424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe409c9a0 con 0x56328a4f2b90 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4fdaffd640 1 -- 172.21.15.32:0/1626129424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe40a0d50 con 0x56328a4f2b90 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.582+0000 7f4fdaffd640 1 -- 172.21.15.32:0/1626129424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40ce3d0 con 0x56328a4f2b90 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1626129424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f2b90 msgr2=0x56328a4ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1626129424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f2b90 0x56328a4ffec0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40ce1b0 tx=0x7f4fe400ff30 comp rx=0 tx=0).stop 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1626129424 shutdown_connections 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1626129424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a518020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1626129424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f34b0 0x56328a50c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1626129424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f2b90 0x56328a4ffec0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1626129424 >> 172.21.15.32:0/1626129424 conn(0x56328a4ed590 msgr2=0x56328a4ec730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1626129424 shutdown_connections 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1626129424 wait complete. 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- start start 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 Processor -- start 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4fa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a4fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41ab550 con 0x56328a4f34b0 2024-04-16T05:40:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41ac9c0 con 0x56328a500770 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.583+0000 7f4ffe00e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41abc20 con 0x56328a4f2b90 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ffe00e740 10 client.0 _ll_register_callbacks cb 0x7f4fe8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60694/0 (socket says 172.21.15.32:60694) 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 -- 172.21.15.32:0/1370547079 learned_addr learned my addr 172.21.15.32:0/1370547079 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 -- 172.21.15.32:0/1370547079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 msgr2=0x56328a4fa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4fa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 -- 172.21.15.32:0/1370547079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 msgr2=0x56328a4fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a4fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 -- 172.21.15.32:0/1370547079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328a575890 con 0x56328a4f34b0 2024-04-16T05:40:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ff89c7640 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fa860 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4fe802bc70 tx=0x7f4fe80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d305d28cfc13f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/1370547079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8004320 con 0x56328a4f34b0 2024-04-16T05:40:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/1370547079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4fe803da60 con 0x56328a4f34b0 2024-04-16T05:40:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1370547079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe40a0d50 con 0x56328a4f34b0 2024-04-16T05:40:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/1370547079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4fe803dc60 con 0x56328a4f34b0 2024-04-16T05:40:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ffe00e740 10 client.17091 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1370547079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56328a576a70 con 0x56328a4f34b0 2024-04-16T05:40:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.584+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/1370547079 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe40a0d50 con 0x56328a4f34b0 2024-04-16T05:40:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/1370547079 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4fe8034560 con 0x56328a4f34b0 2024-04-16T05:40:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ffe00e740 10 client.17091 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ffe00e740 10 client.17091 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:40:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ffe00e740 20 client.17091 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56328a577020 0x56328a579440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ffe00e740 4 client.17091 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1370547079 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56328a45f970 con 0x56328a577020 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56328a577020 0x56328a579440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4fd8ff9640 10 client.17091 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ff91c8640 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56328a577020 0x56328a579440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b911e3df26ca0822 server_cookie=bd0c01a8ca54c4f5 in_seq=0 out_seq=0 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4fd8ff9640 1 -- 172.21.15.32:0/1370547079 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56328a45f970 con 0x56328a577020 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4fd8ff9640 10 client.17091 handle_command_reply: tid=0 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.585+0000 7f4ffe00e740 1 client.17091 shutdown 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1370547079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 msgr2=0x56328a4fa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fa860 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4fe802bc70 tx=0x7f4fe80045f0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1370547079 shutdown_connections 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56328a577020 0x56328a579440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56328a500770 0x56328a4fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56328a4f34b0 0x56328a4fa860 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4fd8ff9640 0 client.17091 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 --2- 172.21.15.32:0/1370547079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56328a4f2b90 0x56328a4fa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1370547079 >> 172.21.15.32:0/1370547079 conn(0x56328a4ed590 msgr2=0x56328a4fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1370547079 shutdown_connections 2024-04-16T05:40:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 1 -- 172.21.15.32:0/1370547079 wait complete. 2024-04-16T05:40:58.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffe00e740 20 client.17091 trim_cache size 0 max 16384 2024-04-16T05:40:58.595 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:58.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 -- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4fdc0a3d20 msgr2=0x7f4fdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4fdc0a3d20 0x7f4fdc0a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005c40 tx=0x7f4fec009980 comp rx=0 tx=0).stop 2024-04-16T05:40:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 -- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101130 msgr2=0x7f4ff4199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101130 0x7f4ff4199650 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c920 tx=0x7f4fe400cde0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 -- 172.21.15.32:0/4162972694 shutdown_connections 2024-04-16T05:40:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ff419a0d0 0x7f4ff41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff4106200 0x7f4ff4199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4fdc0a3d20 0x7f4fdc0a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 --2- 172.21.15.32:0/4162972694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ff4101130 0x7f4ff4199650 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 -- 172.21.15.32:0/4162972694 >> 172.21.15.32:0/4162972694 conn(0x7f4ff40780b0 msgr2=0x7f4ff40fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 -- 172.21.15.32:0/4162972694 shutdown_connections 2024-04-16T05:40:58.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.586+0000 7f4ffb42c640 1 -- 172.21.15.32:0/4162972694 wait complete. 2024-04-16T05:40:58.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:40:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 -- 172.21.15.32:0/413538342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 msgr2=0x7fade410c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 --2- 172.21.15.32:0/413538342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 0x7fade410c070 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009900 tx=0x7fadd402bcd0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 -- 172.21.15.32:0/413538342 shutdown_connections 2024-04-16T05:40:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 --2- 172.21.15.32:0/413538342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade410c640 0x7fade410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 --2- 172.21.15.32:0/413538342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 0x7fade410c070 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 --2- 172.21.15.32:0/413538342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4103270 0x7fade4103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 -- 172.21.15.32:0/413538342 >> 172.21.15.32:0/413538342 conn(0x7fade4078020 msgr2=0x7fade4078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 -- 172.21.15.32:0/413538342 shutdown_connections 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.707+0000 7fadeb493640 1 -- 172.21.15.32:0/413538342 wait complete. 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fadeb493640 1 Processor -- start 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fadeb493640 1 -- start start 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fadeb493640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade4103270 0x7fade419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fadeb493640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 0x7fade419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fadeb493640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade410c640 0x7fade41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fadeb493640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fade41123d0 con 0x7fade4103b90 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fadeb493640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fade4112250 con 0x7fade4103270 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fadeb493640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fade4112550 con 0x7fade410c640 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fade8a2e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 0x7fade419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fade8a2e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 0x7fade419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60706/0 (socket says 172.21.15.32:60706) 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fade8a2e640 1 -- 172.21.15.32:0/3464487934 learned_addr learned my addr 172.21.15.32:0/3464487934 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fade922f640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade4103270 0x7fade419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.708+0000 7fade9a30640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade410c640 0x7fade41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fade8a2e640 1 -- 172.21.15.32:0/3464487934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade410c640 msgr2=0x7fade41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fade8a2e640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade410c640 0x7fade41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fade8a2e640 1 -- 172.21.15.32:0/3464487934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade4103270 msgr2=0x7fade419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fade8a2e640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade4103270 0x7fade419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fade8a2e640 1 -- 172.21.15.32:0/3464487934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade4198f70 con 0x7fade4103b90 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fade8a2e640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 0x7fade419eae0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fadd4005940 tx=0x7fadd402cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fadda7fc640 1 -- 172.21.15.32:0/3464487934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4041070 con 0x7fade4103b90 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fadda7fc640 1 -- 172.21.15.32:0/3464487934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd4004450 con 0x7fade4103b90 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fadda7fc640 1 -- 172.21.15.32:0/3464487934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4003df0 con 0x7fade4103b90 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fadeb493640 1 -- 172.21.15.32:0/3464487934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade4199200 con 0x7fade4103b90 2024-04-16T05:40:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.709+0000 7fadeb493640 1 -- 172.21.15.32:0/3464487934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41a5520 con 0x7fade4103b90 2024-04-16T05:40:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.711+0000 7fadda7fc640 1 -- 172.21.15.32:0/3464487934 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fadd4044070 con 0x7fade4103b90 2024-04-16T05:40:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadee082740 1 Processor -- start 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadee082740 1 -- start start 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadee082740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943a3e60 0x5584943a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadee082740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadee082740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943ad360 0x5584943af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadee082740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5584943090e0 con 0x5584943ad360 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadee082740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5584942a6520 con 0x5584943a4780 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadda7fc640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fadd00a3da0 0x7fadd00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadee082740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584943050a0 con 0x5584943a3e60 2024-04-16T05:40:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade8a2e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943ad360 0x5584943af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade9a30640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fadd00a3da0 0x7fadd00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade8a2e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943ad360 0x5584943af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60722/0 (socket says 172.21.15.32:60722) 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade8a2e640 1 -- 172.21.15.32:0/1177285447 learned_addr learned my addr 172.21.15.32:0/1177285447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fadda7fc640 1 -- 172.21.15.32:0/3464487934 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fadd40f2470 con 0x7fade4103b90 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade9a30640 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943a3e60 0x5584943a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade922f640 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade8a2e640 1 -- 172.21.15.32:0/1177285447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943a3e60 msgr2=0x5584943a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade8a2e640 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943a3e60 0x5584943a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade8a2e640 1 -- 172.21.15.32:0/1177285447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 msgr2=0x5584943ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade8a2e640 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.712+0000 7fade8a2e640 1 -- 172.21.15.32:0/1177285447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584942ce8d0 con 0x5584943ad360 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.713+0000 7fade9a30640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fadd00a3da0 0x7fadd00a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faddc007310 tx=0x7faddc013ac0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.713+0000 7fade8a2e640 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943ad360 0x5584943af750 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fadd40ba6f0 tx=0x7fadd408db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3163504c18ff966 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.713+0000 7fadcf7fe640 1 -- 172.21.15.32:0/1177285447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd404b070 con 0x5584943ad360 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.713+0000 7fadcf7fe640 1 -- 172.21.15.32:0/1177285447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd4004c80 con 0x5584943ad360 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.713+0000 7fadcf7fe640 1 -- 172.21.15.32:0/1177285447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40c2590 con 0x5584943ad360 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.713+0000 7fadee082740 1 -- 172.21.15.32:0/1177285447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943ad360 msgr2=0x5584943af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.713+0000 7fadee082740 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943ad360 0x5584943af750 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fadd40ba6f0 tx=0x7fadd408db70 comp rx=0 tx=0).stop 2024-04-16T05:40:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 -- 172.21.15.32:0/1177285447 shutdown_connections 2024-04-16T05:40:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943ad360 0x5584943af750 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 --2- 172.21.15.32:0/1177285447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943a3e60 0x5584943a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 -- 172.21.15.32:0/1177285447 >> 172.21.15.32:0/1177285447 conn(0x55849439e590 msgr2=0x55849439e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 -- 172.21.15.32:0/1177285447 shutdown_connections 2024-04-16T05:40:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 -- 172.21.15.32:0/1177285447 wait complete. 2024-04-16T05:40:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 -- start start 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 Processor -- start 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a3e60 0x5584943aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a4780 0x5584943aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5584943aff10 con 0x5584943a4780 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5584942c24d0 con 0x5584943a3e60 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584943b0090 con 0x5584943ad360 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fadee082740 10 client.0 _ll_register_callbacks cb 0x5584942a72c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.714+0000 7fade8a2e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a4780 0x5584943aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade8a2e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a4780 0x5584943aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60732/0 (socket says 172.21.15.32:60732) 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade8a2e640 1 -- 172.21.15.32:0/1685773468 learned_addr learned my addr 172.21.15.32:0/1685773468 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade922f640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a3e60 0x5584943aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade9a30640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade8a2e640 1 -- 172.21.15.32:0/1685773468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 msgr2=0x5584943aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade8a2e640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade8a2e640 1 -- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a3e60 msgr2=0x5584943aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade8a2e640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a3e60 0x5584943aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade8a2e640 1 -- 172.21.15.32:0/1685773468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849401b3f0 con 0x5584943a4780 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade9a30640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade922f640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a3e60 0x5584943aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fade8a2e640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a4780 0x5584943aa8c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fadd4000c00 tx=0x7fadd40be100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e170ba26d9db3c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fadccff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fadccff9640 1 -- 172.21.15.32:0/1685773468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd404b050 con 0x5584943a4780 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fadccff9640 1 -- 172.21.15.32:0/1685773468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fadd40ba900 con 0x5584943a4780 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fadee082740 1 -- 172.21.15.32:0/1685773468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd4004c80 con 0x5584943a4780 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fadccff9640 1 -- 172.21.15.32:0/1685773468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadd40c2590 con 0x5584943a4780 2024-04-16T05:40:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fadee082740 10 client.17103 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fadee082740 1 -- 172.21.15.32:0/1685773468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5584940e57d0 con 0x5584943a4780 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.715+0000 7fadccff9640 1 -- 172.21.15.32:0/1685773468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd4004c80 con 0x5584943a4780 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fadccff9640 1 -- 172.21.15.32:0/1685773468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fadd408dbe0 con 0x5584943a4780 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fadee082740 10 client.17103 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fadee082740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fadee082740 20 client.17103 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fadee082740 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5584943bcef0 0x5584943bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fadee082740 4 client.17103 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fadee082740 1 -- 172.21.15.32:0/1685773468 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558494316b00 con 0x5584943bcef0 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fade922f640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5584943bcef0 0x5584943bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fadccff9640 10 client.17103 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.716+0000 7fade922f640 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5584943bcef0 0x5584943bf2e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88affbcf10bc0e96 server_cookie=8ee4789c37c50f35 in_seq=0 out_seq=0 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadccff9640 1 -- 172.21.15.32:0/1685773468 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558494316b00 con 0x5584943bcef0 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadccff9640 10 client.17103 handle_command_reply: tid=0 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 client.17103 shutdown 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 -- 172.21.15.32:0/1685773468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a4780 msgr2=0x5584943aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a4780 0x5584943aa8c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fadd4000c00 tx=0x7fadd40be100 comp rx=0 tx=0).stop 2024-04-16T05:40:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 -- 172.21.15.32:0/1685773468 shutdown_connections 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5584943bcef0 0x5584943bf2e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a4780 0x5584943aa8c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 --2- 172.21.15.32:0/1685773468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a3e60 0x5584943aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 -- 172.21.15.32:0/1685773468 >> 172.21.15.32:0/1685773468 conn(0x55849439e590 msgr2=0x5584943a02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadccff9640 0 client.17103 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 -- 172.21.15.32:0/1685773468 shutdown_connections 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 1 -- 172.21.15.32:0/1685773468 wait complete. 2024-04-16T05:40:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.717+0000 7fadee082740 20 client.17103 trim_cache size 0 max 16384 2024-04-16T05:40:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fadee082740 1 Processor -- start 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fadee082740 1 -- start start 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fadee082740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fadee082740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fadee082740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fadee082740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fade41b1af0 con 0x5584943a3e60 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fadee082740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fade41af3c0 con 0x5584943a4780 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fadee082740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ad210 con 0x5584943ad360 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60734/0 (socket says 172.21.15.32:60734) 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 -- 172.21.15.32:0/2462608150 learned_addr learned my addr 172.21.15.32:0/2462608150 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade8a2e640 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 -- 172.21.15.32:0/2462608150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 msgr2=0x5584943995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 -- 172.21.15.32:0/2462608150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 msgr2=0x5584943b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 -- 172.21.15.32:0/2462608150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584943a9330 con 0x5584943a3e60 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade8a2e640 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:58.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.731+0000 7fade922f640 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943acab0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fade0009fd0 tx=0x7fade0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cace65e2409e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadceffd640 1 -- 172.21.15.32:0/2462608150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade001e640 con 0x5584943a3e60 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadceffd640 1 -- 172.21.15.32:0/2462608150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade0022030 con 0x5584943a3e60 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadceffd640 1 -- 172.21.15.32:0/2462608150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0004e10 con 0x5584943a3e60 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 -- 172.21.15.32:0/2462608150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 msgr2=0x5584943acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943acab0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fade0009fd0 tx=0x7fade0005f90 comp rx=0 tx=0).stop 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 -- 172.21.15.32:0/2462608150 shutdown_connections 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 --2- 172.21.15.32:0/2462608150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943acab0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 -- 172.21.15.32:0/2462608150 >> 172.21.15.32:0/2462608150 conn(0x55849439e590 msgr2=0x55849439cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 -- 172.21.15.32:0/2462608150 shutdown_connections 2024-04-16T05:40:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.732+0000 7fadee082740 1 -- 172.21.15.32:0/2462608150 wait complete. 2024-04-16T05:40:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 1 -- start start 2024-04-16T05:40:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 1 Processor -- start 2024-04-16T05:40:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ad880 con 0x5584943a3e60 2024-04-16T05:40:58.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fade41aecf0 con 0x5584943a4780 2024-04-16T05:40:58.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fade41adf50 con 0x5584943ad360 2024-04-16T05:40:58.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fadee082740 10 client.0 _ll_register_callbacks cb 0x7fadd40a8f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:58.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fade922f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fade922f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60740/0 (socket says 172.21.15.32:60740) 2024-04-16T05:40:58.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fade922f640 1 -- 172.21.15.32:0/2907797546 learned_addr learned my addr 172.21.15.32:0/2907797546 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fade8a2e640 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fade9a30640 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.733+0000 7fade922f640 1 -- 172.21.15.32:0/2907797546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 msgr2=0x5584943b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fade922f640 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fade922f640 1 -- 172.21.15.32:0/2907797546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 msgr2=0x5584943b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fade922f640 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fade922f640 1 -- 172.21.15.32:0/2907797546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584943b1170 con 0x5584943a3e60 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fade9a30640 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:40:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fade922f640 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943b5280 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fade0005bc0 tx=0x7fade0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27695ba5ce5ada10 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadccff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadccff9640 1 -- 172.21.15.32:0/2907797546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0004df0 con 0x5584943a3e60 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadccff9640 1 -- 172.21.15.32:0/2907797546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fade0004050 con 0x5584943a3e60 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 1 -- 172.21.15.32:0/2907797546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade0022030 con 0x5584943a3e60 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadccff9640 1 -- 172.21.15.32:0/2907797546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fade0034050 con 0x5584943a3e60 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 10 client.17109 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 1 -- 172.21.15.32:0/2907797546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558494422f60 con 0x5584943a3e60 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadccff9640 1 -- 172.21.15.32:0/2907797546 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade0022030 con 0x5584943a3e60 2024-04-16T05:40:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadccff9640 1 -- 172.21.15.32:0/2907797546 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fade00054b0 con 0x5584943a3e60 2024-04-16T05:40:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 10 client.17109 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 20 client.17109 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5584944235e0 0x558494425a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 4 client.17109 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:40:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.734+0000 7fadee082740 1 -- 172.21.15.32:0/2907797546 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558494310970 con 0x5584944235e0 2024-04-16T05:40:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fade8a2e640 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5584944235e0 0x558494425a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadccff9640 10 client.17109 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fade8a2e640 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5584944235e0 0x558494425a00 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156f6a8535b772bb server_cookie=d6b24f3ece1b91ed in_seq=0 out_seq=0 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadccff9640 1 -- 172.21.15.32:0/2907797546 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558494310970 con 0x5584944235e0 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadccff9640 10 client.17109 handle_command_reply: tid=0 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 client.17109 shutdown 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 -- 172.21.15.32:0/2907797546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 msgr2=0x5584943b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943b5280 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fade0005bc0 tx=0x7fade0004a50 comp rx=0 tx=0).stop 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 -- 172.21.15.32:0/2907797546 shutdown_connections 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5584944235e0 0x558494425a00 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584943ad360 0x5584943b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584943a4780 0x5584943b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadccff9640 0 client.17109 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 --2- 172.21.15.32:0/2907797546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5584943a3e60 0x5584943b5280 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.735+0000 7fadee082740 1 -- 172.21.15.32:0/2907797546 >> 172.21.15.32:0/2907797546 conn(0x55849439e590 msgr2=0x55849439d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadee082740 1 -- 172.21.15.32:0/2907797546 shutdown_connections 2024-04-16T05:40:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadee082740 1 -- 172.21.15.32:0/2907797546 wait complete. 2024-04-16T05:40:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadee082740 20 client.17109 trim_cache size 0 max 16384 2024-04-16T05:40:58.746 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:40:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 -- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fadd00a3da0 msgr2=0x7fadd00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fadd00a3da0 0x7fadd00a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faddc007310 tx=0x7faddc013ac0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 -- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 msgr2=0x7fade419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 0x7fade419eae0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fadd4005940 tx=0x7fadd402cc40 comp rx=0 tx=0).stop 2024-04-16T05:40:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 -- 172.21.15.32:0/3464487934 shutdown_connections 2024-04-16T05:40:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade410c640 0x7fade41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fadd00a3da0 0x7fadd00a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4103b90 0x7fade419eae0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 --2- 172.21.15.32:0/3464487934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade4103270 0x7fade419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 -- 172.21.15.32:0/3464487934 >> 172.21.15.32:0/3464487934 conn(0x7fade4078020 msgr2=0x7fade40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 -- 172.21.15.32:0/3464487934 shutdown_connections 2024-04-16T05:40:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:58.736+0000 7fadeb493640 1 -- 172.21.15.32:0/3464487934 wait complete. 2024-04-16T05:40:58.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/67932635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/67932635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:59.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/67932635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:40:59.749 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:40:59.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:40:59.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:40:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 -- 172.21.15.32:0/231459860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0101b80 msgr2=0x7f1af010dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 --2- 172.21.15.32:0/231459860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0101b80 0x7f1af010dda0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009900 tx=0x7f1ae802bc80 comp rx=0 tx=0).stop 2024-04-16T05:40:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 -- 172.21.15.32:0/231459860 shutdown_connections 2024-04-16T05:40:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 --2- 172.21.15.32:0/231459860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0101b80 0x7f1af010dda0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 --2- 172.21.15.32:0/231459860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af01011e0 0x7f1af0101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 --2- 172.21.15.32:0/231459860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af0105f50 0x7f1af0106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 -- 172.21.15.32:0/231459860 >> 172.21.15.32:0/231459860 conn(0x7f1af0077ee0 msgr2=0x7f1af00782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 -- 172.21.15.32:0/231459860 shutdown_connections 2024-04-16T05:40:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 -- 172.21.15.32:0/231459860 wait complete. 2024-04-16T05:40:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.874+0000 7f1af53c8640 1 Processor -- start 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1af53c8640 1 -- start start 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1af53c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af01011e0 0x7f1af019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1af53c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0101b80 0x7f1af019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1af53c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0105f50 0x7f1af01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1af53c8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0112550 con 0x7f1af0105f50 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1af53c8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01123d0 con 0x7f1af01011e0 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1af53c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01126d0 con 0x7f1af0101b80 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1aef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0105f50 0x7f1af01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1aef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0105f50 0x7f1af01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60754/0 (socket says 172.21.15.32:60754) 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1aef7fe640 1 -- 172.21.15.32:0/2722253557 learned_addr learned my addr 172.21.15.32:0/2722253557 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0101b80 0x7f1af019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af01011e0 0x7f1af019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1aef7fe640 1 -- 172.21.15.32:0/2722253557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0101b80 msgr2=0x7f1af019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.875+0000 7f1aef7fe640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0101b80 0x7f1af019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1aef7fe640 1 -- 172.21.15.32:0/2722253557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af01011e0 msgr2=0x7f1af019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1aef7fe640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af01011e0 0x7f1af019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1aef7fe640 1 -- 172.21.15.32:0/2722253557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af0198fa0 con 0x7f1af0105f50 2024-04-16T05:40:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1aef7fe640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0105f50 0x7f1af01987d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8005940 tx=0x7f1ae80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1ad7fff640 1 -- 172.21.15.32:0/2722253557 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8041070 con 0x7f1af0105f50 2024-04-16T05:40:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1af53c8640 1 -- 172.21.15.32:0/2722253557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af0199230 con 0x7f1af0105f50 2024-04-16T05:40:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1ad7fff640 1 -- 172.21.15.32:0/2722253557 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae802ccb0 con 0x7f1af0105f50 2024-04-16T05:40:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1ad7fff640 1 -- 172.21.15.32:0/2722253557 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8003d40 con 0x7f1af0105f50 2024-04-16T05:40:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.876+0000 7f1af53c8640 1 -- 172.21.15.32:0/2722253557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af01a54b0 con 0x7f1af0105f50 2024-04-16T05:40:59.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1af7fb1740 1 Processor -- start 2024-04-16T05:40:59.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1ad7fff640 1 -- 172.21.15.32:0/2722253557 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ae8044070 con 0x7f1af0105f50 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1af7fb1740 1 -- start start 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad40e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad413d0 0x55fa1ad49a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad4c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fa1aca5e90 con 0x55fa1ad413d0 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ac434c0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.879+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fa1aca1e50 con 0x55fa1ad49fb0 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad413d0 0x55fa1ad49a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad413d0 0x55fa1ad49a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60762/0 (socket says 172.21.15.32:60762) 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 -- 172.21.15.32:0/761207142 learned_addr learned my addr 172.21.15.32:0/761207142 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad4c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad40e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1ad7fff640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1ad80a3d20 0x7f1ad80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 -- 172.21.15.32:0/761207142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 msgr2=0x55fa1ad4c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad4c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 -- 172.21.15.32:0/761207142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad40ab0 msgr2=0x55fa1ad40e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad40e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 -- 172.21.15.32:0/761207142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa1aa807e0 con 0x55fa1ad413d0 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad40e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aef7fe640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1ad80a3d20 0x7f1ad80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1ad7fff640 1 -- 172.21.15.32:0/2722253557 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1ae80f2480 con 0x7f1af0105f50 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad413d0 0x55fa1ad49a70 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0003a20 tx=0x7f1ae0003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b45a52845ba37f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1ad57fa640 1 -- 172.21.15.32:0/761207142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0004860 con 0x55fa1ad413d0 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1ad57fa640 1 -- 172.21.15.32:0/761207142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae0015030 con 0x55fa1ad413d0 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.880+0000 7f1ad57fa640 1 -- 172.21.15.32:0/761207142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0010a70 con 0x55fa1ad413d0 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 -- 172.21.15.32:0/761207142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad413d0 msgr2=0x55fa1ad49a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1aef7fe640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1ad80a3d20 0x7f1ad80a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8090d60 tx=0x7f1ae808fce0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:59.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad413d0 0x55fa1ad49a70 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0003a20 tx=0x7f1ae0003ee0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 -- 172.21.15.32:0/761207142 shutdown_connections 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad4c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad413d0 0x55fa1ad49a70 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/761207142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad40e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 -- 172.21.15.32:0/761207142 >> 172.21.15.32:0/761207142 conn(0x55fa1ad3b1e0 msgr2=0x55fa1ad3b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 -- 172.21.15.32:0/761207142 shutdown_connections 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 -- 172.21.15.32:0/761207142 wait complete. 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.881+0000 7f1af7fb1740 1 -- start start 2024-04-16T05:40:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1af7fb1740 1 Processor -- start 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad46fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad47a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ad4cda0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ac2e410 con 0x55fa1ad49fb0 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ad4cf20 con 0x55fa1ad413d0 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1af7fb1740 10 client.0 _ll_register_callbacks cb 0x55fa1ac29b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad46fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad46fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60768/0 (socket says 172.21.15.32:60768) 2024-04-16T05:40:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 -- 172.21.15.32:0/974937353 learned_addr learned my addr 172.21.15.32:0/974937353 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 -- 172.21.15.32:0/974937353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 msgr2=0x55fa1ad47500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 -- 172.21.15.32:0/974937353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad49fb0 msgr2=0x55fa1ad47a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad47a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 -- 172.21.15.32:0/974937353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa1aae0f70 con 0x55fa1ad40ab0 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad46fc0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40042d0 tx=0x7f1ae40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f22ba4e456067c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.882+0000 7f1ad2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/974937353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4005220 con 0x55fa1ad40ab0 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/974937353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1ae40053c0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 1 -- 172.21.15.32:0/974937353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae0015030 con 0x55fa1ad40ab0 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/974937353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ae4026050 con 0x55fa1ad40ab0 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 10 client.17121 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 1 -- 172.21.15.32:0/974937353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa1a978170 con 0x55fa1ad40ab0 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/974937353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae0015030 con 0x55fa1ad40ab0 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/974937353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ae401c030 con 0x55fa1ad40ab0 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 10 client.17121 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 20 client.17121 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fa1ad5a3a0 0x55fa1ad5c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 4 client.17121 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.883+0000 7f1af7fb1740 1 -- 172.21.15.32:0/974937353 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa1acb38b0 con 0x55fa1ad5a3a0 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.884+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fa1ad5a3a0 0x55fa1ad5c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.884+0000 7f1ad2ff5640 10 client.17121 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.884+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fa1ad5a3a0 0x55fa1ad5c7b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d629349efddd12f server_cookie=963cb0ec7c168f83 in_seq=0 out_seq=0 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/974937353 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa1acb38b0 con 0x55fa1ad5a3a0 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1ad2ff5640 10 client.17121 handle_command_reply: tid=0 2024-04-16T05:40:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 client.17121 shutdown 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 -- 172.21.15.32:0/974937353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 msgr2=0x55fa1ad46fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad46fc0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40042d0 tx=0x7f1ae40039c0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 -- 172.21.15.32:0/974937353 shutdown_connections 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad47a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fa1ad5a3a0 0x55fa1ad5c7b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/974937353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad46fc0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1ad2ff5640 0 client.17121 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 -- 172.21.15.32:0/974937353 >> 172.21.15.32:0/974937353 conn(0x55fa1ad3b1e0 msgr2=0x55fa1ad3cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 -- 172.21.15.32:0/974937353 shutdown_connections 2024-04-16T05:40:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 1 -- 172.21.15.32:0/974937353 wait complete. 2024-04-16T05:40:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.885+0000 7f1af7fb1740 20 client.17121 trim_cache size 0 max 16384 2024-04-16T05:40:59.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1af7fb1740 1 Processor -- start 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1af7fb1740 1 -- start start 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad62080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01b1b40 con 0x55fa1ad40ab0 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01af410 con 0x55fa1ad49fb0 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01ad260 con 0x55fa1ad413d0 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aee7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aee7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60774/0 (socket says 172.21.15.32:60774) 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aee7fc640 1 -- 172.21.15.32:0/332995979 learned_addr learned my addr 172.21.15.32:0/332995979 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aee7fc640 1 -- 172.21.15.32:0/332995979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 msgr2=0x55fa1ad47860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aee7fc640 1 -- 172.21.15.32:0/332995979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad49fb0 msgr2=0x55fa1ad62080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad62080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aee7fc640 1 -- 172.21.15.32:0/332995979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa1ad3bdb0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.903+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad496f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4024040 tx=0x7f1ae400b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e74174e557da1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1ad4ff9640 1 -- 172.21.15.32:0/332995979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae402f070 con 0x55fa1ad40ab0 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1ad4ff9640 1 -- 172.21.15.32:0/332995979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae4010bb0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1ad4ff9640 1 -- 172.21.15.32:0/332995979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae402a690 con 0x55fa1ad40ab0 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 -- 172.21.15.32:0/332995979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 msgr2=0x55fa1ad496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad496f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4024040 tx=0x7f1ae400b6f0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 -- 172.21.15.32:0/332995979 shutdown_connections 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad413d0 0x55fa1ad47860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/332995979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad496f0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 -- 172.21.15.32:0/332995979 >> 172.21.15.32:0/332995979 conn(0x55fa1ad3b1e0 msgr2=0x55fa1ad3c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 -- 172.21.15.32:0/332995979 shutdown_connections 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.904+0000 7f1af7fb1740 1 -- 172.21.15.32:0/332995979 wait complete. 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 1 -- start start 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 1 Processor -- start 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad413d0 0x55fa1ad4e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad4eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01ad8d0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01aed40 con 0x55fa1ad413d0 2024-04-16T05:40:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01adfa0 con 0x55fa1ad49fb0 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1af7fb1740 10 client.0 _ll_register_callbacks cb 0x7f1ae400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aee7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aee7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60788/0 (socket says 172.21.15.32:60788) 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aee7fc640 1 -- 172.21.15.32:0/1298341333 learned_addr learned my addr 172.21.15.32:0/1298341333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad413d0 0x55fa1ad4e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aee7fc640 1 -- 172.21.15.32:0/1298341333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 msgr2=0x55fa1ad4eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad4eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aee7fc640 1 -- 172.21.15.32:0/1298341333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad413d0 msgr2=0x55fa1ad4e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:40:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad413d0 0x55fa1ad4e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.905+0000 7f1aee7fc640 1 -- 172.21.15.32:0/1298341333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa1ad4f720 con 0x55fa1ad40ab0 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1aee7fc640 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad525c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400bfd0 tx=0x7f1ae400b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a2809c52c9ac05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1ad2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/1298341333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae402f030 con 0x55fa1ad40ab0 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/1298341333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1ae4005030 con 0x55fa1ad40ab0 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 1 -- 172.21.15.32:0/1298341333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae4010bb0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/1298341333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ae403c050 con 0x55fa1ad40ab0 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 10 client.17127 fetch_fsmap learned FSMap version 25 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 1 -- 172.21.15.32:0/1298341333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa1adc19b0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/1298341333 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae4010bb0 con 0x55fa1ad40ab0 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/1298341333 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ae401c070 con 0x55fa1ad40ab0 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 10 client.17127 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 20 client.17127 populate_metadata read hostname 'smithi032' 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fa1adc2010 0x55fa1adc4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 4 client.17127 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.906+0000 7f1af7fb1740 1 -- 172.21.15.32:0/1298341333 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa1acad720 con 0x55fa1adc2010 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.907+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fa1adc2010 0x55fa1adc4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.907+0000 7f1aeeffd640 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fa1adc2010 0x55fa1adc4430 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fa8139b68ac162 server_cookie=cd6855c3c40f3832 in_seq=0 out_seq=0 2024-04-16T05:40:59.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.907+0000 7f1ad2ff5640 10 client.17127 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1ad2ff5640 1 -- 172.21.15.32:0/1298341333 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa1acad720 con 0x55fa1adc2010 2024-04-16T05:40:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1ad2ff5640 10 client.17127 handle_command_reply: tid=0 2024-04-16T05:40:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 client.17127 shutdown 2024-04-16T05:40:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 -- 172.21.15.32:0/1298341333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 msgr2=0x55fa1ad525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad525c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400bfd0 tx=0x7f1ae400b400 comp rx=0 tx=0).stop 2024-04-16T05:40:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 -- 172.21.15.32:0/1298341333 shutdown_connections 2024-04-16T05:40:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55fa1adc2010 0x55fa1adc4430 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1ad49fb0 0x55fa1ad4eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa1ad413d0 0x55fa1ad4e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 --2- 172.21.15.32:0/1298341333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa1ad40ab0 0x55fa1ad525c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 -- 172.21.15.32:0/1298341333 >> 172.21.15.32:0/1298341333 conn(0x55fa1ad3b1e0 msgr2=0x55fa1ad3cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:59.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1ad2ff5640 0 client.17127 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:40:59.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.908+0000 7f1af7fb1740 1 -- 172.21.15.32:0/1298341333 shutdown_connections 2024-04-16T05:40:59.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af7fb1740 1 -- 172.21.15.32:0/1298341333 wait complete. 2024-04-16T05:40:59.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af7fb1740 20 client.17127 trim_cache size 0 max 16384 2024-04-16T05:40:59.916 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:40:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:40:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "255aab1f-ee21-4946-9c4f-fd3790ae4e72", 2024-04-16T05:40:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:40:59.916 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:40:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 -- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1ad80a3d20 msgr2=0x7f1ad80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1ad80a3d20 0x7f1ad80a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8090d60 tx=0x7f1ae808fce0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 -- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0105f50 msgr2=0x7f1af01987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:40:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0105f50 0x7f1af01987d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8005940 tx=0x7f1ae80043b0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 -- 172.21.15.32:0/2722253557 shutdown_connections 2024-04-16T05:40:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f1ad80a3d20 0x7f1ad80a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1af0105f50 0x7f1af01987d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af0101b80 0x7f1af019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 --2- 172.21.15.32:0/2722253557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af01011e0 0x7f1af019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:40:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 -- 172.21.15.32:0/2722253557 >> 172.21.15.32:0/2722253557 conn(0x7f1af0077ee0 msgr2=0x7f1af00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:40:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 -- 172.21.15.32:0/2722253557 shutdown_connections 2024-04-16T05:40:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:40:59.909+0000 7f1af53c8640 1 -- 172.21.15.32:0/2722253557 wait complete. 2024-04-16T05:40:59.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:40:59.919 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:41:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 -- 172.21.15.32:0/3625060773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 msgr2=0x7faa001073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 --2- 172.21.15.32:0/3625060773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 0x7faa001073c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009cd0 tx=0x7fa9f002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 -- 172.21.15.32:0/3625060773 shutdown_connections 2024-04-16T05:41:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 --2- 172.21.15.32:0/3625060773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa00100ec0 0x7faa0010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 --2- 172.21.15.32:0/3625060773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa00100500 0x7faa00100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 --2- 172.21.15.32:0/3625060773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 0x7faa001073c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 -- 172.21.15.32:0/3625060773 >> 172.21.15.32:0/3625060773 conn(0x7faa000774c0 msgr2=0x7faa000fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 -- 172.21.15.32:0/3625060773 shutdown_connections 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.028+0000 7faa07fba640 1 -- 172.21.15.32:0/3625060773 wait complete. 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa07fba640 1 Processor -- start 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa07fba640 1 -- start start 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa07fba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa00100500 0x7faa001016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa07fba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa00100ec0 0x7faa00101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa07fba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 0x7faa00102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa07fba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa00112470 con 0x7faa00106fe0 2024-04-16T05:41:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa07fba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa001122f0 con 0x7faa00100ec0 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa07fba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa001125f0 con 0x7faa00100500 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa06557640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 0x7faa00102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa06557640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 0x7faa00102160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60798/0 (socket says 172.21.15.32:60798) 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa06557640 1 -- 172.21.15.32:0/2345965346 learned_addr learned my addr 172.21.15.32:0/2345965346 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa05555640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa00100ec0 0x7faa00101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa05d56640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa00100500 0x7faa001016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa06557640 1 -- 172.21.15.32:0/2345965346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa00100500 msgr2=0x7faa001016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa06557640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa00100500 0x7faa001016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa06557640 1 -- 172.21.15.32:0/2345965346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa00100ec0 msgr2=0x7faa00101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa06557640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa00100ec0 0x7faa00101c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.029+0000 7faa06557640 1 -- 172.21.15.32:0/2345965346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa001a6ed0 con 0x7faa00106fe0 2024-04-16T05:41:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.030+0000 7faa06557640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 0x7faa00102160 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800d750 tx=0x7fa9f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.030+0000 7fa9feffd640 1 -- 172.21.15.32:0/2345965346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8017070 con 0x7faa00106fe0 2024-04-16T05:41:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.030+0000 7fa9feffd640 1 -- 172.21.15.32:0/2345965346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f8005500 con 0x7faa00106fe0 2024-04-16T05:41:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.030+0000 7faa07fba640 1 -- 172.21.15.32:0/2345965346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001a71c0 con 0x7faa00106fe0 2024-04-16T05:41:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.030+0000 7fa9feffd640 1 -- 172.21.15.32:0/2345965346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004d90 con 0x7faa00106fe0 2024-04-16T05:41:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.030+0000 7faa07fba640 1 -- 172.21.15.32:0/2345965346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a76c0 con 0x7faa00106fe0 2024-04-16T05:41:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.032+0000 7fa9feffd640 1 -- 172.21.15.32:0/2345965346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9f8004f30 con 0x7faa00106fe0 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.032+0000 7faa0aba1740 1 Processor -- start 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.032+0000 7faa0aba1740 1 -- start start 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178be50 0x55d71178c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178c770 0x55d711794e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d711795350 0x55d711797740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa0aba1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d7116f1180 con 0x55d71178be50 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa0aba1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d71168e360 con 0x55d711795350 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa0aba1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7116ed140 con 0x55d71178c770 2024-04-16T05:41:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7fa9feffd640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9e80a3d20 0x7fa9e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178be50 0x55d71178c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178be50 0x55d71178c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60802/0 (socket says 172.21.15.32:60802) 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 -- 172.21.15.32:0/2594374418 learned_addr learned my addr 172.21.15.32:0/2594374418 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05d56640 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178c770 0x55d711794e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa06557640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9e80a3d20 0x7fa9e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d711795350 0x55d711797740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7fa9feffd640 1 -- 172.21.15.32:0/2345965346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fa9f80d0480 con 0x7faa00106fe0 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 -- 172.21.15.32:0/2594374418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178c770 msgr2=0x55d711794e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178c770 0x55d711794e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 -- 172.21.15.32:0/2594374418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d711795350 msgr2=0x55d711797740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d711795350 0x55d711797740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 -- 172.21.15.32:0/2594374418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7114cd7d0 con 0x55d71178be50 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa05555640 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178be50 0x55d71178c230 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40125b0 tx=0x7fa9f400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8417085877f0061e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7faa06557640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9e80a3d20 0x7fa9e80a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8061b40 tx=0x7fa9f806db70 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.033+0000 7fa9e7fff640 1 -- 172.21.15.32:0/2594374418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4005020 con 0x55d71178be50 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7fa9e7fff640 1 -- 172.21.15.32:0/2594374418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f40056a0 con 0x55d71178be50 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7fa9e7fff640 1 -- 172.21.15.32:0/2594374418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40106c0 con 0x55d71178be50 2024-04-16T05:41:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 -- 172.21.15.32:0/2594374418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178be50 msgr2=0x55d71178c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178be50 0x55d71178c230 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40125b0 tx=0x7fa9f400b650 comp rx=0 tx=0).stop 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 -- 172.21.15.32:0/2594374418 shutdown_connections 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d711795350 0x55d711797740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178c770 0x55d711794e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 --2- 172.21.15.32:0/2594374418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178be50 0x55d71178c230 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 -- 172.21.15.32:0/2594374418 >> 172.21.15.32:0/2594374418 conn(0x55d711786580 msgr2=0x55d711786960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 -- 172.21.15.32:0/2594374418 shutdown_connections 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.034+0000 7faa0aba1740 1 -- 172.21.15.32:0/2594374418 wait complete. 2024-04-16T05:41:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 1 -- start start 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 1 Processor -- start 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178be50 0x55d711792380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178c770 0x55d7117928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d711795350 0x55d711792e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d711798140 con 0x55d711795350 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d7116aa2c0 con 0x55d71178c770 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7117982c0 con 0x55d71178be50 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa0aba1740 10 client.0 _ll_register_callbacks cb 0x55d711691090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa05555640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d711795350 0x55d711792e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa05555640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d711795350 0x55d711792e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60814/0 (socket says 172.21.15.32:60814) 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa05555640 1 -- 172.21.15.32:0/1970867426 learned_addr learned my addr 172.21.15.32:0/1970867426 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa05555640 1 -- 172.21.15.32:0/1970867426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178be50 msgr2=0x55d711792380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa05555640 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178be50 0x55d711792380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa05555640 1 -- 172.21.15.32:0/1970867426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178c770 msgr2=0x55d7117928c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.035+0000 7faa05555640 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178c770 0x55d7117928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa05555640 1 -- 172.21.15.32:0/1970867426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7114033f0 con 0x55d711795350 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa05555640 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d711795350 0x55d711792e00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4004df0 tx=0x7fa9f4012300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae30fc88eb65da78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7fa9e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1970867426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4004020 con 0x55d711795350 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1970867426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fa9f40041c0 con 0x55d711795350 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 1 -- 172.21.15.32:0/1970867426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f40056a0 con 0x55d711795350 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1970867426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9f402f050 con 0x55d711795350 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 10 client.17139 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 1 -- 172.21.15.32:0/1970867426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7113c30f0 con 0x55d711795350 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1970867426 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f40056a0 con 0x55d711795350 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1970867426 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9f4003a30 con 0x55d711795350 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 10 client.17139 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 20 client.17139 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d7117a5560 0x55d7117a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 4 client.17139 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.036+0000 7faa0aba1740 1 -- 172.21.15.32:0/1970867426 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7116feba0 con 0x55d7117a5560 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.037+0000 7faa05d56640 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d7117a5560 0x55d7117a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.037+0000 7fa9e57fa640 10 client.17139 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.037+0000 7faa05d56640 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d7117a5560 0x55d7117a7950 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db379fc354d8aa5d server_cookie=949bedd23fc7da54 in_seq=0 out_seq=0 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1970867426 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7116feba0 con 0x55d7117a5560 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7fa9e57fa640 10 client.17139 handle_command_reply: tid=0 2024-04-16T05:41:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 client.17139 shutdown 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 -- 172.21.15.32:0/1970867426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d711795350 msgr2=0x55d711792e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d711795350 0x55d711792e00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4004df0 tx=0x7fa9f4012300 comp rx=0 tx=0).stop 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 -- 172.21.15.32:0/1970867426 shutdown_connections 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d711795350 0x55d711792e00 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178c770 0x55d7117928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d7117a5560 0x55d7117a7950 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1970867426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d71178be50 0x55d711792380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 -- 172.21.15.32:0/1970867426 >> 172.21.15.32:0/1970867426 conn(0x55d711786580 msgr2=0x55d7117882d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7fa9e57fa640 0 client.17139 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 -- 172.21.15.32:0/1970867426 shutdown_connections 2024-04-16T05:41:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 1 -- 172.21.15.32:0/1970867426 wait complete. 2024-04-16T05:41:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.038+0000 7faa0aba1740 20 client.17139 trim_cache size 0 max 16384 2024-04-16T05:41:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa0aba1740 1 Processor -- start 2024-04-16T05:41:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa0aba1740 1 -- start start 2024-04-16T05:41:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 0x55d7117a1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d711784ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d711791c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa0aba1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b3ec0 con 0x55d71178c770 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa0aba1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b1790 con 0x55d71178be50 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa0aba1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa001af5e0 con 0x55d711795350 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05d56640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d711784ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05d56640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d711784ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60820/0 (socket says 172.21.15.32:60820) 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05d56640 1 -- 172.21.15.32:0/2345795125 learned_addr learned my addr 172.21.15.32:0/2345795125 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05555640 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d711791c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05555640 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 0x55d7117a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05d56640 1 -- 172.21.15.32:0/2345795125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 msgr2=0x55d711791c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05d56640 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d711791c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05d56640 1 -- 172.21.15.32:0/2345795125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 msgr2=0x55d7117a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05d56640 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 0x55d7117a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.056+0000 7faa05d56640 1 -- 172.21.15.32:0/2345795125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7117908d0 con 0x55d71178c770 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa05d56640 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d711784ed0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009ca0 tx=0x7fa9f0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3879df62b3696a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7fa9e77fe640 1 -- 172.21.15.32:0/2345795125 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0039040 con 0x55d71178c770 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7fa9e77fe640 1 -- 172.21.15.32:0/2345795125 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0032c90 con 0x55d71178c770 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7fa9e77fe640 1 -- 172.21.15.32:0/2345795125 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0003cf0 con 0x55d71178c770 2024-04-16T05:41:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa0aba1740 1 -- 172.21.15.32:0/2345795125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 msgr2=0x55d711784ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa0aba1740 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d711784ed0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009ca0 tx=0x7fa9f0005c50 comp rx=0 tx=0).stop 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa0aba1740 1 -- 172.21.15.32:0/2345795125 shutdown_connections 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa0aba1740 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d711791c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa0aba1740 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d711784ed0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa0aba1740 1 --2- 172.21.15.32:0/2345795125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 0x55d7117a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa0aba1740 1 -- 172.21.15.32:0/2345795125 >> 172.21.15.32:0/2345795125 conn(0x55d711786580 msgr2=0x55d711792bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.057+0000 7faa0aba1740 1 -- 172.21.15.32:0/2345795125 shutdown_connections 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 -- 172.21.15.32:0/2345795125 wait complete. 2024-04-16T05:41:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 -- start start 2024-04-16T05:41:00.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 Processor -- start 2024-04-16T05:41:00.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 0x55d71179d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d71179c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d71179c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa001afc50 con 0x55d71178c770 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b10c0 con 0x55d71178be50 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b0320 con 0x55d711795350 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa0aba1740 10 client.0 _ll_register_callbacks cb 0x7fa9f0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa05d56640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d71179c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa05d56640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d71179c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60828/0 (socket says 172.21.15.32:60828) 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.058+0000 7faa05d56640 1 -- 172.21.15.32:0/1445492777 learned_addr learned my addr 172.21.15.32:0/1445492777 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05555640 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 0x55d71179d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05555640 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d71179c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05d56640 1 -- 172.21.15.32:0/1445492777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 msgr2=0x55d71179c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05d56640 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d71179c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05d56640 1 -- 172.21.15.32:0/1445492777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 msgr2=0x55d71179d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05d56640 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 0x55d71179d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05d56640 1 -- 172.21.15.32:0/1445492777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d71179cb70 con 0x55d71178c770 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05555640 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d71179c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa05d56640 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d71179c0f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0002230 tx=0x7fa9f0032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae67f2839b4e5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7fa9e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1445492777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0039050 con 0x55d71178c770 2024-04-16T05:41:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa0aba1740 1 -- 172.21.15.32:0/1445492777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f0032c90 con 0x55d71178c770 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1445492777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fa9f0032790 con 0x55d71178c770 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1445492777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9f00329d0 con 0x55d71178c770 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa0aba1740 10 client.17145 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7faa0aba1740 1 -- 172.21.15.32:0/1445492777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d71180b700 con 0x55d71178c770 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.059+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1445492777 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0032c90 con 0x55d71178c770 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1445492777 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9f0034440 con 0x55d71178c770 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa0aba1740 10 client.17145 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa0aba1740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa0aba1740 20 client.17145 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d71180bdf0 0x55d71180e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa0aba1740 4 client.17145 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa0aba1740 1 -- 172.21.15.32:0/1445492777 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7116f8a10 con 0x55d71180bdf0 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa05555640 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d71180bdf0 0x55d71180e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa05555640 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d71180bdf0 0x55d71180e210 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd9e822c6c71360e server_cookie=5b554fc28e7afa01 in_seq=0 out_seq=0 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7fa9e57fa640 10 client.17145 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7fa9e57fa640 1 -- 172.21.15.32:0/1445492777 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d7116f8a10 con 0x55d71180bdf0 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7fa9e57fa640 10 client.17145 handle_command_reply: tid=0 2024-04-16T05:41:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.060+0000 7faa0aba1740 1 client.17145 shutdown 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 -- 172.21.15.32:0/1445492777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 msgr2=0x55d71179c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d71179c0f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0002230 tx=0x7fa9f0032ec0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 -- 172.21.15.32:0/1445492777 shutdown_connections 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d71180bdf0 0x55d71180e210 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d711795350 0x55d71179c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71178c770 0x55d71179c0f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7fa9e57fa640 0 client.17145 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 --2- 172.21.15.32:0/1445492777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d71178be50 0x55d71179d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 -- 172.21.15.32:0/1445492777 >> 172.21.15.32:0/1445492777 conn(0x55d711786580 msgr2=0x55d711787580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 -- 172.21.15.32:0/1445492777 shutdown_connections 2024-04-16T05:41:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 1 -- 172.21.15.32:0/1445492777 wait complete. 2024-04-16T05:41:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa0aba1740 20 client.17145 trim_cache size 0 max 16384 2024-04-16T05:41:00.069 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:41:00.069 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:41:00.069 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:41:00.069 INFO:teuthology.orchestra.run.smithi032.stdout: "255aab1f-ee21-4946-9c4f-fd3790ae4e72": { 2024-04-16T05:41:00.069 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:41:00.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "255aab1f-ee21-4946-9c4f-fd3790ae4e72", 2024-04-16T05:41:00.070 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:41:00.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:41:00.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:41:00.070 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:41:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 -- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9e80a3d20 msgr2=0x7fa9e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9e80a3d20 0x7fa9e80a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8061b40 tx=0x7fa9f806db70 comp rx=0 tx=0).stop 2024-04-16T05:41:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 -- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 msgr2=0x7faa00102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 0x7faa00102160 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800d750 tx=0x7fa9f800dc10 comp rx=0 tx=0).stop 2024-04-16T05:41:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 -- 172.21.15.32:0/2345965346 shutdown_connections 2024-04-16T05:41:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa9e80a3d20 0x7fa9e80a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00106fe0 0x7faa00102160 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa00100ec0 0x7faa00101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 --2- 172.21.15.32:0/2345965346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa00100500 0x7faa001016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 -- 172.21.15.32:0/2345965346 >> 172.21.15.32:0/2345965346 conn(0x7faa000774c0 msgr2=0x7faa00105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 -- 172.21.15.32:0/2345965346 shutdown_connections 2024-04-16T05:41:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:00.061+0000 7faa07fba640 1 -- 172.21.15.32:0/2345965346 wait complete. 2024-04-16T05:41:00.072 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:40:59 smithi032 ceph-mon[32351]: pgmap v184: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:00.073 INFO:tasks.cephfs.filesystem:scrub status for tag:255aab1f-ee21-4946-9c4f-fd3790ae4e72 - {'path': '/', 'tag': '255aab1f-ee21-4946-9c4f-fd3790ae4e72', 'options': 'recursive,force'} 2024-04-16T05:41:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:40:59 smithi134 ceph-mon[36153]: pgmap v184: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:00.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:40:59 smithi082 ceph-mon[36195]: pgmap v184: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:01.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:00 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:41:01.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:00 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:01.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:00 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:41:01.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:00 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:01.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:00 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:41:01.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:00 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:41:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:00 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:00 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:41:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:00 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:00 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:41:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:00 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:41:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:00 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:00 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:41:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:00 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:00 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:41:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:01 smithi134 ceph-mon[36153]: pgmap v185: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:01 smithi082 ceph-mon[36195]: pgmap v185: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:01 smithi032 ceph-mon[32351]: pgmap v185: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:04.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:03 smithi134 ceph-mon[36153]: pgmap v186: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:04.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:03 smithi082 ceph-mon[36195]: pgmap v186: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:04.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:03 smithi032 ceph-mon[32351]: pgmap v186: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:06.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:05 smithi134 ceph-mon[36153]: pgmap v187: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:06.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:05 smithi082 ceph-mon[36195]: pgmap v187: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:06.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:05 smithi032 ceph-mon[32351]: pgmap v187: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:07.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:06 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:41:07.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:06 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:41:07.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:06 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:41:08.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:07 smithi134 ceph-mon[36153]: pgmap v188: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:08.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:07 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:41:08.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:07 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:41:08.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:07 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:41:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:07 smithi082 ceph-mon[36195]: pgmap v188: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:07 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:41:08.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:07 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:41:08.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:07 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:41:08.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:07 smithi032 ceph-mon[32351]: pgmap v188: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:08.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:07 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:41:08.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:07 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:41:08.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:07 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:41:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:09 smithi134 ceph-mon[36153]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:10.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:09 smithi082 ceph-mon[36195]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:10.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:09 smithi032 ceph-mon[32351]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:11 smithi134 ceph-mon[36153]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:12.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:11 smithi082 ceph-mon[36195]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:12.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:11 smithi032 ceph-mon[32351]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-16T05:41:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:13 smithi134 ceph-mon[36153]: pgmap v191: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:14.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:13 smithi082 ceph-mon[36195]: pgmap v191: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:13 smithi032 ceph-mon[32351]: pgmap v191: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-16T05:41:16.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:15 smithi134 ceph-mon[36153]: pgmap v192: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:41:16.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:15 smithi082 ceph-mon[36195]: pgmap v192: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:41:16.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:15 smithi032 ceph-mon[32351]: pgmap v192: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:41:18.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:17 smithi134 ceph-mon[36153]: pgmap v193: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:41:18.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:17 smithi082 ceph-mon[36195]: pgmap v193: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:41:18.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:17 smithi032 ceph-mon[32351]: pgmap v193: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-16T05:41:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:19 smithi134 ceph-mon[36153]: pgmap v194: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:41:20.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:19 smithi082 ceph-mon[36195]: pgmap v194: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:41:20.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:19 smithi032 ceph-mon[32351]: pgmap v194: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T05:41:22.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:21 smithi134 ceph-mon[36153]: pgmap v195: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:22.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:21 smithi082 ceph-mon[36195]: pgmap v195: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:22.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:21 smithi032 ceph-mon[32351]: pgmap v195: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:24.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:23 smithi134 ceph-mon[36153]: pgmap v196: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:24.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:23 smithi082 ceph-mon[36195]: pgmap v196: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:24.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:23 smithi032 ceph-mon[32351]: pgmap v196: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:25 smithi134 ceph-mon[36153]: pgmap v197: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:26.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:25 smithi082 ceph-mon[36195]: pgmap v197: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:25 smithi032 ceph-mon[32351]: pgmap v197: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:28.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:27 smithi134 ceph-mon[36153]: pgmap v198: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:28.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:27 smithi082 ceph-mon[36195]: pgmap v198: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:28.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:27 smithi032 ceph-mon[32351]: pgmap v198: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:30.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:41:30.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:41:30.105 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:29 smithi032 ceph-mon[32351]: pgmap v199: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:30.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:29 smithi134 ceph-mon[36153]: pgmap v199: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:30.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:29 smithi082 ceph-mon[36195]: pgmap v199: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:30.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.196+0000 7f14d3fe9640 1 -- 172.21.15.32:0/769707606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc107310 msgr2=0x7f14cc10eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.196+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/769707606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc107310 0x7f14cc10eee0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009900 tx=0x7f14c402bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.196+0000 7f14d3fe9640 1 -- 172.21.15.32:0/769707606 shutdown_connections 2024-04-16T05:41:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.196+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/769707606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc107310 0x7f14cc10eee0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.196+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/769707606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc102bf0 0x7f14cc106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/769707606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc102240 0x7f14cc102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 -- 172.21.15.32:0/769707606 >> 172.21.15.32:0/769707606 conn(0x7f14cc079450 msgr2=0x7f14cc079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 -- 172.21.15.32:0/769707606 shutdown_connections 2024-04-16T05:41:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 -- 172.21.15.32:0/769707606 wait complete. 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 Processor -- start 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 -- start start 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc102240 0x7f14cc19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc107310 0x7f14cc19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc198630 0x7f14cc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1123b0 con 0x7f14cc102240 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc112230 con 0x7f14cc198630 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.197+0000 7f14d3fe9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc112530 con 0x7f14cc107310 2024-04-16T05:41:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc102240 0x7f14cc19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc102240 0x7f14cc19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57394/0 (socket says 172.21.15.32:57394) 2024-04-16T05:41:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 -- 172.21.15.32:0/2813334344 learned_addr learned my addr 172.21.15.32:0/2813334344 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1584640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc107310 0x7f14cc19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d2586640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc198630 0x7f14cc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 -- 172.21.15.32:0/2813334344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc107310 msgr2=0x7f14cc19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc107310 0x7f14cc19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 -- 172.21.15.32:0/2813334344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc198630 msgr2=0x7f14cc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc198630 0x7f14cc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 -- 172.21.15.32:0/2813334344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14cc199230 con 0x7f14cc102240 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d2586640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc198630 0x7f14cc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1584640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc107310 0x7f14cc19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d1d85640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc102240 0x7f14cc19e570 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f14c800db80 tx=0x7f14c800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14c2ffd640 1 -- 172.21.15.32:0/2813334344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8017070 con 0x7f14cc102240 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14c2ffd640 1 -- 172.21.15.32:0/2813334344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8003b70 con 0x7f14cc102240 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d3fe9640 1 -- 172.21.15.32:0/2813334344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14cc1a5310 con 0x7f14cc102240 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14c2ffd640 1 -- 172.21.15.32:0/2813334344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80106c0 con 0x7f14cc102240 2024-04-16T05:41:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.198+0000 7f14d3fe9640 1 -- 172.21.15.32:0/2813334344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14cc1a5770 con 0x7f14cc102240 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.201+0000 7f14c2ffd640 1 -- 172.21.15.32:0/2813334344 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14c8004020 con 0x7f14cc102240 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.201+0000 7f14d6bd3740 1 Processor -- start 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.201+0000 7f14d6bd3740 1 -- start start 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c6131230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6131770 0x5610c6139e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c613c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5610c6096180 con 0x5610c6130e50 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5610c6033360 con 0x5610c613a350 2024-04-16T05:41:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610c6092140 con 0x5610c6131770 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c6131230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c6131230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57404/0 (socket says 172.21.15.32:57404) 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 -- 172.21.15.32:0/3416586914 learned_addr learned my addr 172.21.15.32:0/3416586914 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14c2ffd640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14b80a3da0 0x7f14b80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d1d85640 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6131770 0x5610c6139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d1584640 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c613c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 -- 172.21.15.32:0/3416586914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6131770 msgr2=0x5610c6139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6131770 0x5610c6139e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 -- 172.21.15.32:0/3416586914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 msgr2=0x5610c613c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c613c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 -- 172.21.15.32:0/3416586914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c5e727d0 con 0x5610c6130e50 2024-04-16T05:41:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14c2ffd640 1 -- 172.21.15.32:0/2813334344 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f14c80d0960 con 0x7f14cc102240 2024-04-16T05:41:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14b80a3da0 0x7f14b80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14d2586640 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c6131230 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f14c4006010 tx=0x7f14c402c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e46158c3d49913b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.202+0000 7f14b7fff640 1 -- 172.21.15.32:0/3416586914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4043070 con 0x5610c6130e50 2024-04-16T05:41:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14b7fff640 1 -- 172.21.15.32:0/3416586914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c4004ba0 con 0x5610c6130e50 2024-04-16T05:41:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14b7fff640 1 -- 172.21.15.32:0/3416586914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c403f8c0 con 0x5610c6130e50 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d2586640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14b80a3da0 0x7f14b80a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14c4036040 tx=0x7f14c4046000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3416586914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 msgr2=0x5610c6131230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c6131230 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f14c4006010 tx=0x7f14c402c720 comp rx=0 tx=0).stop 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3416586914 shutdown_connections 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c613c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6131770 0x5610c6139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/3416586914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c6131230 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3416586914 >> 172.21.15.32:0/3416586914 conn(0x5610c612b580 msgr2=0x5610c612b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3416586914 shutdown_connections 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.203+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3416586914 wait complete. 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 1 -- start start 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 1 Processor -- start 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c61373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6131770 0x5610c61378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 0x5610c6137e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5610c613d140 con 0x5610c6130e50 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5610c604f2c0 con 0x5610c6131770 2024-04-16T05:41:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610c613d2c0 con 0x5610c613a350 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d1d85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c61373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d6bd3740 10 client.0 _ll_register_callbacks cb 0x5610c6036090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d1d85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c61373b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57412/0 (socket says 172.21.15.32:57412) 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.204+0000 7f14d1d85640 1 -- 172.21.15.32:0/1219835404 learned_addr learned my addr 172.21.15.32:0/1219835404 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d1584640 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6131770 0x5610c61378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d2586640 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 0x5610c6137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d1d85640 1 -- 172.21.15.32:0/1219835404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 msgr2=0x5610c6137e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d1d85640 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 0x5610c6137e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d1d85640 1 -- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6131770 msgr2=0x5610c61378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d1d85640 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6131770 0x5610c61378f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d1d85640 1 -- 172.21.15.32:0/1219835404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c5da83f0 con 0x5610c6130e50 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d1584640 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6131770 0x5610c61378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d1d85640 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c61373b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f14c8098f70 tx=0x7f14c809bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b36e290b3ce70f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14b57fa640 1 -- 172.21.15.32:0/1219835404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c809c5a0 con 0x5610c6130e50 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14b57fa640 1 -- 172.21.15.32:0/1219835404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f14c8098c10 con 0x5610c6130e50 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1219835404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c4004ba0 con 0x5610c6130e50 2024-04-16T05:41:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14b57fa640 1 -- 172.21.15.32:0/1219835404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f14c80a0950 con 0x5610c6130e50 2024-04-16T05:41:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d6bd3740 10 client.17157 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1219835404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5610c5d680f0 con 0x5610c6130e50 2024-04-16T05:41:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.205+0000 7f14b57fa640 1 -- 172.21.15.32:0/1219835404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c4004ba0 con 0x5610c6130e50 2024-04-16T05:41:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14b57fa640 1 -- 172.21.15.32:0/1219835404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f14c809c800 con 0x5610c6130e50 2024-04-16T05:41:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14d6bd3740 10 client.17157 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14d6bd3740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14d6bd3740 20 client.17157 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5610c614a730 0x5610c614cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14d6bd3740 4 client.17157 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1219835404 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610c60a3ba0 con 0x5610c614a730 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14d1584640 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5610c614a730 0x5610c614cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14d1584640 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5610c614a730 0x5610c614cbf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10679fdec4c3a5f1 server_cookie=77632eae43908ca6 in_seq=0 out_seq=0 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.206+0000 7f14b57fa640 10 client.17157 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14b57fa640 1 -- 172.21.15.32:0/1219835404 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610c60a3ba0 con 0x5610c614a730 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14b57fa640 10 client.17157 handle_command_reply: tid=0 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 client.17157 shutdown 2024-04-16T05:41:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1219835404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 msgr2=0x5610c61373b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c61373b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f14c8098f70 tx=0x7f14c809bf00 comp rx=0 tx=0).stop 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1219835404 shutdown_connections 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5610c614a730 0x5610c614cbf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 0x5610c6137e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6131770 0x5610c61378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1219835404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6130e50 0x5610c61373b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14b57fa640 0 client.17157 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1219835404 >> 172.21.15.32:0/1219835404 conn(0x5610c612b580 msgr2=0x5610c612d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1219835404 shutdown_connections 2024-04-16T05:41:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1219835404 wait complete. 2024-04-16T05:41:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.207+0000 7f14d6bd3740 20 client.17157 trim_cache size 0 max 16384 2024-04-16T05:41:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d6bd3740 1 Processor -- start 2024-04-16T05:41:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d6bd3740 1 -- start start 2024-04-16T05:41:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6130e50 0x5610c6146030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6137c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c61524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1b1ba0 con 0x5610c6131770 2024-04-16T05:41:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1af470 con 0x5610c613a350 2024-04-16T05:41:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1ad2c0 con 0x5610c6130e50 2024-04-16T05:41:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d1584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d1584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6137c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57426/0 (socket says 172.21.15.32:57426) 2024-04-16T05:41:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.225+0000 7f14d1584640 1 -- 172.21.15.32:0/3573156437 learned_addr learned my addr 172.21.15.32:0/3573156437 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d2586640 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c61524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d1d85640 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6130e50 0x5610c6146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d1584640 1 -- 172.21.15.32:0/3573156437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6130e50 msgr2=0x5610c6146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d1584640 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6130e50 0x5610c6146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d1584640 1 -- 172.21.15.32:0/3573156437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 msgr2=0x5610c61524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d1584640 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c61524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d1584640 1 -- 172.21.15.32:0/3573156437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c613d020 con 0x5610c6131770 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d2586640 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c61524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14d1584640 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6137c50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f14bc005e10 tx=0x7f14bc005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3edc5744ec8c8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14b77fe640 1 -- 172.21.15.32:0/3573156437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0130d0 con 0x5610c6131770 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14b77fe640 1 -- 172.21.15.32:0/3573156437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14bc013270 con 0x5610c6131770 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.226+0000 7f14b77fe640 1 -- 172.21.15.32:0/3573156437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc005290 con 0x5610c6131770 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3573156437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 msgr2=0x5610c6137c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6137c50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f14bc005e10 tx=0x7f14bc005da0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3573156437 shutdown_connections 2024-04-16T05:41:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c613a350 0x5610c61524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6137c50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/3573156437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c6130e50 0x5610c6146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3573156437 >> 172.21.15.32:0/3573156437 conn(0x5610c612b580 msgr2=0x5610c612a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3573156437 shutdown_connections 2024-04-16T05:41:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 -- 172.21.15.32:0/3573156437 wait complete. 2024-04-16T05:41:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 -- start start 2024-04-16T05:41:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.227+0000 7f14d6bd3740 1 Processor -- start 2024-04-16T05:41:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6130e50 0x5610c61401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6140720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d6bd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 0x5610c6140c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1ad930 con 0x5610c6131770 2024-04-16T05:41:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1aeda0 con 0x5610c6130e50 2024-04-16T05:41:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d6bd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1ae000 con 0x5610c613a350 2024-04-16T05:41:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d6bd3740 10 client.0 _ll_register_callbacks cb 0x7f14c809c370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57428/0 (socket says 172.21.15.32:57428) 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 -- 172.21.15.32:0/1166952695 learned_addr learned my addr 172.21.15.32:0/1166952695 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1d85640 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6130e50 0x5610c61401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d2586640 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 0x5610c6140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 -- 172.21.15.32:0/1166952695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 msgr2=0x5610c6140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 0x5610c6140c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 -- 172.21.15.32:0/1166952695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6130e50 msgr2=0x5610c61401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6130e50 0x5610c61401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 -- 172.21.15.32:0/1166952695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c613c740 con 0x5610c6131770 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d1584640 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6140720 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00ccb0 tx=0x7f14bc01ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a7d1c08ed97f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14b57fa640 1 -- 172.21.15.32:0/1166952695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc004050 con 0x5610c6131770 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14b57fa640 1 -- 172.21.15.32:0/1166952695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f14bc01a560 con 0x5610c6131770 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.228+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1166952695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14bc013270 con 0x5610c6131770 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14b57fa640 1 -- 172.21.15.32:0/1166952695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f14bc0041f0 con 0x5610c6131770 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d6bd3740 10 client.17163 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1166952695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5610c61b0ff0 con 0x5610c6131770 2024-04-16T05:41:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14b57fa640 1 -- 172.21.15.32:0/1166952695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14bc013270 con 0x5610c6131770 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14b57fa640 1 -- 172.21.15.32:0/1166952695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f14bc011440 con 0x5610c6131770 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d6bd3740 10 client.17163 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d6bd3740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d6bd3740 20 client.17163 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5610c61b16e0 0x5610c61b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d6bd3740 4 client.17163 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1166952695 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610c609da10 con 0x5610c61b16e0 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.229+0000 7f14d1d85640 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5610c61b16e0 0x5610c61b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d1d85640 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5610c61b16e0 0x5610c61b3b00 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f38c3e672a7e565 server_cookie=312e34115c00b74e in_seq=0 out_seq=0 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14b57fa640 10 client.17163 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14b57fa640 1 -- 172.21.15.32:0/1166952695 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5610c609da10 con 0x5610c61b16e0 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14b57fa640 10 client.17163 handle_command_reply: tid=0 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 client.17163 shutdown 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1166952695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 msgr2=0x5610c6140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6140720 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00ccb0 tx=0x7f14bc01ced0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1166952695 shutdown_connections 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5610c61b16e0 0x5610c61b3b00 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610c613a350 0x5610c6140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610c6131770 0x5610c6140720 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 --2- 172.21.15.32:0/1166952695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610c6130e50 0x5610c61401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14b57fa640 0 client.17163 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:30.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1166952695 >> 172.21.15.32:0/1166952695 conn(0x5610c612b580 msgr2=0x5610c612c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1166952695 shutdown_connections 2024-04-16T05:41:30.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 1 -- 172.21.15.32:0/1166952695 wait complete. 2024-04-16T05:41:30.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.230+0000 7f14d6bd3740 20 client.17163 trim_cache size 0 max 16384 2024-04-16T05:41:30.238 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:41:30.239 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:41:30.239 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:41:30.239 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:41:30.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 -- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14b80a3da0 msgr2=0x7f14b80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14b80a3da0 0x7f14b80a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14c4036040 tx=0x7f14c4046000 comp rx=0 tx=0).stop 2024-04-16T05:41:30.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 -- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc102240 msgr2=0x7f14cc19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc102240 0x7f14cc19e570 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f14c800db80 tx=0x7f14c800b630 comp rx=0 tx=0).stop 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 -- 172.21.15.32:0/2813334344 shutdown_connections 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc198630 0x7f14cc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc107310 0x7f14cc19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14b80a3da0 0x7f14b80a6260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 --2- 172.21.15.32:0/2813334344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc102240 0x7f14cc19e570 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 -- 172.21.15.32:0/2813334344 >> 172.21.15.32:0/2813334344 conn(0x7f14cc079450 msgr2=0x7f14cc0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 -- 172.21.15.32:0/2813334344 shutdown_connections 2024-04-16T05:41:30.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.231+0000 7f14d3fe9640 1 -- 172.21.15.32:0/2813334344 wait complete. 2024-04-16T05:41:30.242 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:41:30.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 -- 172.21.15.32:0/1757526965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0101b70 msgr2=0x7f92b0105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 --2- 172.21.15.32:0/1757526965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0101b70 0x7f92b0105b60 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009900 tx=0x7f92a402bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 -- 172.21.15.32:0/1757526965 shutdown_connections 2024-04-16T05:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 --2- 172.21.15.32:0/1757526965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b0106240 0x7f92b010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 --2- 172.21.15.32:0/1757526965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0101b70 0x7f92b0105b60 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 --2- 172.21.15.32:0/1757526965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92b01011c0 0x7f92b01015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 -- 172.21.15.32:0/1757526965 >> 172.21.15.32:0/1757526965 conn(0x7f92b0078060 msgr2=0x7f92b0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 -- 172.21.15.32:0/1757526965 shutdown_connections 2024-04-16T05:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 -- 172.21.15.32:0/1757526965 wait complete. 2024-04-16T05:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.388+0000 7f92b5c14640 1 Processor -- start 2024-04-16T05:41:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92b5c14640 1 -- start start 2024-04-16T05:41:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92b5c14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92b01011c0 0x7f92b019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92b5c14640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106240 0x7f92b019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92b5c14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b0198770 0x7f92b0198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92b5c14640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0112410 con 0x7f92b0106240 2024-04-16T05:41:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92b5c14640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0112290 con 0x7f92b0198770 2024-04-16T05:41:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92b5c14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0112590 con 0x7f92b01011c0 2024-04-16T05:41:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106240 0x7f92b019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106240 0x7f92b019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57448/0 (socket says 172.21.15.32:57448) 2024-04-16T05:41:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 -- 172.21.15.32:0/2312430535 learned_addr learned my addr 172.21.15.32:0/2312430535 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92af7fe640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92b01011c0 0x7f92b019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92affff640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b0198770 0x7f92b0198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 -- 172.21.15.32:0/2312430535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92b01011c0 msgr2=0x7f92b019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92b01011c0 0x7f92b019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 -- 172.21.15.32:0/2312430535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b0198770 msgr2=0x7f92b0198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b0198770 0x7f92b0198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 -- 172.21.15.32:0/2312430535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b0199370 con 0x7f92b0106240 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.389+0000 7f92aeffd640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106240 0x7f92b019eba0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f92a4036040 tx=0x7f92a402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.390+0000 7f92acff9640 1 -- 172.21.15.32:0/2312430535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4041070 con 0x7f92b0106240 2024-04-16T05:41:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.390+0000 7f92acff9640 1 -- 172.21.15.32:0/2312430535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a40054e0 con 0x7f92b0106240 2024-04-16T05:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.390+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b0199600 con 0x7f92b0106240 2024-04-16T05:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.390+0000 7f92acff9640 1 -- 172.21.15.32:0/2312430535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4034440 con 0x7f92b0106240 2024-04-16T05:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.390+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01a56f0 con 0x7f92b0106240 2024-04-16T05:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.391+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b0102640 con 0x7f92b0106240 2024-04-16T05:41:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.392+0000 7f92acff9640 1 -- 172.21.15.32:0/2312430535 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f92a4044070 con 0x7f92b0106240 2024-04-16T05:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.393+0000 7f92acff9640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92980a3d20 0x7f92980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.393+0000 7f92af7fe640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92980a3d20 0x7f92980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.393+0000 7f92acff9640 1 -- 172.21.15.32:0/2312430535 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f92a40f2be0 con 0x7f92b0106240 2024-04-16T05:41:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.393+0000 7f92af7fe640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92980a3d20 0x7f92980a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009b60 tx=0x7f92a0005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.397+0000 7f92acff9640 1 -- 172.21.15.32:0/2312430535 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92a40bf050 con 0x7f92b0106240 2024-04-16T05:41:30.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.624+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92b0101b70 con 0x7f92b0106240 2024-04-16T05:41:30.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.625+0000 7f92acff9640 1 -- 172.21.15.32:0/2312430535 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f92a40be950 con 0x7f92b0106240 2024-04-16T05:41:30.624 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:41:30.624 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:41:30.624 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:41:30.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92980a3d20 msgr2=0x7f92980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92980a3d20 0x7f92980a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009b60 tx=0x7f92a0005f90 comp rx=0 tx=0).stop 2024-04-16T05:41:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106240 msgr2=0x7f92b019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106240 0x7f92b019eba0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f92a4036040 tx=0x7f92a402cbf0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 shutdown_connections 2024-04-16T05:41:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b0198770 0x7f92b0198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f92980a3d20 0x7f92980a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106240 0x7f92b019eba0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 --2- 172.21.15.32:0/2312430535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92b01011c0 0x7f92b019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 >> 172.21.15.32:0/2312430535 conn(0x7f92b0078060 msgr2=0x7f92b00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 shutdown_connections 2024-04-16T05:41:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.627+0000 7f92b5c14640 1 -- 172.21.15.32:0/2312430535 wait complete. 2024-04-16T05:41:30.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:41:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 -- 172.21.15.32:0/1910049429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38102bd0 msgr2=0x7fef38106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/1910049429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38102bd0 0x7fef38106c10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009900 tx=0x7fef2c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 -- 172.21.15.32:0/1910049429 shutdown_connections 2024-04-16T05:41:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/1910049429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef381072f0 0x7fef3810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/1910049429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef38102bd0 0x7fef38106c10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/1910049429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef38102220 0x7fef38102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 -- 172.21.15.32:0/1910049429 >> 172.21.15.32:0/1910049429 conn(0x7fef380fdac0 msgr2=0x7fef380ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 -- 172.21.15.32:0/1910049429 shutdown_connections 2024-04-16T05:41:30.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 -- 172.21.15.32:0/1910049429 wait complete. 2024-04-16T05:41:30.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 Processor -- start 2024-04-16T05:41:30.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.746+0000 7fef3e3bd640 1 -- start start 2024-04-16T05:41:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3e3bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef38102220 0x7fef3819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3e3bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef381072f0 0x7fef3819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3e3bd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef381986e0 0x7fef38198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3e3bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef38112380 con 0x7fef381072f0 2024-04-16T05:41:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3e3bd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fef38112200 con 0x7fef381986e0 2024-04-16T05:41:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3e3bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef38112500 con 0x7fef38102220 2024-04-16T05:41:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef381072f0 0x7fef3819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef381072f0 0x7fef3819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57456/0 (socket says 172.21.15.32:57456) 2024-04-16T05:41:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 -- 172.21.15.32:0/4120477785 learned_addr learned my addr 172.21.15.32:0/4120477785 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef37fff640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef38102220 0x7fef3819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3c95a640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef381986e0 0x7fef38198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 -- 172.21.15.32:0/4120477785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef38102220 msgr2=0x7fef3819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef38102220 0x7fef3819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 -- 172.21.15.32:0/4120477785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef381986e0 msgr2=0x7fef38198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef381986e0 0x7fef38198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 -- 172.21.15.32:0/4120477785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef381992e0 con 0x7fef381072f0 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3c95a640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef381986e0 0x7fef38198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef37fff640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef38102220 0x7fef3819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef377fe640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef381072f0 0x7fef3819eb10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fef2c036040 tx=0x7fef2c02ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef357fa640 1 -- 172.21.15.32:0/4120477785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c041070 con 0x7fef381072f0 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef357fa640 1 -- 172.21.15.32:0/4120477785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef2c0054e0 con 0x7fef381072f0 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef3e3bd640 1 -- 172.21.15.32:0/4120477785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef38199570 con 0x7fef381072f0 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.747+0000 7fef357fa640 1 -- 172.21.15.32:0/4120477785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c034440 con 0x7fef381072f0 2024-04-16T05:41:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.748+0000 7fef3e3bd640 1 -- 172.21.15.32:0/4120477785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef381a5660 con 0x7fef381072f0 2024-04-16T05:41:30.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef357fa640 1 -- 172.21.15.32:0/4120477785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef2c044070 con 0x7fef381072f0 2024-04-16T05:41:30.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef40fa9740 1 Processor -- start 2024-04-16T05:41:30.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef40fa9740 1 -- start start 2024-04-16T05:41:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 0x558e816b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816b6360 0x558e816b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef357fa640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fef200a3d20 0x7fef200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef40fa9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558e816120e0 con 0x558e816ace60 2024-04-16T05:41:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef40fa9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558e815af520 con 0x558e816b6360 2024-04-16T05:41:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.750+0000 7fef40fa9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e8160e0a0 con 0x558e816ad780 2024-04-16T05:41:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57466/0 (socket says 172.21.15.32:57466) 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 -- 172.21.15.32:0/1026762885 learned_addr learned my addr 172.21.15.32:0/1026762885 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef357fa640 1 -- 172.21.15.32:0/4120477785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fef2c0f2760 con 0x7fef381072f0 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fef200a3d20 0x7fef200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef37fff640 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 0x558e816b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef377fe640 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816b6360 0x558e816b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 -- 172.21.15.32:0/1026762885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 msgr2=0x558e816b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 0x558e816b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 -- 172.21.15.32:0/1026762885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816b6360 msgr2=0x558e816b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816b6360 0x558e816b8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 -- 172.21.15.32:0/1026762885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e815d78d0 con 0x558e816ace60 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816ad240 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fef28012aa0 tx=0x7fef28012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8968facc70e8e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef3c95a640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fef200a3d20 0x7fef200a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef28005fd0 tx=0x7fef28014000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef1e7fc640 1 -- 172.21.15.32:0/1026762885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28020070 con 0x558e816ace60 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef1e7fc640 1 -- 172.21.15.32:0/1026762885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef28012460 con 0x558e816ace60 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.751+0000 7fef1e7fc640 1 -- 172.21.15.32:0/1026762885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2801cdd0 con 0x558e816ace60 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 -- 172.21.15.32:0/1026762885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 msgr2=0x558e816ad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816ad240 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fef28012aa0 tx=0x7fef28012f60 comp rx=0 tx=0).stop 2024-04-16T05:41:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 -- 172.21.15.32:0/1026762885 shutdown_connections 2024-04-16T05:41:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816b6360 0x558e816b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 0x558e816b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 --2- 172.21.15.32:0/1026762885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816ad240 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 -- 172.21.15.32:0/1026762885 >> 172.21.15.32:0/1026762885 conn(0x558e816a7590 msgr2=0x558e816a7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 -- 172.21.15.32:0/1026762885 shutdown_connections 2024-04-16T05:41:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 -- 172.21.15.32:0/1026762885 wait complete. 2024-04-16T05:41:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.752+0000 7fef40fa9740 1 -- start start 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef40fa9740 1 Processor -- start 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ad780 0x558e816b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 0x558e816b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef40fa9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558e816b8fa0 con 0x558e816ad780 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef40fa9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558e815cb4d0 con 0x558e816ace60 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef40fa9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e816b9120 con 0x558e816b6360 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef40fa9740 10 client.0 _ll_register_callbacks cb 0x558e815b02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ad780 0x558e816b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ad780 0x558e816b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57468/0 (socket says 172.21.15.32:57468) 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef377fe640 1 -- 172.21.15.32:0/534915835 learned_addr learned my addr 172.21.15.32:0/534915835 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef37fff640 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef3c95a640 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 0x558e816b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef377fe640 1 -- 172.21.15.32:0/534915835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 msgr2=0x558e816b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef377fe640 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 0x558e816b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef377fe640 1 -- 172.21.15.32:0/534915835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 msgr2=0x558e816b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef377fe640 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef377fe640 1 -- 172.21.15.32:0/534915835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e813243f0 con 0x558e816ad780 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.753+0000 7fef37fff640 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef377fe640 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ad780 0x558e816b3a10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fef2c04c000 tx=0x7fef2c0bca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62c19bf59ef89f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef177fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef177fe640 1 -- 172.21.15.32:0/534915835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c04b040 con 0x558e816ad780 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef177fe640 1 -- 172.21.15.32:0/534915835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fef2c0efd90 con 0x558e816ad780 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 1 -- 172.21.15.32:0/534915835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef28012460 con 0x558e816ad780 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef177fe640 1 -- 172.21.15.32:0/534915835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef2c0eff90 con 0x558e816ad780 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 10 client.17181 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 1 -- 172.21.15.32:0/534915835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558e813ee7d0 con 0x558e816ad780 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef177fe640 1 -- 172.21.15.32:0/534915835 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef28012460 con 0x558e816ad780 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef177fe640 1 -- 172.21.15.32:0/534915835 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fef2c0c6c50 con 0x558e816ad780 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 10 client.17181 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 10 client.17181 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 20 client.17181 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558e816c6300 0x558e816c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 4 client.17181 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.754+0000 7fef40fa9740 1 -- 172.21.15.32:0/534915835 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e8161fb00 con 0x558e816c6300 2024-04-16T05:41:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.755+0000 7fef37fff640 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558e816c6300 0x558e816c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.755+0000 7fef177fe640 10 client.17181 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:41:30.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.755+0000 7fef37fff640 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558e816c6300 0x558e816c8710 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfb67411e82628c1 server_cookie=6690851d19e602d5 in_seq=0 out_seq=0 2024-04-16T05:41:30.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef177fe640 1 -- 172.21.15.32:0/534915835 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e8161fb00 con 0x558e816c6300 2024-04-16T05:41:30.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef177fe640 10 client.17181 handle_command_reply: tid=0 2024-04-16T05:41:30.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 client.17181 shutdown 2024-04-16T05:41:30.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 -- 172.21.15.32:0/534915835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ad780 msgr2=0x558e816b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ad780 0x558e816b3a10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fef2c04c000 tx=0x7fef2c0bca10 comp rx=0 tx=0).stop 2024-04-16T05:41:30.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 -- 172.21.15.32:0/534915835 shutdown_connections 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558e816c6300 0x558e816c8710 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 0x558e816b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef177fe640 0 client.17181 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ad780 0x558e816b3a10 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 --2- 172.21.15.32:0/534915835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 -- 172.21.15.32:0/534915835 >> 172.21.15.32:0/534915835 conn(0x558e816a7590 msgr2=0x558e816a92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 -- 172.21.15.32:0/534915835 shutdown_connections 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 1 -- 172.21.15.32:0/534915835 wait complete. 2024-04-16T05:41:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.756+0000 7fef40fa9740 20 client.17181 trim_cache size 0 max 16384 2024-04-16T05:41:30.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef40fa9740 1 Processor -- start 2024-04-16T05:41:30.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef40fa9740 1 -- start start 2024-04-16T05:41:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ad780 0x558e816c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 0x558e816a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef40fa9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef381b1b40 con 0x558e816ace60 2024-04-16T05:41:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef40fa9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fef381af410 con 0x558e816ad780 2024-04-16T05:41:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef40fa9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef381ad260 con 0x558e816b6360 2024-04-16T05:41:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57476/0 (socket says 172.21.15.32:57476) 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef37fff640 1 -- 172.21.15.32:0/2832033370 learned_addr learned my addr 172.21.15.32:0/2832033370 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.771+0000 7fef3c95a640 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 0x558e816a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef377fe640 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ad780 0x558e816c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef37fff640 1 -- 172.21.15.32:0/2832033370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 msgr2=0x558e816a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef37fff640 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 0x558e816a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef37fff640 1 -- 172.21.15.32:0/2832033370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ad780 msgr2=0x558e816c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef37fff640 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ad780 0x558e816c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef37fff640 1 -- 172.21.15.32:0/2832033370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e816b2480 con 0x558e816ace60 2024-04-16T05:41:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef377fe640 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ad780 0x558e816c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef37fff640 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816b5c00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x558e816c8f90 tx=0x7fef30013b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b140722738a261 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.772+0000 7fef1dffb640 1 -- 172.21.15.32:0/2832033370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30004d20 con 0x558e816ace60 2024-04-16T05:41:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef1dffb640 1 -- 172.21.15.32:0/2832033370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef30023030 con 0x558e816ace60 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef1dffb640 1 -- 172.21.15.32:0/2832033370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30005280 con 0x558e816ace60 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 -- 172.21.15.32:0/2832033370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 msgr2=0x558e816b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816b5c00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x558e816c8f90 tx=0x7fef30013b90 comp rx=0 tx=0).stop 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 -- 172.21.15.32:0/2832033370 shutdown_connections 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816b6360 0x558e816a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ad780 0x558e816c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 --2- 172.21.15.32:0/2832033370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816ace60 0x558e816b5c00 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 -- 172.21.15.32:0/2832033370 >> 172.21.15.32:0/2832033370 conn(0x558e816a7590 msgr2=0x558e816a9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 -- 172.21.15.32:0/2832033370 shutdown_connections 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.773+0000 7fef40fa9740 1 -- 172.21.15.32:0/2832033370 wait complete. 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 1 -- start start 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 1 Processor -- start 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 0x558e816ba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816b6360 0x558e816ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef381ad8d0 con 0x558e816b6360 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fef381aed40 con 0x558e816ace60 2024-04-16T05:41:30.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef381adfa0 con 0x558e816ad780 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef40fa9740 10 client.0 _ll_register_callbacks cb 0x7fef2c0be3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef3c95a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816b6360 0x558e816ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef3c95a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816b6360 0x558e816ba930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57492/0 (socket says 172.21.15.32:57492) 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef3c95a640 1 -- 172.21.15.32:0/3590283248 learned_addr learned my addr 172.21.15.32:0/3590283248 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef37fff640 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.774+0000 7fef377fe640 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 0x558e816ba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef3c95a640 1 -- 172.21.15.32:0/3590283248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 msgr2=0x558e816ba3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef3c95a640 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 0x558e816ba3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef3c95a640 1 -- 172.21.15.32:0/3590283248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 msgr2=0x558e816b9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef3c95a640 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef3c95a640 1 -- 172.21.15.32:0/3590283248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8172b860 con 0x558e816b6360 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef37fff640 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef3c95a640 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816b6360 0x558e816ba930 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fef28012690 tx=0x7fef280126c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13be55d81af2ae57 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef177fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef177fe640 1 -- 172.21.15.32:0/3590283248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28020040 con 0x558e816b6360 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef177fe640 1 -- 172.21.15.32:0/3590283248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fef2801c600 con 0x558e816b6360 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef40fa9740 1 -- 172.21.15.32:0/3590283248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef30023030 con 0x558e816b6360 2024-04-16T05:41:30.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef177fe640 1 -- 172.21.15.32:0/3590283248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef28039050 con 0x558e816b6360 2024-04-16T05:41:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef40fa9740 10 client.17187 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef40fa9740 1 -- 172.21.15.32:0/3590283248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558e8172cab0 con 0x558e816b6360 2024-04-16T05:41:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef177fe640 1 -- 172.21.15.32:0/3590283248 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef30023030 con 0x558e816b6360 2024-04-16T05:41:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.775+0000 7fef177fe640 1 -- 172.21.15.32:0/3590283248 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fef2801b030 con 0x558e816b6360 2024-04-16T05:41:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef40fa9740 10 client.17187 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef40fa9740 10 client.17187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:41:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef40fa9740 20 client.17187 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:30.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef40fa9740 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558e8172cf50 0x558e8172f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef40fa9740 4 client.17187 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef40fa9740 1 -- 172.21.15.32:0/3590283248 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e81619970 con 0x558e8172cf50 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef37fff640 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558e8172cf50 0x558e8172f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef37fff640 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558e8172cf50 0x558e8172f370 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99e5190fb80ed807 server_cookie=90f612a552ae7494 in_seq=0 out_seq=0 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.776+0000 7fef177fe640 10 client.17187 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef177fe640 1 -- 172.21.15.32:0/3590283248 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e81619970 con 0x558e8172cf50 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef177fe640 10 client.17187 handle_command_reply: tid=0 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 client.17187 shutdown 2024-04-16T05:41:30.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 -- 172.21.15.32:0/3590283248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816b6360 msgr2=0x558e816ba930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816b6360 0x558e816ba930 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fef28012690 tx=0x7fef280126c0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 -- 172.21.15.32:0/3590283248 shutdown_connections 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x558e8172cf50 0x558e8172f370 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e816b6360 0x558e816ba930 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e816ad780 0x558e816ba3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 --2- 172.21.15.32:0/3590283248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e816ace60 0x558e816b9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef177fe640 0 client.17187 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 -- 172.21.15.32:0/3590283248 >> 172.21.15.32:0/3590283248 conn(0x558e816a7590 msgr2=0x558e816b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 -- 172.21.15.32:0/3590283248 shutdown_connections 2024-04-16T05:41:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 1 -- 172.21.15.32:0/3590283248 wait complete. 2024-04-16T05:41:30.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.777+0000 7fef40fa9740 20 client.17187 trim_cache size 0 max 16384 2024-04-16T05:41:30.785 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:41:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 -- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fef200a3d20 msgr2=0x7fef200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fef200a3d20 0x7fef200a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef28005fd0 tx=0x7fef28014000 comp rx=0 tx=0).stop 2024-04-16T05:41:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 -- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef381072f0 msgr2=0x7fef3819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef381072f0 0x7fef3819eb10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fef2c036040 tx=0x7fef2c02ce20 comp rx=0 tx=0).stop 2024-04-16T05:41:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 -- 172.21.15.32:0/4120477785 shutdown_connections 2024-04-16T05:41:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef381986e0 0x7fef38198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fef200a3d20 0x7fef200a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef381072f0 0x7fef3819eb10 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 --2- 172.21.15.32:0/4120477785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef38102220 0x7fef3819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 -- 172.21.15.32:0/4120477785 >> 172.21.15.32:0/4120477785 conn(0x7fef380fdac0 msgr2=0x7fef380ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 -- 172.21.15.32:0/4120477785 shutdown_connections 2024-04-16T05:41:30.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.778+0000 7fef3e3bd640 1 -- 172.21.15.32:0/4120477785 wait complete. 2024-04-16T05:41:30.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:41:30.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1350324390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0107320 msgr2=0x7fd2e010eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1350324390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0107320 0x7fd2e010eef0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009ca0 tx=0x7fd2d802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1350324390 shutdown_connections 2024-04-16T05:41:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1350324390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0107320 0x7fd2e010eef0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1350324390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2e0102c00 0x7fd2e0106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1350324390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2e0102250 0x7fd2e0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1350324390 >> 172.21.15.32:0/1350324390 conn(0x7fd2e0079490 msgr2=0x7fd2e00798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1350324390 shutdown_connections 2024-04-16T05:41:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1350324390 wait complete. 2024-04-16T05:41:30.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.898+0000 7fd2e7b82640 1 Processor -- start 2024-04-16T05:41:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e7b82640 1 -- start start 2024-04-16T05:41:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e7b82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0102250 0x7fd2e019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e7b82640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2e0107320 0x7fd2e019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e7b82640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2e0198700 0x7fd2e0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e7b82640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01123e0 con 0x7fd2e0102250 2024-04-16T05:41:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e7b82640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0112260 con 0x7fd2e0107320 2024-04-16T05:41:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e7b82640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0112560 con 0x7fd2e0198700 2024-04-16T05:41:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0102250 0x7fd2e019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0102250 0x7fd2e019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57510/0 (socket says 172.21.15.32:57510) 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 -- 172.21.15.32:0/1728806180 learned_addr learned my addr 172.21.15.32:0/1728806180 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e511d640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2e0107320 0x7fd2e019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e611f640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2e0198700 0x7fd2e0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 -- 172.21.15.32:0/1728806180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2e0198700 msgr2=0x7fd2e0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2e0198700 0x7fd2e0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 -- 172.21.15.32:0/1728806180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2e0107320 msgr2=0x7fd2e019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2e0107320 0x7fd2e019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 -- 172.21.15.32:0/1728806180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e0199300 con 0x7fd2e0102250 2024-04-16T05:41:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e611f640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2e0198700 0x7fd2e0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e591e640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0102250 0x7fd2e019e5f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00d770 tx=0x7fd2dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2d6ffd640 1 -- 172.21.15.32:0/1728806180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc017070 con 0x7fd2e0102250 2024-04-16T05:41:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2d6ffd640 1 -- 172.21.15.32:0/1728806180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc005500 con 0x7fd2e0102250 2024-04-16T05:41:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1728806180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e01a5390 con 0x7fd2e0102250 2024-04-16T05:41:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.899+0000 7fd2d6ffd640 1 -- 172.21.15.32:0/1728806180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc004e00 con 0x7fd2e0102250 2024-04-16T05:41:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.900+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1728806180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e01a5740 con 0x7fd2e0102250 2024-04-16T05:41:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.902+0000 7fd2d6ffd640 1 -- 172.21.15.32:0/1728806180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd2dc004fa0 con 0x7fd2e0102250 2024-04-16T05:41:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.902+0000 7fd2ea76d740 1 Processor -- start 2024-04-16T05:41:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.902+0000 7fd2ea76d740 1 -- start start 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284160d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841615e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841618750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5628415720e0 con 0x562841616360 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56284150f520 con 0x56284160ce60 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56284156e0a0 con 0x56284160d780 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2d6ffd640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd2cc0a3d20 0x7fd2cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841618750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841618750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57512/0 (socket says 172.21.15.32:57512) 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 -- 172.21.15.32:0/246008052 learned_addr learned my addr 172.21.15.32:0/246008052 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e611f640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd2cc0a3d20 0x7fd2cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e611f640 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284160d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2d6ffd640 1 -- 172.21.15.32:0/1728806180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fd2dc0d0670 con 0x7fd2e0102250 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e591e640 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841615e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 -- 172.21.15.32:0/246008052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 msgr2=0x562841615e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841615e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 -- 172.21.15.32:0/246008052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 msgr2=0x56284160d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284160d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 -- 172.21.15.32:0/246008052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628415378d0 con 0x562841616360 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e611f640 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284160d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.903+0000 7fd2e511d640 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841618750 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000cb10 tx=0x7fd2d000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcc10d6299b1222 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2cbfff640 1 -- 172.21.15.32:0/246008052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0010430 con 0x562841616360 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2e611f640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd2cc0a3d20 0x7fd2cc0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8004af0 tx=0x7fd2d802e040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2cbfff640 1 -- 172.21.15.32:0/246008052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d0010ab0 con 0x562841616360 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2cbfff640 1 -- 172.21.15.32:0/246008052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d001ab40 con 0x562841616360 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 -- 172.21.15.32:0/246008052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 msgr2=0x562841618750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841618750 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000cb10 tx=0x7fd2d000e350 comp rx=0 tx=0).stop 2024-04-16T05:41:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 -- 172.21.15.32:0/246008052 shutdown_connections 2024-04-16T05:41:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841618750 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841615e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/246008052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284160d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 -- 172.21.15.32:0/246008052 >> 172.21.15.32:0/246008052 conn(0x562841607590 msgr2=0x562841607970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 -- 172.21.15.32:0/246008052 shutdown_connections 2024-04-16T05:41:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.904+0000 7fd2ea76d740 1 -- 172.21.15.32:0/246008052 wait complete. 2024-04-16T05:41:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 1 -- start start 2024-04-16T05:41:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 1 Processor -- start 2024-04-16T05:41:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x5628416132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841613820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841613d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562841618fa0 con 0x562841616360 2024-04-16T05:41:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56284152b4d0 con 0x56284160ce60 2024-04-16T05:41:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562841619120 con 0x56284160d780 2024-04-16T05:41:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2ea76d740 10 client.0 _ll_register_callbacks cb 0x5628415102c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2e611f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841613d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2e611f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841613d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57520/0 (socket says 172.21.15.32:57520) 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.905+0000 7fd2e611f640 1 -- 172.21.15.32:0/1226563861 learned_addr learned my addr 172.21.15.32:0/1226563861 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e591e640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x5628416132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e511d640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841613820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e611f640 1 -- 172.21.15.32:0/1226563861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 msgr2=0x562841613820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e611f640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841613820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e611f640 1 -- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 msgr2=0x5628416132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e611f640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x5628416132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e611f640 1 -- 172.21.15.32:0/1226563861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628412843f0 con 0x562841616360 2024-04-16T05:41:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e511d640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841613820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e591e640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x5628416132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2e611f640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841613d60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009ca0 tx=0x7fd2d8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59c622b05742f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2c97fa640 1 -- 172.21.15.32:0/1226563861 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d804e020 con 0x562841616360 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2ea76d740 1 -- 172.21.15.32:0/1226563861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2d0010ab0 con 0x562841616360 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2c97fa640 1 -- 172.21.15.32:0/1226563861 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fd2d8004040 con 0x562841616360 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2c97fa640 1 -- 172.21.15.32:0/1226563861 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2d8059050 con 0x562841616360 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2ea76d740 10 client.17199 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2ea76d740 1 -- 172.21.15.32:0/1226563861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56284134e7d0 con 0x562841616360 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.906+0000 7fd2c97fa640 1 -- 172.21.15.32:0/1226563861 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d0010ab0 con 0x562841616360 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2c97fa640 1 -- 172.21.15.32:0/1226563861 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd2d8049030 con 0x562841616360 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2ea76d740 10 client.17199 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2ea76d740 10 client.17199 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2ea76d740 20 client.17199 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5628416261e0 0x562841628660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2ea76d740 4 client.17199 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2ea76d740 1 -- 172.21.15.32:0/1226563861 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56284157fb00 con 0x5628416261e0 2024-04-16T05:41:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2e591e640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5628416261e0 0x562841628660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2e591e640 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5628416261e0 0x562841628660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1fa53a94b2f535f server_cookie=31d0ef5950e0b2c5 in_seq=0 out_seq=0 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.907+0000 7fd2c97fa640 10 client.17199 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2c97fa640 1 -- 172.21.15.32:0/1226563861 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56284157fb00 con 0x5628416261e0 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2c97fa640 10 client.17199 handle_command_reply: tid=0 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 client.17199 shutdown 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 -- 172.21.15.32:0/1226563861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 msgr2=0x562841613d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841613d60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009ca0 tx=0x7fd2d8005c50 comp rx=0 tx=0).stop 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 -- 172.21.15.32:0/1226563861 shutdown_connections 2024-04-16T05:41:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5628416261e0 0x562841628660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x562841613d60 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x562841613820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2c97fa640 0 client.17199 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:41:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/1226563861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x5628416132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 -- 172.21.15.32:0/1226563861 >> 172.21.15.32:0/1226563861 conn(0x562841607590 msgr2=0x5628416092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 -- 172.21.15.32:0/1226563861 shutdown_connections 2024-04-16T05:41:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.908+0000 7fd2ea76d740 1 -- 172.21.15.32:0/1226563861 wait complete. 2024-04-16T05:41:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.909+0000 7fd2ea76d740 20 client.17199 trim_cache size 0 max 16384 2024-04-16T05:41:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2ea76d740 1 Processor -- start 2024-04-16T05:41:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2ea76d740 1 -- start start 2024-04-16T05:41:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160ce60 0x562841621fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160d780 0x56284162df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284162e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01b1bb0 con 0x562841616360 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01af480 con 0x56284160d780 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01ad2d0 con 0x56284160ce60 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e611f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284162e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e611f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284162e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57530/0 (socket says 172.21.15.32:57530) 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e611f640 1 -- 172.21.15.32:0/106534769 learned_addr learned my addr 172.21.15.32:0/106534769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e591e640 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160ce60 0x562841621fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e511d640 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160d780 0x56284162df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e611f640 1 -- 172.21.15.32:0/106534769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160ce60 msgr2=0x562841621fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e611f640 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160ce60 0x562841621fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e611f640 1 -- 172.21.15.32:0/106534769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160d780 msgr2=0x56284162df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e611f640 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160d780 0x56284162df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e611f640 1 -- 172.21.15.32:0/106534769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56284160e100 con 0x562841616360 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e511d640 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160d780 0x56284162df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.922+0000 7fd2e591e640 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160ce60 0x562841621fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2e611f640 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284162e470 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8005e30 tx=0x7fd2d8005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bc2f8968badd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2cb7fe640 1 -- 172.21.15.32:0/106534769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8062070 con 0x562841616360 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2cb7fe640 1 -- 172.21.15.32:0/106534769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d8040480 con 0x562841616360 2024-04-16T05:41:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2cb7fe640 1 -- 172.21.15.32:0/106534769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d804e040 con 0x562841616360 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 -- 172.21.15.32:0/106534769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 msgr2=0x56284162e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284162e470 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8005e30 tx=0x7fd2d8005f50 comp rx=0 tx=0).stop 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 -- 172.21.15.32:0/106534769 shutdown_connections 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284162e470 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160d780 0x56284162df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/106534769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160ce60 0x562841621fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 -- 172.21.15.32:0/106534769 >> 172.21.15.32:0/106534769 conn(0x562841607590 msgr2=0x5628416025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 -- 172.21.15.32:0/106534769 shutdown_connections 2024-04-16T05:41:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.923+0000 7fd2ea76d740 1 -- 172.21.15.32:0/106534769 wait complete. 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 1 -- start start 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 1 Processor -- start 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284161e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x56284161ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284160f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01ad940 con 0x562841616360 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01aedb0 con 0x56284160ce60 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01ae010 con 0x56284160d780 2024-04-16T05:41:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2ea76d740 10 client.0 _ll_register_callbacks cb 0x7fd2d8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2e611f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284160f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2e611f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284160f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57532/0 (socket says 172.21.15.32:57532) 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2e611f640 1 -- 172.21.15.32:0/3649891387 learned_addr learned my addr 172.21.15.32:0/3649891387 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.924+0000 7fd2e591e640 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284161e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2e511d640 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x56284161ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2e611f640 1 -- 172.21.15.32:0/3649891387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 msgr2=0x56284161ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2e611f640 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x56284161ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2e611f640 1 -- 172.21.15.32:0/3649891387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 msgr2=0x56284161e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2e611f640 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284161e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2e611f640 1 -- 172.21.15.32:0/3649891387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628416100e0 con 0x562841616360 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2e511d640 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x56284161ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2e611f640 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284160f900 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8042010 tx=0x7fd2d8003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9301f13bfd03d1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2c97fa640 1 -- 172.21.15.32:0/3649891387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8062070 con 0x562841616360 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2c97fa640 1 -- 172.21.15.32:0/3649891387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fd2d8004240 con 0x562841616360 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2ea76d740 1 -- 172.21.15.32:0/3649891387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2d8040480 con 0x562841616360 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2c97fa640 1 -- 172.21.15.32:0/3649891387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2d804e040 con 0x562841616360 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2ea76d740 10 client.17205 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:30.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2ea76d740 1 -- 172.21.15.32:0/3649891387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56284168c1f0 con 0x562841616360 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2c97fa640 1 -- 172.21.15.32:0/3649891387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d8040480 con 0x562841616360 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2c97fa640 1 -- 172.21.15.32:0/3649891387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd2d8049070 con 0x562841616360 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2ea76d740 10 client.17205 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2ea76d740 10 client.17205 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.925+0000 7fd2ea76d740 20 client.17205 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56284168c7a0 0x56284168eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2ea76d740 4 client.17205 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2ea76d740 1 -- 172.21.15.32:0/3649891387 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562841579970 con 0x56284168c7a0 2024-04-16T05:41:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2e591e640 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56284168c7a0 0x56284168eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2e591e640 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56284168c7a0 0x56284168eb90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0e4fa3f4e41977f server_cookie=a4161356ba4c623 in_seq=0 out_seq=0 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2c97fa640 10 client.17205 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2c97fa640 1 -- 172.21.15.32:0/3649891387 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562841579970 con 0x56284168c7a0 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2c97fa640 10 client.17205 handle_command_reply: tid=0 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2ea76d740 1 client.17205 shutdown 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2ea76d740 1 -- 172.21.15.32:0/3649891387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 msgr2=0x56284160f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.926+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284160f900 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8042010 tx=0x7fd2d8003dc0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 1 -- 172.21.15.32:0/3649891387 shutdown_connections 2024-04-16T05:41:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56284168c7a0 0x56284168eb90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562841616360 0x56284160f900 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56284160d780 0x56284161ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 1 --2- 172.21.15.32:0/3649891387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56284160ce60 0x56284161e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2c97fa640 0 client.17205 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:41:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 1 -- 172.21.15.32:0/3649891387 >> 172.21.15.32:0/3649891387 conn(0x562841607590 msgr2=0x562841605ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 1 -- 172.21.15.32:0/3649891387 shutdown_connections 2024-04-16T05:41:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 1 -- 172.21.15.32:0/3649891387 wait complete. 2024-04-16T05:41:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2ea76d740 20 client.17205 trim_cache size 0 max 16384 2024-04-16T05:41:30.936 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:41:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd2cc0a3d20 msgr2=0x7fd2cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd2cc0a3d20 0x7fd2cc0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8004af0 tx=0x7fd2d802e040 comp rx=0 tx=0).stop 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0102250 msgr2=0x7fd2e019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0102250 0x7fd2e019e5f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00d770 tx=0x7fd2dc00dc30 comp rx=0 tx=0).stop 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1728806180 shutdown_connections 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2e0198700 0x7fd2e0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2e0107320 0x7fd2e019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd2cc0a3d20 0x7fd2cc0a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 --2- 172.21.15.32:0/1728806180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2e0102250 0x7fd2e019e5f0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1728806180 >> 172.21.15.32:0/1728806180 conn(0x7fd2e0079490 msgr2=0x7fd2e00ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1728806180 shutdown_connections 2024-04-16T05:41:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:30.927+0000 7fd2e7b82640 1 -- 172.21.15.32:0/1728806180 wait complete. 2024-04-16T05:41:30.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:30 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2312430535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:30.939 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:41:31.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.047+0000 7f25941a8640 1 -- 172.21.15.32:0/3992358145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 msgr2=0x7f258c10eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.047+0000 7f25941a8640 1 --2- 172.21.15.32:0/3992358145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 0x7f258c10eec0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2584009ca0 tx=0x7f258402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 -- 172.21.15.32:0/3992358145 shutdown_connections 2024-04-16T05:41:31.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 --2- 172.21.15.32:0/3992358145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 0x7f258c10eec0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 --2- 172.21.15.32:0/3992358145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258c102bd0 0x7f258c106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 --2- 172.21.15.32:0/3992358145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f258c102220 0x7f258c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 -- 172.21.15.32:0/3992358145 >> 172.21.15.32:0/3992358145 conn(0x7f258c079450 msgr2=0x7f258c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 -- 172.21.15.32:0/3992358145 shutdown_connections 2024-04-16T05:41:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 -- 172.21.15.32:0/3992358145 wait complete. 2024-04-16T05:41:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 Processor -- start 2024-04-16T05:41:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 -- start start 2024-04-16T05:41:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f258c102220 0x7f258c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 0x7f258c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258c1986f0 0x7f258c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1123b0 con 0x7f258c1072f0 2024-04-16T05:41:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f258c112230 con 0x7f258c1986f0 2024-04-16T05:41:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.048+0000 7f25941a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f258c112530 con 0x7f258c102220 2024-04-16T05:41:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 0x7f258c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 0x7f258c19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57552/0 (socket says 172.21.15.32:57552) 2024-04-16T05:41:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 -- 172.21.15.32:0/866351425 learned_addr learned my addr 172.21.15.32:0/866351425 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2592745640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258c1986f0 0x7f258c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591f44640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f258c102220 0x7f258c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 -- 172.21.15.32:0/866351425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f258c102220 msgr2=0x7f258c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f258c102220 0x7f258c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 -- 172.21.15.32:0/866351425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258c1986f0 msgr2=0x7f258c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258c1986f0 0x7f258c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 -- 172.21.15.32:0/866351425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c1992f0 con 0x7f258c1072f0 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591f44640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f258c102220 0x7f258c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2592745640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258c1986f0 0x7f258c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:31.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2591743640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 0x7f258c19eb20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f257c00d770 tx=0x7f257c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:31.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2582ffd640 1 -- 172.21.15.32:0/866351425 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c017070 con 0x7f258c1072f0 2024-04-16T05:41:31.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2582ffd640 1 -- 172.21.15.32:0/866351425 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f257c009c30 con 0x7f258c1072f0 2024-04-16T05:41:31.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f25941a8640 1 -- 172.21.15.32:0/866351425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c1a5380 con 0x7f258c1072f0 2024-04-16T05:41:31.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f2582ffd640 1 -- 172.21.15.32:0/866351425 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c00f620 con 0x7f258c1072f0 2024-04-16T05:41:31.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.049+0000 7f25941a8640 1 -- 172.21.15.32:0/866351425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c1a5780 con 0x7f258c1072f0 2024-04-16T05:41:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.051+0000 7f2582ffd640 1 -- 172.21.15.32:0/866351425 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f257c01a070 con 0x7f258c1072f0 2024-04-16T05:41:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2596d98740 1 Processor -- start 2024-04-16T05:41:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2596d98740 1 -- start start 2024-04-16T05:41:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2596d98740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2596d98740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2596d98740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x5650405f7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2596d98740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5650405510e0 con 0x5650405ec780 2024-04-16T05:41:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2596d98740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5650404ee520 con 0x5650405f5360 2024-04-16T05:41:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2596d98740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56504054d0a0 con 0x5650405ebe60 2024-04-16T05:41:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2591f44640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2591f44640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57562/0 (socket says 172.21.15.32:57562) 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2591f44640 1 -- 172.21.15.32:0/2825688427 learned_addr learned my addr 172.21.15.32:0/2825688427 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2591743640 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x5650405f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.052+0000 7f2592745640 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2591f44640 1 -- 172.21.15.32:0/2825688427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 msgr2=0x5650405ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2582ffd640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f25780a3d20 0x7f25780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2591f44640 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405ec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2591f44640 1 -- 172.21.15.32:0/2825688427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 msgr2=0x5650405f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2591f44640 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x5650405f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2591f44640 1 -- 172.21.15.32:0/2825688427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650405168d0 con 0x5650405ec780 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2591743640 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x5650405f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2592745640 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2592745640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f25780a3d20 0x7f25780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2582ffd640 1 -- 172.21.15.32:0/866351425 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f257c0d06a0 con 0x7f258c1072f0 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2591f44640 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f4e20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2588009c30 tx=0x7f2588010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64ec49ee164a918 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2577fff640 1 -- 172.21.15.32:0/2825688427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588004020 con 0x5650405ec780 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2577fff640 1 -- 172.21.15.32:0/2825688427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25880041c0 con 0x5650405ec780 2024-04-16T05:41:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.053+0000 7f2577fff640 1 -- 172.21.15.32:0/2825688427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588012690 con 0x5650405ec780 2024-04-16T05:41:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 -- 172.21.15.32:0/2825688427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 msgr2=0x5650405f4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2592745640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f25780a3d20 0x7f25780a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2584006010 tx=0x7f258402e040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f4e20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2588009c30 tx=0x7f2588010740 comp rx=0 tx=0).stop 2024-04-16T05:41:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 -- 172.21.15.32:0/2825688427 shutdown_connections 2024-04-16T05:41:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x5650405f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f4e20 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 --2- 172.21.15.32:0/2825688427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 -- 172.21.15.32:0/2825688427 >> 172.21.15.32:0/2825688427 conn(0x5650405e6590 msgr2=0x5650405e6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 -- 172.21.15.32:0/2825688427 shutdown_connections 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 -- 172.21.15.32:0/2825688427 wait complete. 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 -- start start 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.054+0000 7f2596d98740 1 Processor -- start 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2596d98740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ebe60 0x5650405f2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2596d98740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2596d98740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405f5360 0x5650405f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2596d98740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5650405f80c0 con 0x5650405ec780 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2596d98740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56504050a4d0 con 0x5650405ebe60 2024-04-16T05:41:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2596d98740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5650405f8240 con 0x5650405f5360 2024-04-16T05:41:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2596d98740 10 client.0 _ll_register_callbacks cb 0x5650404ef2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57568/0 (socket says 172.21.15.32:57568) 2024-04-16T05:41:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 -- 172.21.15.32:0/1707036876 learned_addr learned my addr 172.21.15.32:0/1707036876 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591f44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ebe60 0x5650405f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2592745640 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405f5360 0x5650405f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 -- 172.21.15.32:0/1707036876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405f5360 msgr2=0x5650405f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:31.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405f5360 0x5650405f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 -- 172.21.15.32:0/1707036876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ebe60 msgr2=0x5650405f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ebe60 0x5650405f2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 -- 172.21.15.32:0/1707036876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650402633f0 con 0x5650405ec780 2024-04-16T05:41:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f2591743640 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f2840 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f257c09c000 tx=0x7f257c09bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d3d2a8c208b95c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f25757fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f25757fa640 1 -- 172.21.15.32:0/1707036876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c021040 con 0x5650405ec780 2024-04-16T05:41:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.055+0000 7f25757fa640 1 -- 172.21.15.32:0/1707036876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f257c0cdd30 con 0x5650405ec780 2024-04-16T05:41:31.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 1 -- 172.21.15.32:0/1707036876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25880041c0 con 0x5650405ec780 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f25757fa640 1 -- 172.21.15.32:0/1707036876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f257c0d0500 con 0x5650405ec780 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 10 client.17217 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 1 -- 172.21.15.32:0/1707036876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56504032d7d0 con 0x5650405ec780 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f25757fa640 1 -- 172.21.15.32:0/1707036876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25880041c0 con 0x5650405ec780 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f25757fa640 1 -- 172.21.15.32:0/1707036876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f257c09b8e0 con 0x5650405ec780 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 10 client.17217 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 20 client.17217 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x565040605360 0x565040607750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 4 client.17217 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2596d98740 1 -- 172.21.15.32:0/1707036876 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56504055eb00 con 0x565040605360 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.056+0000 7f2591f44640 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x565040605360 0x565040607750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.057+0000 7f2591f44640 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x565040605360 0x565040607750 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73522e4c63a6060 server_cookie=7c6dd0bd29911728 in_seq=0 out_seq=0 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.057+0000 7f25757fa640 10 client.17217 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f25757fa640 1 -- 172.21.15.32:0/1707036876 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56504055eb00 con 0x565040605360 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f25757fa640 10 client.17217 handle_command_reply: tid=0 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 client.17217 shutdown 2024-04-16T05:41:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 -- 172.21.15.32:0/1707036876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 msgr2=0x5650405f2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f2840 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f257c09c000 tx=0x7f257c09bd20 comp rx=0 tx=0).stop 2024-04-16T05:41:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 -- 172.21.15.32:0/1707036876 shutdown_connections 2024-04-16T05:41:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x565040605360 0x565040607750 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405f5360 0x5650405f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ec780 0x5650405f2840 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f25757fa640 0 client.17217 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 --2- 172.21.15.32:0/1707036876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ebe60 0x5650405f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 -- 172.21.15.32:0/1707036876 >> 172.21.15.32:0/1707036876 conn(0x5650405e6590 msgr2=0x5650405e82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:31.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 -- 172.21.15.32:0/1707036876 shutdown_connections 2024-04-16T05:41:31.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 1 -- 172.21.15.32:0/1707036876 wait complete. 2024-04-16T05:41:31.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.058+0000 7f2596d98740 20 client.17217 trim_cache size 0 max 16384 2024-04-16T05:41:31.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2596d98740 1 Processor -- start 2024-04-16T05:41:31.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2596d98740 1 -- start start 2024-04-16T05:41:31.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2596d98740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ebe60 0x5650406010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2596d98740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ec780 0x5650405f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2596d98740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x56504060d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2596d98740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1b1c60 con 0x5650405ebe60 2024-04-16T05:41:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2596d98740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1af530 con 0x5650405f5360 2024-04-16T05:41:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2596d98740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1ad380 con 0x5650405ec780 2024-04-16T05:41:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.071+0000 7f2591f44640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ebe60 0x5650406010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2591f44640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ebe60 0x5650406010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57578/0 (socket says 172.21.15.32:57578) 2024-04-16T05:41:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2591f44640 1 -- 172.21.15.32:0/2909202863 learned_addr learned my addr 172.21.15.32:0/2909202863 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2592745640 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x56504060d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2591f44640 1 -- 172.21.15.32:0/2909202863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ec780 msgr2=0x5650405f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2591f44640 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ec780 0x5650405f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2591f44640 1 -- 172.21.15.32:0/2909202863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 msgr2=0x56504060d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2591f44640 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x56504060d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2591f44640 1 -- 172.21.15.32:0/2909202863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650405e63c0 con 0x5650405ebe60 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2592745640 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x56504060d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f2591f44640 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ebe60 0x5650406010c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x5650405eb930 tx=0x7f2588005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cfcf771d3c8361 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f25777fe640 1 -- 172.21.15.32:0/2909202863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258802e070 con 0x5650405ebe60 2024-04-16T05:41:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f25777fe640 1 -- 172.21.15.32:0/2909202863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25880043f0 con 0x5650405ebe60 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.072+0000 7f25777fe640 1 -- 172.21.15.32:0/2909202863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588005040 con 0x5650405ebe60 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 -- 172.21.15.32:0/2909202863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ebe60 msgr2=0x5650406010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ebe60 0x5650406010c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x5650405eb930 tx=0x7f2588005850 comp rx=0 tx=0).stop 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 -- 172.21.15.32:0/2909202863 shutdown_connections 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405f5360 0x56504060d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ec780 0x5650405f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 --2- 172.21.15.32:0/2909202863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405ebe60 0x5650406010c0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 -- 172.21.15.32:0/2909202863 >> 172.21.15.32:0/2909202863 conn(0x5650405e6590 msgr2=0x5650405f22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 -- 172.21.15.32:0/2909202863 shutdown_connections 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 -- 172.21.15.32:0/2909202863 wait complete. 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 -- start start 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.073+0000 7f2596d98740 1 Processor -- start 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2596d98740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2596d98740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ec780 0x5650405f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2596d98740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405f5360 0x5650405f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2596d98740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1ad9f0 con 0x5650405f5360 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2596d98740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1aee60 con 0x5650405ec780 2024-04-16T05:41:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2596d98740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1ae0c0 con 0x5650405ebe60 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2596d98740 10 client.0 _ll_register_callbacks cb 0x7f257c09c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405f5360 0x5650405f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405f5360 0x5650405f9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57584/0 (socket says 172.21.15.32:57584) 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 -- 172.21.15.32:0/250434103 learned_addr learned my addr 172.21.15.32:0/250434103 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2591743640 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ec780 0x5650405f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2591f44640 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 -- 172.21.15.32:0/250434103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 msgr2=0x5650405f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405f9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 -- 172.21.15.32:0/250434103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ec780 msgr2=0x5650405f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ec780 0x5650405f96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 -- 172.21.15.32:0/250434103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504066aa60 con 0x5650405f5360 2024-04-16T05:41:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2591f44640 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2592745640 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405f5360 0x5650405f9c10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f258402c700 tx=0x7f2584004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de762a86c611b4f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f25757fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f25757fa640 1 -- 172.21.15.32:0/250434103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584039040 con 0x5650405f5360 2024-04-16T05:41:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f25757fa640 1 -- 172.21.15.32:0/250434103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f2584003b70 con 0x5650405f5360 2024-04-16T05:41:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.074+0000 7f2596d98740 1 -- 172.21.15.32:0/250434103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25880043f0 con 0x5650405f5360 2024-04-16T05:41:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f25757fa640 1 -- 172.21.15.32:0/250434103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2584032e90 con 0x5650405f5360 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2596d98740 10 client.17223 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2596d98740 1 -- 172.21.15.32:0/250434103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56504066bc90 con 0x5650405f5360 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f25757fa640 1 -- 172.21.15.32:0/250434103 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25880043f0 con 0x5650405f5360 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f25757fa640 1 -- 172.21.15.32:0/250434103 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2584034740 con 0x5650405f5360 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2596d98740 10 client.17223 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2596d98740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2596d98740 20 client.17223 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2596d98740 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56504066c460 0x56504066e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2596d98740 4 client.17223 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2596d98740 1 -- 172.21.15.32:0/250434103 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565040558970 con 0x56504066c460 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2591f44640 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56504066c460 0x56504066e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f2591f44640 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56504066c460 0x56504066e880 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0dfdd102ed7d93e server_cookie=589bdc54dece910f in_seq=0 out_seq=0 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.075+0000 7f25757fa640 10 client.17223 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f25757fa640 1 -- 172.21.15.32:0/250434103 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565040558970 con 0x56504066c460 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f25757fa640 10 client.17223 handle_command_reply: tid=0 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 client.17223 shutdown 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 -- 172.21.15.32:0/250434103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405f5360 msgr2=0x5650405f9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405f5360 0x5650405f9c10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f258402c700 tx=0x7f2584004a40 comp rx=0 tx=0).stop 2024-04-16T05:41:31.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 -- 172.21.15.32:0/250434103 shutdown_connections 2024-04-16T05:41:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5650405f5360 0x5650405f9c10 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5650405ec780 0x5650405f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56504066c460 0x56504066e880 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 --2- 172.21.15.32:0/250434103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650405ebe60 0x5650405f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 -- 172.21.15.32:0/250434103 >> 172.21.15.32:0/250434103 conn(0x5650405e6590 msgr2=0x5650405f22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f25757fa640 0 client.17223 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 -- 172.21.15.32:0/250434103 shutdown_connections 2024-04-16T05:41:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 1 -- 172.21.15.32:0/250434103 wait complete. 2024-04-16T05:41:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f2596d98740 20 client.17223 trim_cache size 0 max 16384 2024-04-16T05:41:31.085 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:41:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f25941a8640 1 -- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f25780a3d20 msgr2=0x7f25780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f25941a8640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f25780a3d20 0x7f25780a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2584006010 tx=0x7f258402e040 comp rx=0 tx=0).stop 2024-04-16T05:41:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f25941a8640 1 -- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 msgr2=0x7f258c19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.076+0000 7f25941a8640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 0x7f258c19eb20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f257c00d770 tx=0x7f257c00dc30 comp rx=0 tx=0).stop 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.077+0000 7f25941a8640 1 -- 172.21.15.32:0/866351425 shutdown_connections 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.077+0000 7f25941a8640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258c1986f0 0x7f258c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.077+0000 7f25941a8640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f25780a3d20 0x7f25780a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.077+0000 7f25941a8640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c1072f0 0x7f258c19eb20 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.077+0000 7f25941a8640 1 --2- 172.21.15.32:0/866351425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f258c102220 0x7f258c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.077+0000 7f25941a8640 1 -- 172.21.15.32:0/866351425 >> 172.21.15.32:0/866351425 conn(0x7f258c079450 msgr2=0x7f258c0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.077+0000 7f25941a8640 1 -- 172.21.15.32:0/866351425 shutdown_connections 2024-04-16T05:41:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:31.077+0000 7f25941a8640 1 -- 172.21.15.32:0/866351425 wait complete. 2024-04-16T05:41:31.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:30 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2312430535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:31.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:30 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2312430535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:41:32.089 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:41:32.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:41:32.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:41:32.119 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:31 smithi032 ceph-mon[32351]: pgmap v200: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:32.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:31 smithi134 ceph-mon[36153]: pgmap v200: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:32.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:31 smithi082 ceph-mon[36195]: pgmap v200: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 -- 172.21.15.32:0/253964901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 msgr2=0x7fde7c101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 --2- 172.21.15.32:0/253964901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 0x7fde7c101520 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009900 tx=0x7fde6c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:41:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 -- 172.21.15.32:0/253964901 shutdown_connections 2024-04-16T05:41:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 --2- 172.21.15.32:0/253964901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c106210 0x7fde7c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 --2- 172.21.15.32:0/253964901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7c101af0 0x7fde7c105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 --2- 172.21.15.32:0/253964901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 0x7fde7c101520 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 -- 172.21.15.32:0/253964901 >> 172.21.15.32:0/253964901 conn(0x7fde7c078000 msgr2=0x7fde7c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 -- 172.21.15.32:0/253964901 shutdown_connections 2024-04-16T05:41:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.214+0000 7fde824d9640 1 -- 172.21.15.32:0/253964901 wait complete. 2024-04-16T05:41:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde824d9640 1 Processor -- start 2024-04-16T05:41:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde824d9640 1 -- start start 2024-04-16T05:41:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde824d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 0x7fde7c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde824d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7c106210 0x7fde7c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde824d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c1986c0 0x7fde7c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde824d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1123e0 con 0x7fde7c101140 2024-04-16T05:41:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde824d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c112260 con 0x7fde7c106210 2024-04-16T05:41:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde824d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c112560 con 0x7fde7c1986c0 2024-04-16T05:41:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 0x7fde7c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 0x7fde7c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57612/0 (socket says 172.21.15.32:57612) 2024-04-16T05:41:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde7bfff640 1 -- 172.21.15.32:0/98541194 learned_addr learned my addr 172.21.15.32:0/98541194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde80a76640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c1986c0 0x7fde7c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde73fff640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7c106210 0x7fde7c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde7bfff640 1 -- 172.21.15.32:0/98541194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c1986c0 msgr2=0x7fde7c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde7bfff640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c1986c0 0x7fde7c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde7bfff640 1 -- 172.21.15.32:0/98541194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7c106210 msgr2=0x7fde7c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.215+0000 7fde7bfff640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7c106210 0x7fde7c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.216+0000 7fde7bfff640 1 -- 172.21.15.32:0/98541194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c1992c0 con 0x7fde7c101140 2024-04-16T05:41:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.216+0000 7fde7bfff640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 0x7fde7c19e600 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fde6c006010 tx=0x7fde6c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.216+0000 7fde79ffb640 1 -- 172.21.15.32:0/98541194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c041070 con 0x7fde7c101140 2024-04-16T05:41:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.216+0000 7fde79ffb640 1 -- 172.21.15.32:0/98541194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c004450 con 0x7fde7c101140 2024-04-16T05:41:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.216+0000 7fde824d9640 1 -- 172.21.15.32:0/98541194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c199550 con 0x7fde7c101140 2024-04-16T05:41:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.216+0000 7fde79ffb640 1 -- 172.21.15.32:0/98541194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c034440 con 0x7fde7c101140 2024-04-16T05:41:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.216+0000 7fde824d9640 1 -- 172.21.15.32:0/98541194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1a5690 con 0x7fde7c101140 2024-04-16T05:41:32.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.218+0000 7fde79ffb640 1 -- 172.21.15.32:0/98541194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde6c044070 con 0x7fde7c101140 2024-04-16T05:41:32.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde851a0740 1 Processor -- start 2024-04-16T05:41:32.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde851a0740 1 -- start start 2024-04-16T05:41:32.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde79ffb640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fde680a3d20 0x7fde680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde851a0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a55e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde851a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a563d0 0x564f87a5ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde851a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a5efb0 0x564f87a613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde851a0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564f879bae90 con 0x564f87a55ab0 2024-04-16T05:41:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde851a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564f879584c0 con 0x564f87a563d0 2024-04-16T05:41:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde851a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f879b6e50 con 0x564f87a5efb0 2024-04-16T05:41:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a55e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a55e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57618/0 (socket says 172.21.15.32:57618) 2024-04-16T05:41:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde7bfff640 1 -- 172.21.15.32:0/3612325507 learned_addr learned my addr 172.21.15.32:0/3612325507 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde80a76640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fde680a3d20 0x7fde680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde79ffb640 1 -- 172.21.15.32:0/98541194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fde6c0f26a0 con 0x7fde7c101140 2024-04-16T05:41:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde73fff640 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a563d0 0x564f87a5ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde80a76640 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a5efb0 0x564f87a613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde7bfff640 1 -- 172.21.15.32:0/3612325507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a5efb0 msgr2=0x564f87a613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde7bfff640 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a5efb0 0x564f87a613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde7bfff640 1 -- 172.21.15.32:0/3612325507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a563d0 msgr2=0x564f87a5ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde7bfff640 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a563d0 0x564f87a5ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde7bfff640 1 -- 172.21.15.32:0/3612325507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f877957e0 con 0x564f87a55ab0 2024-04-16T05:41:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.219+0000 7fde80a76640 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a5efb0 0x564f87a613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde7bfff640 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a55e90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0efbc0 tx=0x7fde6c003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39561f4aa4e92acb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde80a76640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fde680a3d20 0x7fde680a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x564f87a51370 tx=0x7fde74009a60 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde727fc640 1 -- 172.21.15.32:0/3612325507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c04b070 con 0x564f87a55ab0 2024-04-16T05:41:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde727fc640 1 -- 172.21.15.32:0/3612325507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c0beab0 con 0x564f87a55ab0 2024-04-16T05:41:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde727fc640 1 -- 172.21.15.32:0/3612325507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0c3690 con 0x564f87a55ab0 2024-04-16T05:41:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde851a0740 1 -- 172.21.15.32:0/3612325507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 msgr2=0x564f87a55e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde851a0740 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a55e90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0efbc0 tx=0x7fde6c003990 comp rx=0 tx=0).stop 2024-04-16T05:41:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde851a0740 1 -- 172.21.15.32:0/3612325507 shutdown_connections 2024-04-16T05:41:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde851a0740 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a5efb0 0x564f87a613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde851a0740 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a563d0 0x564f87a5ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde851a0740 1 --2- 172.21.15.32:0/3612325507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a55e90 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde851a0740 1 -- 172.21.15.32:0/3612325507 >> 172.21.15.32:0/3612325507 conn(0x564f87a501e0 msgr2=0x564f87a505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.220+0000 7fde851a0740 1 -- 172.21.15.32:0/3612325507 shutdown_connections 2024-04-16T05:41:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 -- 172.21.15.32:0/3612325507 wait complete. 2024-04-16T05:41:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 -- start start 2024-04-16T05:41:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 Processor -- start 2024-04-16T05:41:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a55ab0 0x564f87a5bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a5c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a5efb0 0x564f87a5c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564f879bae90 con 0x564f87a5efb0 2024-04-16T05:41:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564f87943410 con 0x564f87a55ab0 2024-04-16T05:41:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f87a61b60 con 0x564f87a563d0 2024-04-16T05:41:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde851a0740 10 client.0 _ll_register_callbacks cb 0x564f8793eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde80a76640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a5efb0 0x564f87a5c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde80a76640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a5efb0 0x564f87a5c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57628/0 (socket says 172.21.15.32:57628) 2024-04-16T05:41:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.221+0000 7fde80a76640 1 -- 172.21.15.32:0/3318354400 learned_addr learned my addr 172.21.15.32:0/3318354400 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde7bfff640 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a55ab0 0x564f87a5bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde73fff640 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a5c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde80a76640 1 -- 172.21.15.32:0/3318354400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 msgr2=0x564f87a5c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde80a76640 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a5c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde80a76640 1 -- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a55ab0 msgr2=0x564f87a5bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde80a76640 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a55ab0 0x564f87a5bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde80a76640 1 -- 172.21.15.32:0/3318354400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f877f5f70 con 0x564f87a5efb0 2024-04-16T05:41:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde7bfff640 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a55ab0 0x564f87a5bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde80a76640 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a5efb0 0x564f87a5c9e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fde74013cb0 tx=0x7fde74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1f59f0cc19d55b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde5b7fe640 1 -- 172.21.15.32:0/3318354400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74027060 con 0x564f87a5efb0 2024-04-16T05:41:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde5b7fe640 1 -- 172.21.15.32:0/3318354400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fde74022670 con 0x564f87a5efb0 2024-04-16T05:41:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde851a0740 1 -- 172.21.15.32:0/3318354400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c0beab0 con 0x564f87a5efb0 2024-04-16T05:41:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde5b7fe640 1 -- 172.21.15.32:0/3318354400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde7402e050 con 0x564f87a5efb0 2024-04-16T05:41:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde851a0740 10 client.17235 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.222+0000 7fde851a0740 1 -- 172.21.15.32:0/3318354400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f8768d170 con 0x564f87a5efb0 2024-04-16T05:41:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde5b7fe640 1 -- 172.21.15.32:0/3318354400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c0beab0 con 0x564f87a5efb0 2024-04-16T05:41:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde5b7fe640 1 -- 172.21.15.32:0/3318354400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fde7400f040 con 0x564f87a5efb0 2024-04-16T05:41:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde851a0740 10 client.17235 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde851a0740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde851a0740 20 client.17235 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde851a0740 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564f87a6ef10 0x564f87a71390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde851a0740 4 client.17235 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde851a0740 1 -- 172.21.15.32:0/3318354400 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f879c88b0 con 0x564f87a6ef10 2024-04-16T05:41:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.223+0000 7fde7bfff640 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564f87a6ef10 0x564f87a71390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.224+0000 7fde7bfff640 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564f87a6ef10 0x564f87a71390 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ae756cc287ce05 server_cookie=f94d45759ae547f8 in_seq=0 out_seq=0 2024-04-16T05:41:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.224+0000 7fde5b7fe640 10 client.17235 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.224+0000 7fde5b7fe640 1 -- 172.21.15.32:0/3318354400 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f879c88b0 con 0x564f87a6ef10 2024-04-16T05:41:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.224+0000 7fde5b7fe640 10 client.17235 handle_command_reply: tid=0 2024-04-16T05:41:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.224+0000 7fde851a0740 1 client.17235 shutdown 2024-04-16T05:41:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.224+0000 7fde851a0740 1 -- 172.21.15.32:0/3318354400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a5efb0 msgr2=0x564f87a5c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.224+0000 7fde851a0740 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a5efb0 0x564f87a5c9e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fde74013cb0 tx=0x7fde74004a40 comp rx=0 tx=0).stop 2024-04-16T05:41:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 1 -- 172.21.15.32:0/3318354400 shutdown_connections 2024-04-16T05:41:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a5efb0 0x564f87a5c9e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564f87a6ef10 0x564f87a71390 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a5c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 1 --2- 172.21.15.32:0/3318354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a55ab0 0x564f87a5bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 1 -- 172.21.15.32:0/3318354400 >> 172.21.15.32:0/3318354400 conn(0x564f87a501e0 msgr2=0x564f87a51f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde5b7fe640 0 client.17235 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 1 -- 172.21.15.32:0/3318354400 shutdown_connections 2024-04-16T05:41:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 1 -- 172.21.15.32:0/3318354400 wait complete. 2024-04-16T05:41:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.225+0000 7fde851a0740 20 client.17235 trim_cache size 0 max 16384 2024-04-16T05:41:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde851a0740 1 Processor -- start 2024-04-16T05:41:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde851a0740 1 -- start start 2024-04-16T05:41:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde851a0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a5e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde851a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a6a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde851a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87a5c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde851a0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1b1c30 con 0x564f87a55ab0 2024-04-16T05:41:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde851a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1af500 con 0x564f87a5efb0 2024-04-16T05:41:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde851a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1ad350 con 0x564f87a563d0 2024-04-16T05:41:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a5e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a5e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57640/0 (socket says 172.21.15.32:57640) 2024-04-16T05:41:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 -- 172.21.15.32:0/28460486 learned_addr learned my addr 172.21.15.32:0/28460486 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde80a76640 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87a5c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde73fff640 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a6a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 -- 172.21.15.32:0/28460486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 msgr2=0x564f87a6a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a6a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 -- 172.21.15.32:0/28460486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 msgr2=0x564f87a5c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87a5c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 -- 172.21.15.32:0/28460486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f87a57ba0 con 0x564f87a55ab0 2024-04-16T05:41:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde80a76640 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87a5c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.243+0000 7fde7bfff640 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a5e690 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0be340 tx=0x7fde6c0be280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03ccc3cae1954aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde71ffb640 1 -- 172.21.15.32:0/28460486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0d6070 con 0x564f87a55ab0 2024-04-16T05:41:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde71ffb640 1 -- 172.21.15.32:0/28460486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c0d2e50 con 0x564f87a55ab0 2024-04-16T05:41:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde71ffb640 1 -- 172.21.15.32:0/28460486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c04b040 con 0x564f87a55ab0 2024-04-16T05:41:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 -- 172.21.15.32:0/28460486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 msgr2=0x564f87a5e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a5e690 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0be340 tx=0x7fde6c0be280 comp rx=0 tx=0).stop 2024-04-16T05:41:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 -- 172.21.15.32:0/28460486 shutdown_connections 2024-04-16T05:41:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87a5c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a6a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 --2- 172.21.15.32:0/28460486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a5e690 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 -- 172.21.15.32:0/28460486 >> 172.21.15.32:0/28460486 conn(0x564f87a501e0 msgr2=0x564f87a5bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 -- 172.21.15.32:0/28460486 shutdown_connections 2024-04-16T05:41:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.244+0000 7fde851a0740 1 -- 172.21.15.32:0/28460486 wait complete. 2024-04-16T05:41:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 1 -- start start 2024-04-16T05:41:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 1 Processor -- start 2024-04-16T05:41:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87ad44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1ad9c0 con 0x564f87a55ab0 2024-04-16T05:41:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1aee30 con 0x564f87a5efb0 2024-04-16T05:41:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1ae090 con 0x564f87a563d0 2024-04-16T05:41:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde851a0740 10 client.0 _ll_register_callbacks cb 0x7fde74013930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57644/0 (socket says 172.21.15.32:57644) 2024-04-16T05:41:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde7bfff640 1 -- 172.21.15.32:0/392230572 learned_addr learned my addr 172.21.15.32:0/392230572 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde7bfff640 1 -- 172.21.15.32:0/392230572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 msgr2=0x564f87a595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde73fff640 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde7bfff640 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde7bfff640 1 -- 172.21.15.32:0/392230572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 msgr2=0x564f87ad44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.245+0000 7fde80a76640 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87ad44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde7bfff640 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87ad44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde7bfff640 1 -- 172.21.15.32:0/392230572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f87ad4a10 con 0x564f87a55ab0 2024-04-16T05:41:32.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde7bfff640 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a590b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0ba810 tx=0x7fde6c0c2760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859f4ff0a5f781a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:32.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde5b7fe640 1 -- 172.21.15.32:0/392230572 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0d6070 con 0x564f87a55ab0 2024-04-16T05:41:32.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde5b7fe640 1 -- 172.21.15.32:0/392230572 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fde6c0dd2b0 con 0x564f87a55ab0 2024-04-16T05:41:32.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 1 -- 172.21.15.32:0/392230572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c0d2e50 con 0x564f87a55ab0 2024-04-16T05:41:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde5b7fe640 1 -- 172.21.15.32:0/392230572 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde6c0beab0 con 0x564f87a55ab0 2024-04-16T05:41:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 10 client.17241 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 1 -- 172.21.15.32:0/392230572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f87ad4a10 con 0x564f87a55ab0 2024-04-16T05:41:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde5b7fe640 1 -- 172.21.15.32:0/392230572 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c0d2e50 con 0x564f87a55ab0 2024-04-16T05:41:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde5b7fe640 1 -- 172.21.15.32:0/392230572 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fde6c0f2280 con 0x564f87a55ab0 2024-04-16T05:41:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 10 client.17241 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 20 client.17241 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564f87ad5cd0 0x564f87ad80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 4 client.17241 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:41:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.246+0000 7fde851a0740 1 -- 172.21.15.32:0/392230572 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f879c2720 con 0x564f87ad5cd0 2024-04-16T05:41:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.247+0000 7fde73fff640 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564f87ad5cd0 0x564f87ad80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.247+0000 7fde73fff640 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564f87ad5cd0 0x564f87ad80c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fde4dedd824a952 server_cookie=b4d710e7d3d461c0 in_seq=0 out_seq=0 2024-04-16T05:41:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.247+0000 7fde5b7fe640 10 client.17241 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde5b7fe640 1 -- 172.21.15.32:0/392230572 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f879c2720 con 0x564f87ad5cd0 2024-04-16T05:41:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde5b7fe640 10 client.17241 handle_command_reply: tid=0 2024-04-16T05:41:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 client.17241 shutdown 2024-04-16T05:41:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 -- 172.21.15.32:0/392230572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 msgr2=0x564f87a590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a590b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0ba810 tx=0x7fde6c0c2760 comp rx=0 tx=0).stop 2024-04-16T05:41:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 -- 172.21.15.32:0/392230572 shutdown_connections 2024-04-16T05:41:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f87a5efb0 0x564f87ad44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564f87ad5cd0 0x564f87ad80c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f87a563d0 0x564f87a595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 --2- 172.21.15.32:0/392230572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f87a55ab0 0x564f87a590b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 -- 172.21.15.32:0/392230572 >> 172.21.15.32:0/392230572 conn(0x564f87a501e0 msgr2=0x564f87a5c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde5b7fe640 0 client.17241 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 -- 172.21.15.32:0/392230572 shutdown_connections 2024-04-16T05:41:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.248+0000 7fde851a0740 1 -- 172.21.15.32:0/392230572 wait complete. 2024-04-16T05:41:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde851a0740 20 client.17241 trim_cache size 0 max 16384 2024-04-16T05:41:32.260 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:41:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:41:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "39168391-2554-464c-ada4-24fee5860cfe", 2024-04-16T05:41:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:41:32.260 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:41:32.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 -- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fde680a3d20 msgr2=0x7fde680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fde680a3d20 0x7fde680a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x564f87a51370 tx=0x7fde74009a60 comp rx=0 tx=0).stop 2024-04-16T05:41:32.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 -- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 msgr2=0x7fde7c19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 0x7fde7c19e600 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fde6c006010 tx=0x7fde6c02cc70 comp rx=0 tx=0).stop 2024-04-16T05:41:32.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 -- 172.21.15.32:0/98541194 shutdown_connections 2024-04-16T05:41:32.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c1986c0 0x7fde7c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7c106210 0x7fde7c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fde680a3d20 0x7fde680a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 --2- 172.21.15.32:0/98541194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7c101140 0x7fde7c19e600 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 -- 172.21.15.32:0/98541194 >> 172.21.15.32:0/98541194 conn(0x7fde7c078000 msgr2=0x7fde7c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 -- 172.21.15.32:0/98541194 shutdown_connections 2024-04-16T05:41:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.249+0000 7fde824d9640 1 -- 172.21.15.32:0/98541194 wait complete. 2024-04-16T05:41:32.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:41:32.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:41:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.411+0000 7f6ef5e93640 1 -- 172.21.15.32:0/1699588044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 msgr2=0x7f6ef010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.411+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/1699588044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 0x7f6ef010ddf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009ca0 tx=0x7f6ee802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 -- 172.21.15.32:0/1699588044 shutdown_connections 2024-04-16T05:41:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/1699588044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 0x7f6ef010ddf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/1699588044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef0101b50 0x7f6ef0105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/1699588044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef01011a0 0x7f6ef0101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 -- 172.21.15.32:0/1699588044 >> 172.21.15.32:0/1699588044 conn(0x7f6ef0078060 msgr2=0x7f6ef0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 -- 172.21.15.32:0/1699588044 shutdown_connections 2024-04-16T05:41:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 -- 172.21.15.32:0/1699588044 wait complete. 2024-04-16T05:41:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 Processor -- start 2024-04-16T05:41:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.412+0000 7f6ef5e93640 1 -- start start 2024-04-16T05:41:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6ef5e93640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef01011a0 0x7f6ef019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6ef5e93640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 0x7f6ef019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6ef5e93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef01986d0 0x7f6ef0198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6ef5e93640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01112e0 con 0x7f6ef0106220 2024-04-16T05:41:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6ef5e93640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0111160 con 0x7f6ef01011a0 2024-04-16T05:41:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6ef5e93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0111460 con 0x7f6ef01986d0 2024-04-16T05:41:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 0x7f6ef019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 0x7f6ef019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57674/0 (socket says 172.21.15.32:57674) 2024-04-16T05:41:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 -- 172.21.15.32:0/2831737108 learned_addr learned my addr 172.21.15.32:0/2831737108 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef01011a0 0x7f6ef019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeffff640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef01986d0 0x7f6ef0198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 -- 172.21.15.32:0/2831737108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef01986d0 msgr2=0x7f6ef0198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef01986d0 0x7f6ef0198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 -- 172.21.15.32:0/2831737108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef01011a0 msgr2=0x7f6ef019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef01011a0 0x7f6ef019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 -- 172.21.15.32:0/2831737108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef01992d0 con 0x7f6ef0106220 2024-04-16T05:41:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeffff640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef01986d0 0x7f6ef0198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:41:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 0x7f6ef019eb50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400e9c0 tx=0x7f6ee400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eecff9640 1 -- 172.21.15.32:0/2831737108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee400cd80 con 0x7f6ef0106220 2024-04-16T05:41:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eecff9640 1 -- 172.21.15.32:0/2831737108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee4005500 con 0x7f6ef0106220 2024-04-16T05:41:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6ef5e93640 1 -- 172.21.15.32:0/2831737108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef01a53b0 con 0x7f6ef0106220 2024-04-16T05:41:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6eecff9640 1 -- 172.21.15.32:0/2831737108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4010690 con 0x7f6ef0106220 2024-04-16T05:41:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.413+0000 7f6ef5e93640 1 -- 172.21.15.32:0/2831737108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef01a57b0 con 0x7f6ef0106220 2024-04-16T05:41:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6eecff9640 1 -- 172.21.15.32:0/2831737108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ee40046c0 con 0x7f6ef0106220 2024-04-16T05:41:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6ef8a80740 1 Processor -- start 2024-04-16T05:41:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6ef8a80740 1 -- start start 2024-04-16T05:41:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01cc7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01ccfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01cd2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563f01c2c180 con 0x563f01cc6e50 2024-04-16T05:41:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563f01bc9360 con 0x563f01cc7770 2024-04-16T05:41:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f01c28140 con 0x563f01cd0350 2024-04-16T05:41:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6eecff9640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ed80a3d20 0x7f6ed80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01cc7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01cc7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57680/0 (socket says 172.21.15.32:57680) 2024-04-16T05:41:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.416+0000 7f6eeffff640 1 -- 172.21.15.32:0/3827741755 learned_addr learned my addr 172.21.15.32:0/3827741755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeffff640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ed80a3d20 0x7f6ed80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01ccfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eecff9640 1 -- 172.21.15.32:0/2831737108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6ee40d0ec0 con 0x7f6ef0106220 2024-04-16T05:41:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01cd2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeffff640 1 -- 172.21.15.32:0/3827741755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 msgr2=0x563f01cd2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeffff640 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01cd2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeffff640 1 -- 172.21.15.32:0/3827741755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 msgr2=0x563f01ccfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeffff640 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01ccfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeffff640 1 -- 172.21.15.32:0/3827741755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f01a087d0 con 0x563f01cc6e50 2024-04-16T05:41:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01ccfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:41:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeffff640 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01cc7230 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8032920 tx=0x7f6ee8032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22e5932b90eac4c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6eeffff640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ed80a3d20 0x7f6ed80a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009a60 tx=0x7f6ee8040000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6ed5ffb640 1 -- 172.21.15.32:0/3827741755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8050070 con 0x563f01cc6e50 2024-04-16T05:41:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6ed5ffb640 1 -- 172.21.15.32:0/3827741755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee803a040 con 0x563f01cc6e50 2024-04-16T05:41:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.417+0000 7f6ed5ffb640 1 -- 172.21.15.32:0/3827741755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8039040 con 0x563f01cc6e50 2024-04-16T05:41:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 -- 172.21.15.32:0/3827741755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 msgr2=0x563f01cc7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01cc7230 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8032920 tx=0x7f6ee8032950 comp rx=0 tx=0).stop 2024-04-16T05:41:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 -- 172.21.15.32:0/3827741755 shutdown_connections 2024-04-16T05:41:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01cd2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01ccfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/3827741755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01cc7230 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 -- 172.21.15.32:0/3827741755 >> 172.21.15.32:0/3827741755 conn(0x563f01cc1580 msgr2=0x563f01cc1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 -- 172.21.15.32:0/3827741755 shutdown_connections 2024-04-16T05:41:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 -- 172.21.15.32:0/3827741755 wait complete. 2024-04-16T05:41:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.418+0000 7f6ef8a80740 1 -- start start 2024-04-16T05:41:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6ef8a80740 1 Processor -- start 2024-04-16T05:41:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 0x563f01ccd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc7770 0x563f01ccd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01ccde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563f01cd30e0 con 0x563f01cc7770 2024-04-16T05:41:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563f01be52c0 con 0x563f01cc6e50 2024-04-16T05:41:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f01cd3260 con 0x563f01cd0350 2024-04-16T05:41:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6ef8a80740 10 client.0 _ll_register_callbacks cb 0x563f01bcc090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc7770 0x563f01ccd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc7770 0x563f01ccd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57690/0 (socket says 172.21.15.32:57690) 2024-04-16T05:41:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 -- 172.21.15.32:0/1668422319 learned_addr learned my addr 172.21.15.32:0/1668422319 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeffff640 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01ccde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 0x563f01ccd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 -- 172.21.15.32:0/1668422319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 msgr2=0x563f01ccde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01ccde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 -- 172.21.15.32:0/1668422319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 msgr2=0x563f01ccd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 0x563f01ccd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 -- 172.21.15.32:0/1668422319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0193e3f0 con 0x563f01cc7770 2024-04-16T05:41:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc7770 0x563f01ccd8c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4098f80 tx=0x7f6ee409c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4bf7bf81bffff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.419+0000 7f6eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6eceffd640 1 -- 172.21.15.32:0/1668422319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40a0ed0 con 0x563f01cc7770 2024-04-16T05:41:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6eceffd640 1 -- 172.21.15.32:0/1668422319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6ee40a5300 con 0x563f01cc7770 2024-04-16T05:41:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1668422319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee803a040 con 0x563f01cc7770 2024-04-16T05:41:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6eceffd640 1 -- 172.21.15.32:0/1668422319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ee40a5790 con 0x563f01cc7770 2024-04-16T05:41:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 10 client.17253 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1668422319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563f018fe0f0 con 0x563f01cc7770 2024-04-16T05:41:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6eceffd640 1 -- 172.21.15.32:0/1668422319 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee803a040 con 0x563f01cc7770 2024-04-16T05:41:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6eceffd640 1 -- 172.21.15.32:0/1668422319 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ee40bbce0 con 0x563f01cc7770 2024-04-16T05:41:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 10 client.17253 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 20 client.17253 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563f01ce06d0 0x563f01ce2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 4 client.17253 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:41:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1668422319 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f01c39ba0 con 0x563f01ce06d0 2024-04-16T05:41:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.420+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563f01ce06d0 0x563f01ce2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.421+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563f01ce06d0 0x563f01ce2ac0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95595bf6e5e65f23 server_cookie=1fd3ca6c93ceaccf in_seq=0 out_seq=0 2024-04-16T05:41:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.421+0000 7f6eceffd640 10 client.17253 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.421+0000 7f6eceffd640 1 -- 172.21.15.32:0/1668422319 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f01c39ba0 con 0x563f01ce06d0 2024-04-16T05:41:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.421+0000 7f6eceffd640 10 client.17253 handle_command_reply: tid=0 2024-04-16T05:41:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 client.17253 shutdown 2024-04-16T05:41:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1668422319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc7770 msgr2=0x563f01ccd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc7770 0x563f01ccd8c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4098f80 tx=0x7f6ee409c720 comp rx=0 tx=0).stop 2024-04-16T05:41:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1668422319 shutdown_connections 2024-04-16T05:41:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563f01ce06d0 0x563f01ce2ac0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01ccde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc7770 0x563f01ccd8c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1668422319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 0x563f01ccd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6eceffd640 0 client.17253 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1668422319 >> 172.21.15.32:0/1668422319 conn(0x563f01cc1580 msgr2=0x563f01cc32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1668422319 shutdown_connections 2024-04-16T05:41:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1668422319 wait complete. 2024-04-16T05:41:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.422+0000 7f6ef8a80740 20 client.17253 trim_cache size 0 max 16384 2024-04-16T05:41:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.439+0000 7f6ef8a80740 1 Processor -- start 2024-04-16T05:41:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.439+0000 7f6ef8a80740 1 -- start start 2024-04-16T05:41:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.439+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01ccfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.439+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01cdbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01cd8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b1c70 con 0x563f01cc6e50 2024-04-16T05:41:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01af540 con 0x563f01cc7770 2024-04-16T05:41:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01ad390 con 0x563f01cd0350 2024-04-16T05:41:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01ccfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01ccfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57694/0 (socket says 172.21.15.32:57694) 2024-04-16T05:41:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 -- 172.21.15.32:0/1323155425 learned_addr learned my addr 172.21.15.32:0/1323155425 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/1323155425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01cdbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 -- 172.21.15.32:0/1323155425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 msgr2=0x563f01cd8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1323155425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01cd8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 -- 172.21.15.32:0/1323155425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 msgr2=0x563f01cdbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1323155425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01cdbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 -- 172.21.15.32:0/1323155425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f01cc60e0 con 0x563f01cc6e50 2024-04-16T05:41:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1323155425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01ccfab0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x563f01ce3290 tx=0x7f6ee000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7823488cc854e33c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6ed57fa640 1 -- 172.21.15.32:0/1323155425 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0026070 con 0x563f01cc6e50 2024-04-16T05:41:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6ed57fa640 1 -- 172.21.15.32:0/1323155425 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee0020690 con 0x563f01cc6e50 2024-04-16T05:41:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.440+0000 7f6ed57fa640 1 -- 172.21.15.32:0/1323155425 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0011cb0 con 0x563f01cc6e50 2024-04-16T05:41:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1323155425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 msgr2=0x563f01ccfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1323155425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01ccfab0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x563f01ce3290 tx=0x7f6ee000b390 comp rx=0 tx=0).stop 2024-04-16T05:41:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1323155425 shutdown_connections 2024-04-16T05:41:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1323155425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cd0350 0x563f01cd8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1323155425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc7770 0x563f01cdbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1323155425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cc6e50 0x563f01ccfab0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1323155425 >> 172.21.15.32:0/1323155425 conn(0x563f01cc1580 msgr2=0x563f01cbfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1323155425 shutdown_connections 2024-04-16T05:41:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1323155425 wait complete. 2024-04-16T05:41:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 -- start start 2024-04-16T05:41:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.441+0000 7f6ef8a80740 1 Processor -- start 2024-04-16T05:41:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 0x563f01d43560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cc7770 0x563f01d43aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6ef8a80740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cd0350 0x563f01ce7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01ada00 con 0x563f01cd0350 2024-04-16T05:41:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01aee70 con 0x563f01cc6e50 2024-04-16T05:41:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6ef8a80740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01ae0d0 con 0x563f01cc7770 2024-04-16T05:41:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6ef8a80740 10 client.0 _ll_register_callbacks cb 0x7f6ee400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:41:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cd0350 0x563f01ce7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cd0350 0x563f01ce7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57704/0 (socket says 172.21.15.32:57704) 2024-04-16T05:41:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 -- 172.21.15.32:0/1905142889 learned_addr learned my addr 172.21.15.32:0/1905142889 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:41:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 0x563f01d43560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeeffd640 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cc7770 0x563f01d43aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 -- 172.21.15.32:0/1905142889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cc7770 msgr2=0x563f01d43aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cc7770 0x563f01d43aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 -- 172.21.15.32:0/1905142889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 msgr2=0x563f01d43560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:41:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 0x563f01d43560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 -- 172.21.15.32:0/1905142889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f01ce8360 con 0x563f01cd0350 2024-04-16T05:41:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eeffff640 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cd0350 0x563f01ce7c20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6ee802c6d0 tx=0x7f6ee80099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b134f91d55037e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:41:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:41:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eceffd640 1 -- 172.21.15.32:0/1905142889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8050040 con 0x563f01cd0350 2024-04-16T05:41:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6eceffd640 1 -- 172.21.15.32:0/1905142889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6ee8004040 con 0x563f01cd0350 2024-04-16T05:41:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.442+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1905142889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee0020690 con 0x563f01cd0350 2024-04-16T05:41:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6eceffd640 1 -- 172.21.15.32:0/1905142889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ee805e050 con 0x563f01cd0350 2024-04-16T05:41:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6ef8a80740 10 client.17259 fetch_fsmap learned FSMap version 25 2024-04-16T05:41:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1905142889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563f01d4ad90 con 0x563f01cd0350 2024-04-16T05:41:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6eceffd640 1 -- 172.21.15.32:0/1905142889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee0020690 con 0x563f01cd0350 2024-04-16T05:41:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6eceffd640 1 -- 172.21.15.32:0/1905142889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ee80556b0 con 0x563f01cd0350 2024-04-16T05:41:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6ef8a80740 10 client.17259 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:41:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6ef8a80740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:41:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6ef8a80740 20 client.17259 populate_metadata read hostname 'smithi032' 2024-04-16T05:41:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563f01d4b110 0x563f01d4d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:41:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6ef8a80740 4 client.17259 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:41:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1905142889 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f01c33a10 con 0x563f01d4b110 2024-04-16T05:41:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.443+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563f01d4b110 0x563f01d4d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:41:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6eef7fe640 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563f01d4b110 0x563f01d4d500 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5077cfdb9200c116 server_cookie=22db2c68bb14f6c2 in_seq=0 out_seq=0 2024-04-16T05:41:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6eceffd640 10 client.17259 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6eceffd640 1 -- 172.21.15.32:0/1905142889 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563f01c33a10 con 0x563f01d4b110 2024-04-16T05:41:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6eceffd640 10 client.17259 handle_command_reply: tid=0 2024-04-16T05:41:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 client.17259 shutdown 2024-04-16T05:41:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1905142889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cd0350 msgr2=0x563f01ce7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cd0350 0x563f01ce7c20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6ee802c6d0 tx=0x7f6ee80099c0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1905142889 shutdown_connections 2024-04-16T05:41:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563f01cd0350 0x563f01ce7c20 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563f01d4b110 0x563f01d4d500 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f01cc7770 0x563f01d43aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 --2- 172.21.15.32:0/1905142889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f01cc6e50 0x563f01d43560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6eceffd640 0 client.17259 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:41:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1905142889 >> 172.21.15.32:0/1905142889 conn(0x563f01cc1580 msgr2=0x563f01ccb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1905142889 shutdown_connections 2024-04-16T05:41:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 1 -- 172.21.15.32:0/1905142889 wait complete. 2024-04-16T05:41:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.444+0000 7f6ef8a80740 20 client.17259 trim_cache size 0 max 16384 2024-04-16T05:41:32.455 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:41:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:41:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:41:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "39168391-2554-464c-ada4-24fee5860cfe": { 2024-04-16T05:41:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:41:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "39168391-2554-464c-ada4-24fee5860cfe", 2024-04-16T05:41:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:41:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:41:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:41:32.456 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:41:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 -- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ed80a3d20 msgr2=0x7f6ed80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ed80a3d20 0x7f6ed80a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009a60 tx=0x7f6ee8040000 comp rx=0 tx=0).stop 2024-04-16T05:41:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 -- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 msgr2=0x7f6ef019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:41:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 0x7f6ef019eb50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400e9c0 tx=0x7f6ee400ee80 comp rx=0 tx=0).stop 2024-04-16T05:41:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 -- 172.21.15.32:0/2831737108 shutdown_connections 2024-04-16T05:41:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef01986d0 0x7f6ef0198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f6ed80a3d20 0x7f6ed80a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0106220 0x7f6ef019eb50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 --2- 172.21.15.32:0/2831737108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef01011a0 0x7f6ef019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:41:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 -- 172.21.15.32:0/2831737108 >> 172.21.15.32:0/2831737108 conn(0x7f6ef0078060 msgr2=0x7f6ef0104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:41:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 -- 172.21.15.32:0/2831737108 shutdown_connections 2024-04-16T05:41:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:41:32.445+0000 7f6ef5e93640 1 -- 172.21.15.32:0/2831737108 wait complete. 2024-04-16T05:41:32.460 INFO:tasks.cephfs.filesystem:scrub status for tag:39168391-2554-464c-ada4-24fee5860cfe - {'path': '/', 'tag': '39168391-2554-464c-ada4-24fee5860cfe', 'options': 'recursive,force'} 2024-04-16T05:41:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:32 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:41:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:32 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:33.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:32 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:41:33.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:32 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:33.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:32 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:41:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:32 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:32 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:41:33.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:32 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:33.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:32 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:41:33.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:32 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:32 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:41:33.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:32 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:41:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:33 smithi134 ceph-mon[36153]: pgmap v201: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:34.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:33 smithi082 ceph-mon[36195]: pgmap v201: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:34.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:33 smithi032 ceph-mon[32351]: pgmap v201: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:35 smithi134 ceph-mon[36153]: pgmap v202: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:35 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:41:36.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:35 smithi082 ceph-mon[36195]: pgmap v202: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:35 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:41:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:35 smithi032 ceph-mon[32351]: pgmap v202: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:36.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:35 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:41:37.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:36 smithi082 ceph-mon[36195]: pgmap v203: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:37.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:36 smithi032 ceph-mon[32351]: pgmap v203: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:37.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:36 smithi134 ceph-mon[36153]: pgmap v203: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:39.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:39 smithi032 ceph-mon[32351]: pgmap v204: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:39.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:39 smithi134 ceph-mon[36153]: pgmap v204: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:39.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:39 smithi082 ceph-mon[36195]: pgmap v204: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:41.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:41 smithi032 ceph-mon[32351]: pgmap v205: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:41.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:41 smithi134 ceph-mon[36153]: pgmap v205: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:41.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:41 smithi082 ceph-mon[36195]: pgmap v205: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:43.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:43 smithi032 ceph-mon[32351]: pgmap v206: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:43.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:43 smithi134 ceph-mon[36153]: pgmap v206: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:43.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:43 smithi082 ceph-mon[36195]: pgmap v206: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:45.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:45 smithi032 ceph-mon[32351]: pgmap v207: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:45.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:45 smithi134 ceph-mon[36153]: pgmap v207: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:45.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:45 smithi082 ceph-mon[36195]: pgmap v207: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:47.452 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:47 smithi082 ceph-mon[36195]: pgmap v208: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:47.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:47 smithi032 ceph-mon[32351]: pgmap v208: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:47.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:47 smithi134 ceph-mon[36153]: pgmap v208: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:41:49.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:49 smithi032 ceph-mon[32351]: pgmap v209: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:49.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:49 smithi134 ceph-mon[36153]: pgmap v209: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:49.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:49 smithi082 ceph-mon[36195]: pgmap v209: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:41:51.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:51 smithi032 ceph-mon[32351]: pgmap v210: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:51.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:51 smithi134 ceph-mon[36153]: pgmap v210: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:51.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:51 smithi082 ceph-mon[36195]: pgmap v210: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:53.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:53 smithi032 ceph-mon[32351]: pgmap v211: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:53.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:53 smithi134 ceph-mon[36153]: pgmap v211: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:53.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:53 smithi082 ceph-mon[36195]: pgmap v211: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:55.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:55 smithi032 ceph-mon[32351]: pgmap v212: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:55.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:55 smithi134 ceph-mon[36153]: pgmap v212: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:55.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:55 smithi082 ceph-mon[36195]: pgmap v212: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:57.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:57 smithi032 ceph-mon[32351]: pgmap v213: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:57.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:57 smithi134 ceph-mon[36153]: pgmap v213: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:57.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:57 smithi082 ceph-mon[36195]: pgmap v213: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:41:59.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:41:59 smithi032 ceph-mon[32351]: pgmap v214: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:59.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:41:59 smithi134 ceph-mon[36153]: pgmap v214: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:41:59.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:41:59 smithi082 ceph-mon[36195]: pgmap v214: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:01.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:01 smithi032 ceph-mon[32351]: pgmap v215: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:01.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:01 smithi134 ceph-mon[36153]: pgmap v215: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:01.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:01 smithi082 ceph-mon[36195]: pgmap v215: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:02.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:42:02.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:42:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 -- 172.21.15.32:0/715916025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 msgr2=0x7f16a006aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 --2- 172.21.15.32:0/715916025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 0x7f16a006aa40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1694009930 tx=0x7f169402bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 -- 172.21.15.32:0/715916025 shutdown_connections 2024-04-16T05:42:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 --2- 172.21.15.32:0/715916025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a006af80 0x7f16a010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 --2- 172.21.15.32:0/715916025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 0x7f16a006aa40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 --2- 172.21.15.32:0/715916025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16a0108070 0x7f16a0108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 -- 172.21.15.32:0/715916025 >> 172.21.15.32:0/715916025 conn(0x7f16a0076d00 msgr2=0x7f16a0077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 -- 172.21.15.32:0/715916025 shutdown_connections 2024-04-16T05:42:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 -- 172.21.15.32:0/715916025 wait complete. 2024-04-16T05:42:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.584+0000 7f16a57a8640 1 Processor -- start 2024-04-16T05:42:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f16a57a8640 1 -- start start 2024-04-16T05:42:02.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f16a57a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 0x7f16a019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f16a57a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a006af80 0x7f16a019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f16a57a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16a0108070 0x7f16a0198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f16a57a8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0112390 con 0x7f16a006a600 2024-04-16T05:42:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f16a57a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0112210 con 0x7f16a0108070 2024-04-16T05:42:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f16a57a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0112510 con 0x7f16a006af80 2024-04-16T05:42:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 0x7f16a019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 0x7f16a019e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37544/0 (socket says 172.21.15.32:37544) 2024-04-16T05:42:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169effd640 1 -- 172.21.15.32:0/2006317878 learned_addr learned my addr 172.21.15.32:0/2006317878 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169e7fc640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a006af80 0x7f16a019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169f7fe640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16a0108070 0x7f16a0198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169effd640 1 -- 172.21.15.32:0/2006317878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a006af80 msgr2=0x7f16a019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169effd640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a006af80 0x7f16a019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169effd640 1 -- 172.21.15.32:0/2006317878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16a0108070 msgr2=0x7f16a0198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169effd640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16a0108070 0x7f16a0198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.585+0000 7f169effd640 1 -- 172.21.15.32:0/2006317878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a0198f50 con 0x7f16a006a600 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.586+0000 7f169effd640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 0x7f16a019e5d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f169000cce0 tx=0x7f169000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.586+0000 7f1687fff640 1 -- 172.21.15.32:0/2006317878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169000f650 con 0x7f16a006a600 2024-04-16T05:42:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.586+0000 7f1687fff640 1 -- 172.21.15.32:0/2006317878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169000fcd0 con 0x7f16a006a600 2024-04-16T05:42:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.586+0000 7f16a57a8640 1 -- 172.21.15.32:0/2006317878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a0199240 con 0x7f16a006a600 2024-04-16T05:42:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.586+0000 7f1687fff640 1 -- 172.21.15.32:0/2006317878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690018c70 con 0x7f16a006a600 2024-04-16T05:42:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.586+0000 7f16a57a8640 1 -- 172.21.15.32:0/2006317878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a01a5550 con 0x7f16a006a600 2024-04-16T05:42:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.588+0000 7f1687fff640 1 -- 172.21.15.32:0/2006317878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f169000f7f0 con 0x7f16a006a600 2024-04-16T05:42:02.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f16a838c740 1 Processor -- start 2024-04-16T05:42:02.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f16a838c740 1 -- start start 2024-04-16T05:42:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f1687fff640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f16880a3d20 0x7f16880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f16a838c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cec2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f16a838c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758ced0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f16a838c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 0x55758ced2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f16a838c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55758ce28180 con 0x55758cec2b00 2024-04-16T05:42:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f16a838c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55758cdc5360 con 0x55758cec3420 2024-04-16T05:42:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f16a838c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55758ce24140 con 0x55758ced0760 2024-04-16T05:42:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169f7fe640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f16880a3d20 0x7f16880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cec2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f1687fff640 1 -- 172.21.15.32:0/2006317878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f16900d0ee0 con 0x7f16a006a600 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 0x55758ced2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cec2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37560/0 (socket says 172.21.15.32:37560) 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169effd640 1 -- 172.21.15.32:0/933676038 learned_addr learned my addr 172.21.15.32:0/933676038 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169e7fc640 1 --2- 172.21.15.32:0/933676038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758ced0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169effd640 1 -- 172.21.15.32:0/933676038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 msgr2=0x55758ced2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169effd640 1 --2- 172.21.15.32:0/933676038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 0x55758ced2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169effd640 1 -- 172.21.15.32:0/933676038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 msgr2=0x55758ced0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169effd640 1 --2- 172.21.15.32:0/933676038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758ced0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.589+0000 7f169effd640 1 -- 172.21.15.32:0/933676038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758cc047d0 con 0x55758cec2b00 2024-04-16T05:42:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f169f7fe640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f16880a3d20 0x7f16880a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1698009710 tx=0x7f1698009ad0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f169effd640 1 --2- 172.21.15.32:0/933676038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cec2ee0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1690000f80 tx=0x7f169000fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32ce6dfc52221bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16857fa640 1 -- 172.21.15.32:0/933676038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169009c4a0 con 0x55758cec2b00 2024-04-16T05:42:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16857fa640 1 -- 172.21.15.32:0/933676038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169009d240 con 0x55758cec2b00 2024-04-16T05:42:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16857fa640 1 -- 172.21.15.32:0/933676038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690018e10 con 0x55758cec2b00 2024-04-16T05:42:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16a838c740 1 -- 172.21.15.32:0/933676038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 msgr2=0x55758cec2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16a838c740 1 --2- 172.21.15.32:0/933676038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cec2ee0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1690000f80 tx=0x7f169000fe80 comp rx=0 tx=0).stop 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16a838c740 1 -- 172.21.15.32:0/933676038 shutdown_connections 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16a838c740 1 --2- 172.21.15.32:0/933676038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 0x55758ced2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16a838c740 1 --2- 172.21.15.32:0/933676038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758ced0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16a838c740 1 --2- 172.21.15.32:0/933676038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cec2ee0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.590+0000 7f16a838c740 1 -- 172.21.15.32:0/933676038 >> 172.21.15.32:0/933676038 conn(0x55758cebd580 msgr2=0x55758cebd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 -- 172.21.15.32:0/933676038 shutdown_connections 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 -- 172.21.15.32:0/933676038 wait complete. 2024-04-16T05:42:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 -- start start 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 Processor -- start 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758cecd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758cecdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758cece0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55758ced35a0 con 0x55758cec3420 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55758cde12c0 con 0x55758ced0760 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55758ced3720 con 0x55758cec2b00 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f16a838c740 10 client.0 _ll_register_callbacks cb 0x55758cdc8090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f169e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758cecdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f169e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758cecdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37566/0 (socket says 172.21.15.32:37566) 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f169e7fc640 1 -- 172.21.15.32:0/3322285316 learned_addr learned my addr 172.21.15.32:0/3322285316 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.591+0000 7f169f7fe640 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758cece0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f169effd640 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758cecd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f169e7fc640 1 -- 172.21.15.32:0/3322285316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 msgr2=0x55758cecd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f169e7fc640 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758cecd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f169e7fc640 1 -- 172.21.15.32:0/3322285316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 msgr2=0x55758cece0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f169e7fc640 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758cece0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f169e7fc640 1 -- 172.21.15.32:0/3322285316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758cb3a3f0 con 0x55758cec3420 2024-04-16T05:42:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f169e7fc640 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758cecdba0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f169402cfd0 tx=0x7f1694005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c16f708ff53b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f1682ff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f1682ff5640 1 -- 172.21.15.32:0/3322285316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694004340 con 0x55758cec3420 2024-04-16T05:42:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f1682ff5640 1 -- 172.21.15.32:0/3322285316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f16940046c0 con 0x55758cec3420 2024-04-16T05:42:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f16a838c740 1 -- 172.21.15.32:0/3322285316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f169009d240 con 0x55758cec3420 2024-04-16T05:42:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f1682ff5640 1 -- 172.21.15.32:0/3322285316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16940044e0 con 0x55758cec3420 2024-04-16T05:42:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f16a838c740 10 client.17271 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f16a838c740 1 -- 172.21.15.32:0/3322285316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55758cafa0f0 con 0x55758cec3420 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.592+0000 7f1682ff5640 1 -- 172.21.15.32:0/3322285316 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169009d240 con 0x55758cec3420 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f1682ff5640 1 -- 172.21.15.32:0/3322285316 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16940035f0 con 0x55758cec3420 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f16a838c740 10 client.17271 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f16a838c740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f16a838c740 20 client.17271 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f16a838c740 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55758cee0ac0 0x55758cee2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f16a838c740 4 client.17271 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f16a838c740 1 -- 172.21.15.32:0/3322285316 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55758ce35ba0 con 0x55758cee0ac0 2024-04-16T05:42:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f169effd640 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55758cee0ac0 0x55758cee2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f169effd640 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55758cee0ac0 0x55758cee2f40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0f49114ed12d1f server_cookie=b8cf71859310ebd9 in_seq=0 out_seq=0 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.593+0000 7f1682ff5640 10 client.17271 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f1682ff5640 1 -- 172.21.15.32:0/3322285316 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55758ce35ba0 con 0x55758cee0ac0 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f1682ff5640 10 client.17271 handle_command_reply: tid=0 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 client.17271 shutdown 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 -- 172.21.15.32:0/3322285316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 msgr2=0x55758cecdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758cecdba0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f169402cfd0 tx=0x7f1694005e10 comp rx=0 tx=0).stop 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 -- 172.21.15.32:0/3322285316 shutdown_connections 2024-04-16T05:42:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758cece0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758cecdba0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55758cee0ac0 0x55758cee2f40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 --2- 172.21.15.32:0/3322285316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758cecd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 -- 172.21.15.32:0/3322285316 >> 172.21.15.32:0/3322285316 conn(0x55758cebd580 msgr2=0x55758cebf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f1682ff5640 0 client.17271 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 -- 172.21.15.32:0/3322285316 shutdown_connections 2024-04-16T05:42:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 1 -- 172.21.15.32:0/3322285316 wait complete. 2024-04-16T05:42:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.594+0000 7f16a838c740 20 client.17271 trim_cache size 0 max 16384 2024-04-16T05:42:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f16a838c740 1 Processor -- start 2024-04-16T05:42:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f16a838c740 1 -- start start 2024-04-16T05:42:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f16a838c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cecfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f16a838c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758cedc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f16a838c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 0x55758cebd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f16a838c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01af960 con 0x55758cec2b00 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f16a838c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01ad230 con 0x55758cec3420 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f16a838c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01ab080 con 0x55758ced0760 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.612+0000 7f169effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cecfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cecfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37582/0 (socket says 172.21.15.32:37582) 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169effd640 1 -- 172.21.15.32:0/614025589 learned_addr learned my addr 172.21.15.32:0/614025589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169e7fc640 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758cedc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169effd640 1 -- 172.21.15.32:0/614025589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 msgr2=0x55758cebd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169effd640 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 0x55758cebd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169effd640 1 -- 172.21.15.32:0/614025589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 msgr2=0x55758cedc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169effd640 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758cedc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169effd640 1 -- 172.21.15.32:0/614025589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758cebd3b0 con 0x55758cec2b00 2024-04-16T05:42:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169e7fc640 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758cedc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f169effd640 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cecfd90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f169000ccb0 tx=0x7f1690020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352651c9f8f14cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f1684ff9640 1 -- 172.21.15.32:0/614025589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169009d240 con 0x55758cec2b00 2024-04-16T05:42:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f1684ff9640 1 -- 172.21.15.32:0/614025589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16900a0ef0 con 0x55758cec2b00 2024-04-16T05:42:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.613+0000 7f1684ff9640 1 -- 172.21.15.32:0/614025589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16900991b0 con 0x55758cec2b00 2024-04-16T05:42:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- 172.21.15.32:0/614025589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 msgr2=0x55758cecfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cecfd90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f169000ccb0 tx=0x7f1690020000 comp rx=0 tx=0).stop 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- 172.21.15.32:0/614025589 shutdown_connections 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758ced0760 0x55758cebd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758cec3420 0x55758cedc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 --2- 172.21.15.32:0/614025589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec2b00 0x55758cecfd90 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- 172.21.15.32:0/614025589 >> 172.21.15.32:0/614025589 conn(0x55758cebd580 msgr2=0x55758cebe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- 172.21.15.32:0/614025589 shutdown_connections 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- 172.21.15.32:0/614025589 wait complete. 2024-04-16T05:42:02.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- start start 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 Processor -- start 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758ced8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758ced5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758ced64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01ab6f0 con 0x55758cec3420 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01acb60 con 0x55758ced0760 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01abdc0 con 0x55758cec2b00 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f16a838c740 10 client.0 _ll_register_callbacks cb 0x7f169402cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f169e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758ced5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.614+0000 7f169e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758ced5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37588/0 (socket says 172.21.15.32:37588) 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169e7fc640 1 -- 172.21.15.32:0/2698817280 learned_addr learned my addr 172.21.15.32:0/2698817280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169f7fe640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758ced64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169effd640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758ced8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169e7fc640 1 -- 172.21.15.32:0/2698817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 msgr2=0x55758ced8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169e7fc640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758ced8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169e7fc640 1 -- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 msgr2=0x55758ced64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169e7fc640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758ced64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169e7fc640 1 -- 172.21.15.32:0/2698817280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758ced3740 con 0x55758cec3420 2024-04-16T05:42:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169effd640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758ced8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169f7fe640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758ced64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f169e7fc640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758ced5fb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f169402c170 tx=0x7f1694003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd878959c4b1062 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f1682ff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f1682ff5640 1 -- 172.21.15.32:0/2698817280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694059070 con 0x55758cec3420 2024-04-16T05:42:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f1682ff5640 1 -- 172.21.15.32:0/2698817280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f1694005360 con 0x55758cec3420 2024-04-16T05:42:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f16a838c740 1 -- 172.21.15.32:0/2698817280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55758ce2fa10 con 0x55758cec3420 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f1682ff5640 1 -- 172.21.15.32:0/2698817280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1694062050 con 0x55758cec3420 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f16a838c740 10 client.17277 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.615+0000 7f16a838c740 1 -- 172.21.15.32:0/2698817280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55758cf46c10 con 0x55758cec3420 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f1682ff5640 1 -- 172.21.15.32:0/2698817280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55758ce2fa10 con 0x55758cec3420 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f1682ff5640 1 -- 172.21.15.32:0/2698817280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f169405c020 con 0x55758cec3420 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f16a838c740 10 client.17277 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f16a838c740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f16a838c740 20 client.17277 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f16a838c740 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55758cf47350 0x55758cf49770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f16a838c740 4 client.17277 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f16a838c740 1 -- 172.21.15.32:0/2698817280 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55758cdab560 con 0x55758cf47350 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f169effd640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55758cf47350 0x55758cf49770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f169effd640 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55758cf47350 0x55758cf49770 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9704c1af167ef009 server_cookie=4d0ee6d76670566f in_seq=0 out_seq=0 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.616+0000 7f1682ff5640 10 client.17277 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f1682ff5640 1 -- 172.21.15.32:0/2698817280 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55758cdab560 con 0x55758cf47350 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f1682ff5640 10 client.17277 handle_command_reply: tid=0 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 client.17277 shutdown 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 -- 172.21.15.32:0/2698817280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 msgr2=0x55758ced5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758ced5fb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f169402c170 tx=0x7f1694003820 comp rx=0 tx=0).stop 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 -- 172.21.15.32:0/2698817280 shutdown_connections 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758ced0760 0x55758ced64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55758cec3420 0x55758ced5fb0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55758cf47350 0x55758cf49770 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 --2- 172.21.15.32:0/2698817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758cec2b00 0x55758ced8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 -- 172.21.15.32:0/2698817280 >> 172.21.15.32:0/2698817280 conn(0x55758cebd580 msgr2=0x55758cecde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f1682ff5640 0 client.17277 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 -- 172.21.15.32:0/2698817280 shutdown_connections 2024-04-16T05:42:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 1 -- 172.21.15.32:0/2698817280 wait complete. 2024-04-16T05:42:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a838c740 20 client.17277 trim_cache size 0 max 16384 2024-04-16T05:42:02.626 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:42:02.626 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:42:02.626 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:42:02.626 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:42:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 -- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f16880a3d20 msgr2=0x7f16880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f16880a3d20 0x7f16880a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1698009710 tx=0x7f1698009ad0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 -- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 msgr2=0x7f16a019e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 0x7f16a019e5d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f169000cce0 tx=0x7f169000a620 comp rx=0 tx=0).stop 2024-04-16T05:42:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 -- 172.21.15.32:0/2006317878 shutdown_connections 2024-04-16T05:42:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16a0108070 0x7f16a0198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a006af80 0x7f16a019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f16880a3d20 0x7f16880a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 --2- 172.21.15.32:0/2006317878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16a006a600 0x7f16a019e5d0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.617+0000 7f16a57a8640 1 -- 172.21.15.32:0/2006317878 >> 172.21.15.32:0/2006317878 conn(0x7f16a0076d00 msgr2=0x7f16a00fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.618+0000 7f16a57a8640 1 -- 172.21.15.32:0/2006317878 shutdown_connections 2024-04-16T05:42:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.618+0000 7f16a57a8640 1 -- 172.21.15.32:0/2006317878 wait complete. 2024-04-16T05:42:02.629 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:42:02.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:42:02.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 -- 172.21.15.32:0/656325316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e281011c0 msgr2=0x7f8e281015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/656325316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e281011c0 0x7f8e281015a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009900 tx=0x7f8e2402bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:02.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 -- 172.21.15.32:0/656325316 shutdown_connections 2024-04-16T05:42:02.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/656325316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e28106240 0x7f8e2810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/656325316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e28101b70 0x7f8e28105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/656325316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e281011c0 0x7f8e281015a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 -- 172.21.15.32:0/656325316 >> 172.21.15.32:0/656325316 conn(0x7f8e28078060 msgr2=0x7f8e28078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 -- 172.21.15.32:0/656325316 shutdown_connections 2024-04-16T05:42:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.736+0000 7f8e2f80d640 1 -- 172.21.15.32:0/656325316 wait complete. 2024-04-16T05:42:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2f80d640 1 Processor -- start 2024-04-16T05:42:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2f80d640 1 -- start start 2024-04-16T05:42:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2f80d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e281011c0 0x7f8e2819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2f80d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e28106240 0x7f8e2819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2f80d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e281975a0 0x7f8e28197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2f80d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28111300 con 0x7f8e28106240 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2f80d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28111180 con 0x7f8e281975a0 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2f80d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28111480 con 0x7f8e281011c0 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2cda8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e28106240 0x7f8e2819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2cda8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e28106240 0x7f8e2819d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37610/0 (socket says 172.21.15.32:37610) 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2cda8640 1 -- 172.21.15.32:0/2373681698 learned_addr learned my addr 172.21.15.32:0/2373681698 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.737+0000 7f8e2d5a9640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e281011c0 0x7f8e2819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2ddaa640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e281975a0 0x7f8e28197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2cda8640 1 -- 172.21.15.32:0/2373681698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e281011c0 msgr2=0x7f8e2819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2cda8640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e281011c0 0x7f8e2819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2cda8640 1 -- 172.21.15.32:0/2373681698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e281975a0 msgr2=0x7f8e28197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2cda8640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e281975a0 0x7f8e28197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2cda8640 1 -- 172.21.15.32:0/2373681698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e281980c0 con 0x7f8e28106240 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2d5a9640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e281011c0 0x7f8e2819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2cda8640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e28106240 0x7f8e2819d930 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800c970 tx=0x7f8e1800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e1e7fc640 1 -- 172.21.15.32:0/2373681698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18014070 con 0x7f8e28106240 2024-04-16T05:42:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e1e7fc640 1 -- 172.21.15.32:0/2373681698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e18004440 con 0x7f8e28106240 2024-04-16T05:42:02.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e281983b0 con 0x7f8e28106240 2024-04-16T05:42:02.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e1e7fc640 1 -- 172.21.15.32:0/2373681698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800f450 con 0x7f8e28106240 2024-04-16T05:42:02.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.738+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2806a660 con 0x7f8e28106240 2024-04-16T05:42:02.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.739+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e28069270 con 0x7f8e28106240 2024-04-16T05:42:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.741+0000 7f8e1e7fc640 1 -- 172.21.15.32:0/2373681698 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e18017070 con 0x7f8e28106240 2024-04-16T05:42:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.742+0000 7f8e1e7fc640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e140a3d20 0x7f8e140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.742+0000 7f8e2d5a9640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e140a3d20 0x7f8e140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.742+0000 7f8e1e7fc640 1 -- 172.21.15.32:0/2373681698 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f8e180d0f00 con 0x7f8e28106240 2024-04-16T05:42:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.742+0000 7f8e2d5a9640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e140a3d20 0x7f8e140a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8e24000c00 tx=0x7f8e2402c880 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.745+0000 7f8e1e7fc640 1 -- 172.21.15.32:0/2373681698 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e1801e020 con 0x7f8e28106240 2024-04-16T05:42:02.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.974+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e28198ad0 con 0x7f8e28106240 2024-04-16T05:42:02.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.974+0000 7f8e1e7fc640 1 -- 172.21.15.32:0/2373681698 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f8e1809d200 con 0x7f8e28106240 2024-04-16T05:42:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:42:02.973 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:42:02.973 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:42:02.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e140a3d20 msgr2=0x7f8e140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e140a3d20 0x7f8e140a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8e24000c00 tx=0x7f8e2402c880 comp rx=0 tx=0).stop 2024-04-16T05:42:02.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e28106240 msgr2=0x7f8e2819d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:02.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e28106240 0x7f8e2819d930 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800c970 tx=0x7f8e1800ce30 comp rx=0 tx=0).stop 2024-04-16T05:42:02.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 shutdown_connections 2024-04-16T05:42:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e281975a0 0x7f8e28197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8e140a3d20 0x7f8e140a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e28106240 0x7f8e2819d930 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 --2- 172.21.15.32:0/2373681698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e281011c0 0x7f8e2819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 >> 172.21.15.32:0/2373681698 conn(0x7f8e28078060 msgr2=0x7f8e280ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 shutdown_connections 2024-04-16T05:42:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:02.976+0000 7f8e2f80d640 1 -- 172.21.15.32:0/2373681698 wait complete. 2024-04-16T05:42:02.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:42:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 -- 172.21.15.32:0/289655658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1178108030 msgr2=0x7f1178108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 --2- 172.21.15.32:0/289655658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1178108030 0x7f1178108410 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f116c009930 tx=0x7f116c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 -- 172.21.15.32:0/289655658 shutdown_connections 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 --2- 172.21.15.32:0/289655658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f117806b0a0 0x7f117810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 --2- 172.21.15.32:0/289655658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f117806a700 0x7f117806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 --2- 172.21.15.32:0/289655658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1178108030 0x7f1178108410 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 -- 172.21.15.32:0/289655658 >> 172.21.15.32:0/289655658 conn(0x7f1178077d50 msgr2=0x7f1178078160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 -- 172.21.15.32:0/289655658 shutdown_connections 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.095+0000 7f117ec14640 1 -- 172.21.15.32:0/289655658 wait complete. 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117ec14640 1 Processor -- start 2024-04-16T05:42:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117ec14640 1 -- start start 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117ec14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f117806a700 0x7f117819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117ec14640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f117806b0a0 0x7f117819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117ec14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1178108030 0x7f117819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117ec14640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1178112360 con 0x7f117806b0a0 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117ec14640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f11781121e0 con 0x7f117806a700 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117ec14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11781124e0 con 0x7f1178108030 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f117806b0a0 0x7f117819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f117806b0a0 0x7f117819b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37626/0 (socket says 172.21.15.32:37626) 2024-04-16T05:42:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 -- 172.21.15.32:0/547763862 learned_addr learned my addr 172.21.15.32:0/547763862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117c9b0640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f117806a700 0x7f117819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f117d1b1640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1178108030 0x7f117819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 -- 172.21.15.32:0/547763862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1178108030 msgr2=0x7f117819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1178108030 0x7f117819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 -- 172.21.15.32:0/547763862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f117806a700 msgr2=0x7f117819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f117806a700 0x7f117819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 -- 172.21.15.32:0/547763862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117819fae0 con 0x7f117806b0a0 2024-04-16T05:42:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.096+0000 7f1177fff640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f117806b0a0 0x7f117819b000 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f116400c920 tx=0x7f116400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.097+0000 7f1175ffb640 1 -- 172.21.15.32:0/547763862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164007c70 con 0x7f117806b0a0 2024-04-16T05:42:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.097+0000 7f1175ffb640 1 -- 172.21.15.32:0/547763862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1164007e10 con 0x7f117806b0a0 2024-04-16T05:42:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.097+0000 7f117ec14640 1 -- 172.21.15.32:0/547763862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117819fdd0 con 0x7f117806b0a0 2024-04-16T05:42:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.097+0000 7f1175ffb640 1 -- 172.21.15.32:0/547763862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164018890 con 0x7f117806b0a0 2024-04-16T05:42:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.097+0000 7f117ec14640 1 -- 172.21.15.32:0/547763862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11781a7660 con 0x7f117806b0a0 2024-04-16T05:42:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.099+0000 7f1175ffb640 1 -- 172.21.15.32:0/547763862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1164018a70 con 0x7f117806b0a0 2024-04-16T05:42:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.099+0000 7f1181800740 1 Processor -- start 2024-04-16T05:42:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.099+0000 7f1181800740 1 -- start start 2024-04-16T05:42:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.099+0000 7f1175ffb640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f11680a3d20 0x7f11680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1181800740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1181800740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646da780 0x5612646e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1181800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646e3360 0x5612646e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1181800740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56126463f0e0 con 0x5612646da780 2024-04-16T05:42:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1181800740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5612645dc520 con 0x5612646d9e60 2024-04-16T05:42:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1181800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56126463b0a0 con 0x5612646e3360 2024-04-16T05:42:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f117d1b1640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f11680a3d20 0x7f11680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1175ffb640 1 -- 172.21.15.32:0/547763862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f11640d0c20 con 0x7f117806b0a0 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646da780 0x5612646e2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646da780 0x5612646e2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37636/0 (socket says 172.21.15.32:37636) 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 -- 172.21.15.32:0/525274372 learned_addr learned my addr 172.21.15.32:0/525274372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f117c9b0640 1 --2- 172.21.15.32:0/525274372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 -- 172.21.15.32:0/525274372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646e3360 msgr2=0x5612646e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 --2- 172.21.15.32:0/525274372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646e3360 0x5612646e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 -- 172.21.15.32:0/525274372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 msgr2=0x5612646da240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 --2- 172.21.15.32:0/525274372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646da240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 -- 172.21.15.32:0/525274372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612646048d0 con 0x5612646da780 2024-04-16T05:42:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f117d1b1640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f11680a3d20 0x7f11680a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1170005fd0 tx=0x7f1170010740 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1177fff640 1 --2- 172.21.15.32:0/525274372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646da780 0x5612646e2e20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f11640a4f60 tx=0x7f116401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f979a84dde4e6295 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1162ffd640 1 -- 172.21.15.32:0/525274372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116409cf20 con 0x5612646da780 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1162ffd640 1 -- 172.21.15.32:0/525274372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116401a0c0 con 0x5612646da780 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.100+0000 7f1162ffd640 1 -- 172.21.15.32:0/525274372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116400fb30 con 0x5612646da780 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 -- 172.21.15.32:0/525274372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646da780 msgr2=0x5612646e2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 --2- 172.21.15.32:0/525274372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646da780 0x5612646e2e20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f11640a4f60 tx=0x7f116401c000 comp rx=0 tx=0).stop 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 -- 172.21.15.32:0/525274372 shutdown_connections 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 --2- 172.21.15.32:0/525274372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646e3360 0x5612646e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 --2- 172.21.15.32:0/525274372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646da780 0x5612646e2e20 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 --2- 172.21.15.32:0/525274372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 -- 172.21.15.32:0/525274372 >> 172.21.15.32:0/525274372 conn(0x5612646d4590 msgr2=0x5612646d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 -- 172.21.15.32:0/525274372 shutdown_connections 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 -- 172.21.15.32:0/525274372 wait complete. 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.101+0000 7f1181800740 1 -- start start 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1181800740 1 Processor -- start 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1181800740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646d9e60 0x5612646e0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1181800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 0x5612646e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1181800740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646e3360 0x5612646e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1181800740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5612646e6000 con 0x5612646d9e60 2024-04-16T05:42:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1181800740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5612645f84d0 con 0x5612646e3360 2024-04-16T05:42:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1181800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612646e6180 con 0x5612646da780 2024-04-16T05:42:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1181800740 10 client.0 _ll_register_callbacks cb 0x5612645dd2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117c9b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646d9e60 0x5612646e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117c9b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646d9e60 0x5612646e0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37646/0 (socket says 172.21.15.32:37646) 2024-04-16T05:42:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117c9b0640 1 -- 172.21.15.32:0/3306696447 learned_addr learned my addr 172.21.15.32:0/3306696447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117d1b1640 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646e3360 0x5612646e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f1177fff640 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 0x5612646e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117c9b0640 1 -- 172.21.15.32:0/3306696447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 msgr2=0x5612646e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117c9b0640 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 0x5612646e08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117c9b0640 1 -- 172.21.15.32:0/3306696447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646e3360 msgr2=0x5612646e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117c9b0640 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646e3360 0x5612646e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.102+0000 7f117c9b0640 1 -- 172.21.15.32:0/3306696447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612643513f0 con 0x5612646d9e60 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f117c9b0640 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646d9e60 0x5612646e0390 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f116c009900 tx=0x7f116c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddaa7075cad457f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1157fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1157fff640 1 -- 172.21.15.32:0/3306696447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0043b0 con 0x5612646d9e60 2024-04-16T05:42:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1157fff640 1 -- 172.21.15.32:0/3306696447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f116c0046c0 con 0x5612646d9e60 2024-04-16T05:42:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 1 -- 172.21.15.32:0/3306696447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116401a0c0 con 0x5612646d9e60 2024-04-16T05:42:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1157fff640 1 -- 172.21.15.32:0/3306696447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f116c0048c0 con 0x5612646d9e60 2024-04-16T05:42:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 10 client.17295 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 1 -- 172.21.15.32:0/3306696447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56126441b7d0 con 0x5612646d9e60 2024-04-16T05:42:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1157fff640 1 -- 172.21.15.32:0/3306696447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116401a0c0 con 0x5612646d9e60 2024-04-16T05:42:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1157fff640 1 -- 172.21.15.32:0/3306696447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f116c003680 con 0x5612646d9e60 2024-04-16T05:42:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 10 client.17295 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 10 client.17295 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:42:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 20 client.17295 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5612646f3320 0x5612646f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 4 client.17295 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.103+0000 7f1181800740 1 -- 172.21.15.32:0/3306696447 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56126464cb00 con 0x5612646f3320 2024-04-16T05:42:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.104+0000 7f1177fff640 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5612646f3320 0x5612646f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.104+0000 7f1177fff640 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5612646f3320 0x5612646f5750 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1200328f4dee6019 server_cookie=47e050c0102ae7d3 in_seq=0 out_seq=0 2024-04-16T05:42:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.104+0000 7f1157fff640 10 client.17295 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:42:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1157fff640 1 -- 172.21.15.32:0/3306696447 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56126464cb00 con 0x5612646f3320 2024-04-16T05:42:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1157fff640 10 client.17295 handle_command_reply: tid=0 2024-04-16T05:42:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 client.17295 shutdown 2024-04-16T05:42:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 -- 172.21.15.32:0/3306696447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646d9e60 msgr2=0x5612646e0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646d9e60 0x5612646e0390 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f116c009900 tx=0x7f116c004040 comp rx=0 tx=0).stop 2024-04-16T05:42:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 -- 172.21.15.32:0/3306696447 shutdown_connections 2024-04-16T05:42:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5612646f3320 0x5612646f5750 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646e3360 0x5612646e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 0x5612646e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 --2- 172.21.15.32:0/3306696447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646d9e60 0x5612646e0390 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 -- 172.21.15.32:0/3306696447 >> 172.21.15.32:0/3306696447 conn(0x5612646d4590 msgr2=0x5612646d62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1157fff640 0 client.17295 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:42:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 -- 172.21.15.32:0/3306696447 shutdown_connections 2024-04-16T05:42:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 1 -- 172.21.15.32:0/3306696447 wait complete. 2024-04-16T05:42:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.105+0000 7f1181800740 20 client.17295 trim_cache size 0 max 16384 2024-04-16T05:42:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1181800740 1 Processor -- start 2024-04-16T05:42:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1181800740 1 -- start start 2024-04-16T05:42:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1181800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646d9e60 0x5612646e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1181800740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646da780 0x5612646e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1181800740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1181800740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f11781b3d20 con 0x5612646e3360 2024-04-16T05:42:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1181800740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f11781b15f0 con 0x5612646da780 2024-04-16T05:42:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1181800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11781af440 con 0x5612646d9e60 2024-04-16T05:42:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117d1b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117d1b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646e2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37648/0 (socket says 172.21.15.32:37648) 2024-04-16T05:42:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117d1b1640 1 -- 172.21.15.32:0/1248226202 learned_addr learned my addr 172.21.15.32:0/1248226202 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f1177fff640 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646da780 0x5612646e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117c9b0640 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646d9e60 0x5612646e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117d1b1640 1 -- 172.21.15.32:0/1248226202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646d9e60 msgr2=0x5612646e1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117d1b1640 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646d9e60 0x5612646e1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117d1b1640 1 -- 172.21.15.32:0/1248226202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646da780 msgr2=0x5612646e1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117d1b1640 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646da780 0x5612646e1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117d1b1640 1 -- 172.21.15.32:0/1248226202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612646d5750 con 0x5612646e3360 2024-04-16T05:42:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.119+0000 7f117c9b0640 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646d9e60 0x5612646e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f117d1b1640 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646e2530 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f11700127e0 tx=0x7f1170015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57975a25c4e2e16e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f11627fc640 1 -- 172.21.15.32:0/1248226202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170003b70 con 0x5612646e3360 2024-04-16T05:42:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f11627fc640 1 -- 172.21.15.32:0/1248226202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1170017040 con 0x5612646e3360 2024-04-16T05:42:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f11627fc640 1 -- 172.21.15.32:0/1248226202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170004540 con 0x5612646e3360 2024-04-16T05:42:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 -- 172.21.15.32:0/1248226202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 msgr2=0x5612646e2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646e2530 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f11700127e0 tx=0x7f1170015000 comp rx=0 tx=0).stop 2024-04-16T05:42:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 -- 172.21.15.32:0/1248226202 shutdown_connections 2024-04-16T05:42:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646e2530 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646da780 0x5612646e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 --2- 172.21.15.32:0/1248226202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646d9e60 0x5612646e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 -- 172.21.15.32:0/1248226202 >> 172.21.15.32:0/1248226202 conn(0x5612646d4590 msgr2=0x5612646cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 -- 172.21.15.32:0/1248226202 shutdown_connections 2024-04-16T05:42:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.120+0000 7f1181800740 1 -- 172.21.15.32:0/1248226202 wait complete. 2024-04-16T05:42:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 1 -- start start 2024-04-16T05:42:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 1 Processor -- start 2024-04-16T05:42:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 0x5612646ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646dc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f11781afab0 con 0x5612646e3360 2024-04-16T05:42:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f11781b0f20 con 0x5612646d9e60 2024-04-16T05:42:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11781b0180 con 0x5612646da780 2024-04-16T05:42:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f1181800740 10 client.0 _ll_register_callbacks cb 0x7f116c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f117d1b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f117d1b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646dc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37654/0 (socket says 172.21.15.32:37654) 2024-04-16T05:42:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f117d1b1640 1 -- 172.21.15.32:0/49997690 learned_addr learned my addr 172.21.15.32:0/49997690 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.121+0000 7f117c9b0640 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1177fff640 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 0x5612646ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f117d1b1640 1 -- 172.21.15.32:0/49997690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 msgr2=0x5612646ddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f117d1b1640 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 0x5612646ddac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f117d1b1640 1 -- 172.21.15.32:0/49997690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 msgr2=0x5612646eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f117d1b1640 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f117d1b1640 1 -- 172.21.15.32:0/49997690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612646dcb60 con 0x5612646e3360 2024-04-16T05:42:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f117c9b0640 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:42:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f117d1b1640 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646dc620 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1170009cc0 tx=0x7f1170003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9547debceef5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1157fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1157fff640 1 -- 172.21.15.32:0/49997690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170004440 con 0x5612646e3360 2024-04-16T05:42:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1157fff640 1 -- 172.21.15.32:0/49997690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f11700045e0 con 0x5612646e3360 2024-04-16T05:42:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1181800740 1 -- 172.21.15.32:0/49997690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1170017040 con 0x5612646e3360 2024-04-16T05:42:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1157fff640 1 -- 172.21.15.32:0/49997690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f11700047e0 con 0x5612646e3360 2024-04-16T05:42:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1181800740 10 client.17301 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1181800740 1 -- 172.21.15.32:0/49997690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561264758dc0 con 0x5612646e3360 2024-04-16T05:42:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.122+0000 7f1157fff640 1 -- 172.21.15.32:0/49997690 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1170017040 con 0x5612646e3360 2024-04-16T05:42:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1157fff640 1 -- 172.21.15.32:0/49997690 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1170032020 con 0x5612646e3360 2024-04-16T05:42:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1181800740 10 client.17301 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1181800740 10 client.17301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:42:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1181800740 20 client.17301 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1181800740 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x561264759420 0x56126475b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1181800740 4 client.17301 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1181800740 1 -- 172.21.15.32:0/49997690 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561264646970 con 0x561264759420 2024-04-16T05:42:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f117c9b0640 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x561264759420 0x56126475b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f117c9b0640 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x561264759420 0x56126475b840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=585bc58e80a067ec server_cookie=d4642cc027de99d9 in_seq=0 out_seq=0 2024-04-16T05:42:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1157fff640 10 client.17301 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:42:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1157fff640 1 -- 172.21.15.32:0/49997690 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561264646970 con 0x561264759420 2024-04-16T05:42:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1157fff640 10 client.17301 handle_command_reply: tid=0 2024-04-16T05:42:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.123+0000 7f1181800740 1 client.17301 shutdown 2024-04-16T05:42:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 -- 172.21.15.32:0/49997690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 msgr2=0x5612646dc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646dc620 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1170009cc0 tx=0x7f1170003ba0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 -- 172.21.15.32:0/49997690 shutdown_connections 2024-04-16T05:42:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x561264759420 0x56126475b840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612646e3360 0x5612646dc620 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612646da780 0x5612646ddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1157fff640 0 client.17301 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:42:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 --2- 172.21.15.32:0/49997690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612646d9e60 0x5612646eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 -- 172.21.15.32:0/49997690 >> 172.21.15.32:0/49997690 conn(0x5612646d4590 msgr2=0x5612646deb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 -- 172.21.15.32:0/49997690 shutdown_connections 2024-04-16T05:42:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 1 -- 172.21.15.32:0/49997690 wait complete. 2024-04-16T05:42:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f1181800740 20 client.17301 trim_cache size 0 max 16384 2024-04-16T05:42:03.136 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:42:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 -- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f11680a3d20 msgr2=0x7f11680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f11680a3d20 0x7f11680a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1170005fd0 tx=0x7f1170010740 comp rx=0 tx=0).stop 2024-04-16T05:42:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 -- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f117806b0a0 msgr2=0x7f117819b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f117806b0a0 0x7f117819b000 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f116400c920 tx=0x7f116400cde0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 -- 172.21.15.32:0/547763862 shutdown_connections 2024-04-16T05:42:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1178108030 0x7f117819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f11680a3d20 0x7f11680a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f117806b0a0 0x7f117819b000 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 --2- 172.21.15.32:0/547763862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f117806a700 0x7f117819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 -- 172.21.15.32:0/547763862 >> 172.21.15.32:0/547763862 conn(0x7f1178077d50 msgr2=0x7f117810d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 -- 172.21.15.32:0/547763862 shutdown_connections 2024-04-16T05:42:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.124+0000 7f117ec14640 1 -- 172.21.15.32:0/547763862 wait complete. 2024-04-16T05:42:03.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:42:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 -- 172.21.15.32:0/3983305425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6950106fd0 msgr2=0x7f69501073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 --2- 172.21.15.32:0/3983305425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6950106fd0 0x7f69501073b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6940009900 tx=0x7f694002bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 -- 172.21.15.32:0/3983305425 shutdown_connections 2024-04-16T05:42:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 --2- 172.21.15.32:0/3983305425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f695006b020 0x7f695010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 --2- 172.21.15.32:0/3983305425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695006a660 0x7f695006aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 --2- 172.21.15.32:0/3983305425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6950106fd0 0x7f69501073b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 -- 172.21.15.32:0/3983305425 >> 172.21.15.32:0/3983305425 conn(0x7f6950077cf0 msgr2=0x7f6950078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 -- 172.21.15.32:0/3983305425 shutdown_connections 2024-04-16T05:42:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.249+0000 7f69549cd640 1 -- 172.21.15.32:0/3983305425 wait complete. 2024-04-16T05:42:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f69549cd640 1 Processor -- start 2024-04-16T05:42:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f69549cd640 1 -- start start 2024-04-16T05:42:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f69549cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695006a660 0x7f695019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f69549cd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f695006b020 0x7f695019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f69549cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950106fd0 0x7f6950198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f69549cd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69501123f0 con 0x7f695006b020 2024-04-16T05:42:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f69549cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6950112270 con 0x7f695006a660 2024-04-16T05:42:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f69549cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6950112570 con 0x7f6950106fd0 2024-04-16T05:42:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f695006b020 0x7f695019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f695006b020 0x7f695019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37674/0 (socket says 172.21.15.32:37674) 2024-04-16T05:42:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694dd9b640 1 -- 172.21.15.32:0/3750988695 learned_addr learned my addr 172.21.15.32:0/3750988695 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694ed9d640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950106fd0 0x7f6950198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694e59c640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695006a660 0x7f695019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694dd9b640 1 -- 172.21.15.32:0/3750988695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950106fd0 msgr2=0x7f6950198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694dd9b640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950106fd0 0x7f6950198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694dd9b640 1 -- 172.21.15.32:0/3750988695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695006a660 msgr2=0x7f695019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694dd9b640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695006a660 0x7f695019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694dd9b640 1 -- 172.21.15.32:0/3750988695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6950198ef0 con 0x7f695006b020 2024-04-16T05:42:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694ed9d640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950106fd0 0x7f6950198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.250+0000 7f694e59c640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695006a660 0x7f695019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.251+0000 7f694dd9b640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f695006b020 0x7f695019eab0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f693800b4a0 tx=0x7f693800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.251+0000 7f69477fe640 1 -- 172.21.15.32:0/3750988695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938014070 con 0x7f695006b020 2024-04-16T05:42:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.251+0000 7f69549cd640 1 -- 172.21.15.32:0/3750988695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69501991e0 con 0x7f695006b020 2024-04-16T05:42:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.251+0000 7f69477fe640 1 -- 172.21.15.32:0/3750988695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6938010af0 con 0x7f695006b020 2024-04-16T05:42:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.251+0000 7f69477fe640 1 -- 172.21.15.32:0/3750988695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938017d10 con 0x7f695006b020 2024-04-16T05:42:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.251+0000 7f69549cd640 1 -- 172.21.15.32:0/3750988695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69501a54f0 con 0x7f695006b020 2024-04-16T05:42:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69477fe640 1 -- 172.21.15.32:0/3750988695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6938017070 con 0x7f695006b020 2024-04-16T05:42:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69575bc740 1 Processor -- start 2024-04-16T05:42:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69575bc740 1 -- start start 2024-04-16T05:42:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69575bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652defe60 0x55f652df0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69575bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 0x55f652df8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69575bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652df9360 0x55f652dfb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69575bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f652d550e0 con 0x55f652df9360 2024-04-16T05:42:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69575bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f652cf2520 con 0x55f652defe60 2024-04-16T05:42:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f69575bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f652d510a0 con 0x55f652df0780 2024-04-16T05:42:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.254+0000 7f694dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652df9360 0x55f652dfb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652df9360 0x55f652dfb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37680/0 (socket says 172.21.15.32:37680) 2024-04-16T05:42:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694dd9b640 1 -- 172.21.15.32:0/3474819 learned_addr learned my addr 172.21.15.32:0/3474819 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f69477fe640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f693c0a3d20 0x7f693c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694e59c640 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 0x55f652df8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694ed9d640 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652defe60 0x55f652df0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f69477fe640 1 -- 172.21.15.32:0/3750988695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f69380cfe60 con 0x7f695006b020 2024-04-16T05:42:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694ed9d640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f693c0a3d20 0x7f693c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694dd9b640 1 -- 172.21.15.32:0/3474819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 msgr2=0x55f652df8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694dd9b640 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 0x55f652df8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694dd9b640 1 -- 172.21.15.32:0/3474819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652defe60 msgr2=0x55f652df0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694dd9b640 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652defe60 0x55f652df0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694dd9b640 1 -- 172.21.15.32:0/3474819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f652d1a8d0 con 0x55f652df9360 2024-04-16T05:42:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694e59c640 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 0x55f652df8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:42:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694dd9b640 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652df9360 0x55f652dfb750 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f693809bb60 tx=0x7f6938097dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d676a796b85bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f694ed9d640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f693c0a3d20 0x7f693c0a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6948007310 tx=0x7f69480072a0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f6944ff9640 1 -- 172.21.15.32:0/3474819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693809bbd0 con 0x55f652df9360 2024-04-16T05:42:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f6944ff9640 1 -- 172.21.15.32:0/3474819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f693809ff80 con 0x55f652df9360 2024-04-16T05:42:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.255+0000 7f6944ff9640 1 -- 172.21.15.32:0/3474819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380cd4f0 con 0x55f652df9360 2024-04-16T05:42:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 -- 172.21.15.32:0/3474819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652df9360 msgr2=0x55f652dfb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652df9360 0x55f652dfb750 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f693809bb60 tx=0x7f6938097dd0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 -- 172.21.15.32:0/3474819 shutdown_connections 2024-04-16T05:42:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652df9360 0x55f652dfb750 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 0x55f652df8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 --2- 172.21.15.32:0/3474819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652defe60 0x55f652df0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 -- 172.21.15.32:0/3474819 >> 172.21.15.32:0/3474819 conn(0x55f652dea590 msgr2=0x55f652dea970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 -- 172.21.15.32:0/3474819 shutdown_connections 2024-04-16T05:42:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.256+0000 7f69575bc740 1 -- 172.21.15.32:0/3474819 wait complete. 2024-04-16T05:42:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 1 -- start start 2024-04-16T05:42:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 1 Processor -- start 2024-04-16T05:42:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 0x55f652df69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 0x55f652df6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f652dfbfa0 con 0x55f652defe60 2024-04-16T05:42:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f652d0e4d0 con 0x55f652df0780 2024-04-16T05:42:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f652dfc120 con 0x55f652df9360 2024-04-16T05:42:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f69575bc740 10 client.0 _ll_register_callbacks cb 0x55f652cf32c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37684/0 (socket says 172.21.15.32:37684) 2024-04-16T05:42:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694e59c640 1 -- 172.21.15.32:0/2521169862 learned_addr learned my addr 172.21.15.32:0/2521169862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694dd9b640 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 0x55f652df69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694ed9d640 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 0x55f652df6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694e59c640 1 -- 172.21.15.32:0/2521169862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 msgr2=0x55f652df6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694e59c640 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 0x55f652df6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694e59c640 1 -- 172.21.15.32:0/2521169862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 msgr2=0x55f652df69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694e59c640 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 0x55f652df69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694e59c640 1 -- 172.21.15.32:0/2521169862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f652a673f0 con 0x55f652defe60 2024-04-16T05:42:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.257+0000 7f694ed9d640 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 0x55f652df6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f694e59c640 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df6470 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6940009900 tx=0x7f6940004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756b7a4cf0a881f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f6935ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f6935ffb640 1 -- 172.21.15.32:0/2521169862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940043030 con 0x55f652defe60 2024-04-16T05:42:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f6935ffb640 1 -- 172.21.15.32:0/2521169862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6940005000 con 0x55f652defe60 2024-04-16T05:42:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 1 -- 172.21.15.32:0/2521169862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f693809ff80 con 0x55f652defe60 2024-04-16T05:42:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f6935ffb640 1 -- 172.21.15.32:0/2521169862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6940050050 con 0x55f652defe60 2024-04-16T05:42:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 10 client.17313 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 1 -- 172.21.15.32:0/2521169862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f652b317d0 con 0x55f652defe60 2024-04-16T05:42:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f6935ffb640 1 -- 172.21.15.32:0/2521169862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f693809ff80 con 0x55f652defe60 2024-04-16T05:42:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f6935ffb640 1 -- 172.21.15.32:0/2521169862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69400035f0 con 0x55f652defe60 2024-04-16T05:42:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 10 client.17313 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 10 client.17313 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:42:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 20 client.17313 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55f652e093e0 0x55f652e0b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 4 client.17313 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.258+0000 7f69575bc740 1 -- 172.21.15.32:0/2521169862 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f652d62b00 con 0x55f652e093e0 2024-04-16T05:42:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.259+0000 7f694dd9b640 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55f652e093e0 0x55f652e0b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.259+0000 7f6935ffb640 10 client.17313 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:42:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.259+0000 7f694dd9b640 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55f652e093e0 0x55f652e0b830 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=837f7cd43fbce3b5 server_cookie=a4156efbe5ff7cb2 in_seq=0 out_seq=0 2024-04-16T05:42:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f6935ffb640 1 -- 172.21.15.32:0/2521169862 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f652d62b00 con 0x55f652e093e0 2024-04-16T05:42:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f6935ffb640 10 client.17313 handle_command_reply: tid=0 2024-04-16T05:42:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 client.17313 shutdown 2024-04-16T05:42:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 -- 172.21.15.32:0/2521169862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 msgr2=0x55f652df6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df6470 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6940009900 tx=0x7f6940004610 comp rx=0 tx=0).stop 2024-04-16T05:42:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 -- 172.21.15.32:0/2521169862 shutdown_connections 2024-04-16T05:42:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55f652e093e0 0x55f652e0b830 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 0x55f652df6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 0x55f652df69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f6935ffb640 0 client.17313 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:42:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 --2- 172.21.15.32:0/2521169862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df6470 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 -- 172.21.15.32:0/2521169862 >> 172.21.15.32:0/2521169862 conn(0x55f652dea590 msgr2=0x55f652dec2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 -- 172.21.15.32:0/2521169862 shutdown_connections 2024-04-16T05:42:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 1 -- 172.21.15.32:0/2521169862 wait complete. 2024-04-16T05:42:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.260+0000 7f69575bc740 20 client.17313 trim_cache size 0 max 16384 2024-04-16T05:42:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.273+0000 7f69575bc740 1 Processor -- start 2024-04-16T05:42:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.273+0000 7f69575bc740 1 -- start start 2024-04-16T05:42:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.273+0000 7f69575bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f69575bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 0x55f652df5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f69575bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df9360 0x55f652e11100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f69575bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69501b1b90 con 0x55f652defe60 2024-04-16T05:42:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f69575bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f69501af460 con 0x55f652df9360 2024-04-16T05:42:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f69575bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69501ad2b0 con 0x55f652df0780 2024-04-16T05:42:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37696/0 (socket says 172.21.15.32:37696) 2024-04-16T05:42:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 -- 172.21.15.32:0/754379926 learned_addr learned my addr 172.21.15.32:0/754379926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 -- 172.21.15.32:0/754379926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 msgr2=0x55f652df5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 --2- 172.21.15.32:0/754379926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 0x55f652df5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 -- 172.21.15.32:0/754379926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df9360 msgr2=0x55f652e11100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 --2- 172.21.15.32:0/754379926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df9360 0x55f652e11100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 -- 172.21.15.32:0/754379926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f652de5e60 con 0x55f652defe60 2024-04-16T05:42:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f694e59c640 1 --2- 172.21.15.32:0/754379926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df8ba0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6940004010 tx=0x7f6940005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6b69436debbfe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f6937fff640 1 -- 172.21.15.32:0/754379926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940043040 con 0x55f652defe60 2024-04-16T05:42:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f6937fff640 1 -- 172.21.15.32:0/754379926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69400054e0 con 0x55f652defe60 2024-04-16T05:42:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.274+0000 7f6937fff640 1 -- 172.21.15.32:0/754379926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69400046c0 con 0x55f652defe60 2024-04-16T05:42:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- 172.21.15.32:0/754379926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 msgr2=0x55f652df8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 --2- 172.21.15.32:0/754379926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df8ba0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6940004010 tx=0x7f6940005d50 comp rx=0 tx=0).stop 2024-04-16T05:42:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- 172.21.15.32:0/754379926 shutdown_connections 2024-04-16T05:42:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 --2- 172.21.15.32:0/754379926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df9360 0x55f652e11100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 --2- 172.21.15.32:0/754379926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df0780 0x55f652df5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 --2- 172.21.15.32:0/754379926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652df8ba0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- 172.21.15.32:0/754379926 >> 172.21.15.32:0/754379926 conn(0x55f652dea590 msgr2=0x55f652de9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- 172.21.15.32:0/754379926 shutdown_connections 2024-04-16T05:42:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- 172.21.15.32:0/754379926 wait complete. 2024-04-16T05:42:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- start start 2024-04-16T05:42:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 Processor -- start 2024-04-16T05:42:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652dfc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 0x55f652dfcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 0x55f652e679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69501ad920 con 0x55f652defe60 2024-04-16T05:42:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f69501aed90 con 0x55f652df0780 2024-04-16T05:42:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.275+0000 7f69575bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69501adff0 con 0x55f652df9360 2024-04-16T05:42:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f69575bc740 10 client.0 _ll_register_callbacks cb 0x7f69400044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652dfc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652dfc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37698/0 (socket says 172.21.15.32:37698) 2024-04-16T05:42:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 -- 172.21.15.32:0/2715855382 learned_addr learned my addr 172.21.15.32:0/2715855382 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694dd9b640 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 0x55f652dfcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 -- 172.21.15.32:0/2715855382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 msgr2=0x55f652e679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 0x55f652e679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 -- 172.21.15.32:0/2715855382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 msgr2=0x55f652dfcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 0x55f652dfcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 -- 172.21.15.32:0/2715855382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f652e67f90 con 0x55f652defe60 2024-04-16T05:42:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f694e59c640 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652dfc7a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6940005e00 tx=0x7f6940005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5ab60ea7630f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f6935ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f6935ffb640 1 -- 172.21.15.32:0/2715855382 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940043040 con 0x55f652defe60 2024-04-16T05:42:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f6935ffb640 1 -- 172.21.15.32:0/2715855382 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f6940005000 con 0x55f652defe60 2024-04-16T05:42:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f69575bc740 1 -- 172.21.15.32:0/2715855382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69400054e0 con 0x55f652defe60 2024-04-16T05:42:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f6935ffb640 1 -- 172.21.15.32:0/2715855382 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6940066050 con 0x55f652defe60 2024-04-16T05:42:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f69575bc740 10 client.17319 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f69575bc740 1 -- 172.21.15.32:0/2715855382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f652e6f330 con 0x55f652defe60 2024-04-16T05:42:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.276+0000 7f6935ffb640 1 -- 172.21.15.32:0/2715855382 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69400054e0 con 0x55f652defe60 2024-04-16T05:42:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f6935ffb640 1 -- 172.21.15.32:0/2715855382 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f694005d450 con 0x55f652defe60 2024-04-16T05:42:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f69575bc740 10 client.17319 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f69575bc740 10 client.17319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:42:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f69575bc740 20 client.17319 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f69575bc740 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55f652e6f940 0x55f652e71d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f69575bc740 4 client.17319 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f69575bc740 1 -- 172.21.15.32:0/2715855382 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f652d5c970 con 0x55f652e6f940 2024-04-16T05:42:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f694dd9b640 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55f652e6f940 0x55f652e71d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f694dd9b640 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55f652e6f940 0x55f652e71d60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=505a6c21ff7a00d9 server_cookie=e15ec804a5c3d81c in_seq=0 out_seq=0 2024-04-16T05:42:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f6935ffb640 10 client.17319 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:42:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f6935ffb640 1 -- 172.21.15.32:0/2715855382 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f652d5c970 con 0x55f652e6f940 2024-04-16T05:42:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f6935ffb640 10 client.17319 handle_command_reply: tid=0 2024-04-16T05:42:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.277+0000 7f69575bc740 1 client.17319 shutdown 2024-04-16T05:42:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 -- 172.21.15.32:0/2715855382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 msgr2=0x55f652dfc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652dfc7a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6940005e00 tx=0x7f6940005690 comp rx=0 tx=0).stop 2024-04-16T05:42:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 -- 172.21.15.32:0/2715855382 shutdown_connections 2024-04-16T05:42:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x55f652e6f940 0x55f652e71d60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f652df9360 0x55f652e679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f652df0780 0x55f652dfcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f6935ffb640 0 client.17319 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:42:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 --2- 172.21.15.32:0/2715855382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f652defe60 0x55f652dfc7a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 -- 172.21.15.32:0/2715855382 >> 172.21.15.32:0/2715855382 conn(0x55f652dea590 msgr2=0x55f652de8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 -- 172.21.15.32:0/2715855382 shutdown_connections 2024-04-16T05:42:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 1 -- 172.21.15.32:0/2715855382 wait complete. 2024-04-16T05:42:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69575bc740 20 client.17319 trim_cache size 0 max 16384 2024-04-16T05:42:03.291 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:42:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 -- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f693c0a3d20 msgr2=0x7f693c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f693c0a3d20 0x7f693c0a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6948007310 tx=0x7f69480072a0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 -- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f695006b020 msgr2=0x7f695019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f695006b020 0x7f695019eab0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f693800b4a0 tx=0x7f693800b960 comp rx=0 tx=0).stop 2024-04-16T05:42:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 -- 172.21.15.32:0/3750988695 shutdown_connections 2024-04-16T05:42:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950106fd0 0x7f6950198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f693c0a3d20 0x7f693c0a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f695006b020 0x7f695019eab0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 --2- 172.21.15.32:0/3750988695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695006a660 0x7f695019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 -- 172.21.15.32:0/3750988695 >> 172.21.15.32:0/3750988695 conn(0x7f6950077cf0 msgr2=0x7f69500fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 -- 172.21.15.32:0/3750988695 shutdown_connections 2024-04-16T05:42:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.278+0000 7f69549cd640 1 -- 172.21.15.32:0/3750988695 wait complete. 2024-04-16T05:42:03.294 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:03 smithi032 ceph-mon[32351]: pgmap v216: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:42:03.294 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:03 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2373681698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:03.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:42:03.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 -- 172.21.15.32:0/2468463126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f006a660 msgr2=0x7f24f006aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 --2- 172.21.15.32:0/2468463126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f006a660 0x7f24f006aae0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009930 tx=0x7f24d802bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:03.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 -- 172.21.15.32:0/2468463126 shutdown_connections 2024-04-16T05:42:03.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 --2- 172.21.15.32:0/2468463126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f006b020 0x7f24f010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 --2- 172.21.15.32:0/2468463126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f006a660 0x7f24f006aae0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 --2- 172.21.15.32:0/2468463126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f0108110 0x7f24f01084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 -- 172.21.15.32:0/2468463126 >> 172.21.15.32:0/2468463126 conn(0x7f24f0077cf0 msgr2=0x7f24f0078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 -- 172.21.15.32:0/2468463126 shutdown_connections 2024-04-16T05:42:03.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.404+0000 7f24f510d640 1 -- 172.21.15.32:0/2468463126 wait complete. 2024-04-16T05:42:03.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24f510d640 1 Processor -- start 2024-04-16T05:42:03.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24f510d640 1 -- start start 2024-04-16T05:42:03.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24f510d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f006a660 0x7f24f019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24f510d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f006b020 0x7f24f019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24f510d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f0108110 0x7f24f01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24f510d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01123a0 con 0x7f24f0108110 2024-04-16T05:42:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24f510d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0112220 con 0x7f24f006b020 2024-04-16T05:42:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24f510d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0112520 con 0x7f24f006a660 2024-04-16T05:42:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24ef59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f0108110 0x7f24f01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24ef59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f0108110 0x7f24f01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37716/0 (socket says 172.21.15.32:37716) 2024-04-16T05:42:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24ef59e640 1 -- 172.21.15.32:0/721834447 learned_addr learned my addr 172.21.15.32:0/721834447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24eed9d640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f006a660 0x7f24f019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24e7fff640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f006b020 0x7f24f019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24ef59e640 1 -- 172.21.15.32:0/721834447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f006a660 msgr2=0x7f24f019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.405+0000 7f24ef59e640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f006a660 0x7f24f019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24ef59e640 1 -- 172.21.15.32:0/721834447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f006b020 msgr2=0x7f24f019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24ef59e640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f006b020 0x7f24f019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24ef59e640 1 -- 172.21.15.32:0/721834447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f0198f80 con 0x7f24f0108110 2024-04-16T05:42:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24eed9d640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f006a660 0x7f24f019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24ef59e640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f0108110 0x7f24f01986c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f24e800eaa0 tx=0x7f24e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24ecd99640 1 -- 172.21.15.32:0/721834447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e800cdb0 con 0x7f24f0108110 2024-04-16T05:42:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24ecd99640 1 -- 172.21.15.32:0/721834447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e8005020 con 0x7f24f0108110 2024-04-16T05:42:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24f510d640 1 -- 172.21.15.32:0/721834447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f0199270 con 0x7f24f0108110 2024-04-16T05:42:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24ecd99640 1 -- 172.21.15.32:0/721834447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e80116b0 con 0x7f24f0108110 2024-04-16T05:42:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.406+0000 7f24f510d640 1 -- 172.21.15.32:0/721834447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01a5580 con 0x7f24f0108110 2024-04-16T05:42:03.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.408+0000 7f24ecd99640 1 -- 172.21.15.32:0/721834447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24e8005560 con 0x7f24f0108110 2024-04-16T05:42:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.408+0000 7f24f7dcc740 1 Processor -- start 2024-04-16T05:42:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24f7dcc740 1 -- start start 2024-04-16T05:42:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b211e60 0x558b6b212240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b212780 0x558b6b21ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b21b360 0x558b6b21d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558b6b1770e0 con 0x558b6b211e60 2024-04-16T05:42:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558b6b114520 con 0x558b6b212780 2024-04-16T05:42:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558b6b1730a0 con 0x558b6b21b360 2024-04-16T05:42:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24ecd99640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f24dc0a3d20 0x7f24dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b211e60 0x558b6b212240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b211e60 0x558b6b212240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37730/0 (socket says 172.21.15.32:37730) 2024-04-16T05:42:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 -- 172.21.15.32:0/2530846633 learned_addr learned my addr 172.21.15.32:0/2530846633 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24ef59e640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f24dc0a3d20 0x7f24dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24eed9d640 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b212780 0x558b6b21ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24ecd99640 1 -- 172.21.15.32:0/721834447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f24e80d0e40 con 0x7f24f0108110 2024-04-16T05:42:03.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b21b360 0x558b6b21d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 -- 172.21.15.32:0/2530846633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b21b360 msgr2=0x558b6b21d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b21b360 0x558b6b21d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 -- 172.21.15.32:0/2530846633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b212780 msgr2=0x558b6b21ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b212780 0x558b6b21ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.409+0000 7f24e7fff640 1 -- 172.21.15.32:0/2530846633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b6b13c8d0 con 0x558b6b211e60 2024-04-16T05:42:03.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.410+0000 7f24ef59e640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f24dc0a3d20 0x7f24dc0a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f24e80ce300 tx=0x7f24e8098710 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.410+0000 7f24e7fff640 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b211e60 0x558b6b212240 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f24d8005940 tx=0x7f24d80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd3f51ac9b572f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.410+0000 7f24e57fa640 1 -- 172.21.15.32:0/2530846633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d802cc20 con 0x558b6b211e60 2024-04-16T05:42:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.410+0000 7f24e57fa640 1 -- 172.21.15.32:0/2530846633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d8005040 con 0x558b6b211e60 2024-04-16T05:42:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.410+0000 7f24e57fa640 1 -- 172.21.15.32:0/2530846633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8004920 con 0x558b6b211e60 2024-04-16T05:42:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.410+0000 7f24f7dcc740 1 -- 172.21.15.32:0/2530846633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b211e60 msgr2=0x558b6b212240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.410+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b211e60 0x558b6b212240 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f24d8005940 tx=0x7f24d80058b0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 -- 172.21.15.32:0/2530846633 shutdown_connections 2024-04-16T05:42:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b21b360 0x558b6b21d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b212780 0x558b6b21ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/2530846633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b211e60 0x558b6b212240 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 -- 172.21.15.32:0/2530846633 >> 172.21.15.32:0/2530846633 conn(0x558b6b20c590 msgr2=0x558b6b20c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 -- 172.21.15.32:0/2530846633 shutdown_connections 2024-04-16T05:42:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 -- 172.21.15.32:0/2530846633 wait complete. 2024-04-16T05:42:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 -- start start 2024-04-16T05:42:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 Processor -- start 2024-04-16T05:42:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b218450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b218990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b218ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558b6b21dfa0 con 0x558b6b21b360 2024-04-16T05:42:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558b6b1304d0 con 0x558b6b211e60 2024-04-16T05:42:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558b6b21e120 con 0x558b6b212780 2024-04-16T05:42:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24f7dcc740 10 client.0 _ll_register_callbacks cb 0x558b6b1152c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b218ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b218ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37740/0 (socket says 172.21.15.32:37740) 2024-04-16T05:42:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.411+0000 7f24e7fff640 1 -- 172.21.15.32:0/4290944622 learned_addr learned my addr 172.21.15.32:0/4290944622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24e7fff640 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b218450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24eed9d640 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b218990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24e7fff640 1 -- 172.21.15.32:0/4290944622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 msgr2=0x558b6b218990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24e7fff640 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b218990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24e7fff640 1 -- 172.21.15.32:0/4290944622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 msgr2=0x558b6b218450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24e7fff640 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b218450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24e7fff640 1 -- 172.21.15.32:0/4290944622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b6ae893f0 con 0x558b6b21b360 2024-04-16T05:42:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24e7fff640 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b218ed0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f24d802c170 tx=0x7f24d8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9b63d99305d83a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24d67fc640 1 -- 172.21.15.32:0/4290944622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8053060 con 0x558b6b21b360 2024-04-16T05:42:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24f7dcc740 1 -- 172.21.15.32:0/4290944622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d8005040 con 0x558b6b21b360 2024-04-16T05:42:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24d67fc640 1 -- 172.21.15.32:0/4290944622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f24d80051f0 con 0x558b6b21b360 2024-04-16T05:42:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24d67fc640 1 -- 172.21.15.32:0/4290944622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24d805a050 con 0x558b6b21b360 2024-04-16T05:42:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24f7dcc740 10 client.17331 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24f7dcc740 1 -- 172.21.15.32:0/4290944622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b6af537d0 con 0x558b6b21b360 2024-04-16T05:42:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.412+0000 7f24d67fc640 1 -- 172.21.15.32:0/4290944622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d8005040 con 0x558b6b21b360 2024-04-16T05:42:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24d67fc640 1 -- 172.21.15.32:0/4290944622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24d80035f0 con 0x558b6b21b360 2024-04-16T05:42:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24f7dcc740 10 client.17331 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24f7dcc740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24f7dcc740 20 client.17331 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558b6b22b180 0x558b6b22d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24f7dcc740 4 client.17331 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24f7dcc740 1 -- 172.21.15.32:0/4290944622 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b6b184b00 con 0x558b6b22b180 2024-04-16T05:42:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24eed9d640 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558b6b22b180 0x558b6b22d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24d67fc640 10 client.17331 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.413+0000 7f24eed9d640 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558b6b22b180 0x558b6b22d570 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b918d5cf3699c789 server_cookie=247010dce1065de9 in_seq=0 out_seq=0 2024-04-16T05:42:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24d67fc640 1 -- 172.21.15.32:0/4290944622 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b6b184b00 con 0x558b6b22b180 2024-04-16T05:42:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24d67fc640 10 client.17331 handle_command_reply: tid=0 2024-04-16T05:42:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 client.17331 shutdown 2024-04-16T05:42:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 -- 172.21.15.32:0/4290944622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 msgr2=0x558b6b218ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b218ed0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f24d802c170 tx=0x7f24d8005bd0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 -- 172.21.15.32:0/4290944622 shutdown_connections 2024-04-16T05:42:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b218ed0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558b6b22b180 0x558b6b22d570 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b218990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/4290944622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b218450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24d67fc640 0 client.17331 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 -- 172.21.15.32:0/4290944622 >> 172.21.15.32:0/4290944622 conn(0x558b6b20c590 msgr2=0x558b6b20e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 -- 172.21.15.32:0/4290944622 shutdown_connections 2024-04-16T05:42:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 1 -- 172.21.15.32:0/4290944622 wait complete. 2024-04-16T05:42:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.414+0000 7f24f7dcc740 20 client.17331 trim_cache size 0 max 16384 2024-04-16T05:42:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.427+0000 7f24f7dcc740 1 Processor -- start 2024-04-16T05:42:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24f7dcc740 1 -- start start 2024-04-16T05:42:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b20b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b230590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b230ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01b1b70 con 0x558b6b21b360 2024-04-16T05:42:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01af440 con 0x558b6b211e60 2024-04-16T05:42:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01ad290 con 0x558b6b212780 2024-04-16T05:42:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b230ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b230ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37742/0 (socket says 172.21.15.32:37742) 2024-04-16T05:42:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 -- 172.21.15.32:0/629803760 learned_addr learned my addr 172.21.15.32:0/629803760 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b20b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24eed9d640 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b230590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 -- 172.21.15.32:0/629803760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 msgr2=0x558b6b230590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b230590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 -- 172.21.15.32:0/629803760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 msgr2=0x558b6b20b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b20b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 -- 172.21.15.32:0/629803760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b6b2125b0 con 0x558b6b21b360 2024-04-16T05:42:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24eed9d640 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b230590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.428+0000 7f24e7fff640 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b230ad0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f24d802c820 tx=0x7f24d8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9902ead73906914a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24e4ff9640 1 -- 172.21.15.32:0/629803760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8053040 con 0x558b6b21b360 2024-04-16T05:42:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24e4ff9640 1 -- 172.21.15.32:0/629803760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d802c880 con 0x558b6b21b360 2024-04-16T05:42:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24e4ff9640 1 -- 172.21.15.32:0/629803760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8040930 con 0x558b6b21b360 2024-04-16T05:42:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 -- 172.21.15.32:0/629803760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 msgr2=0x558b6b230ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b230ad0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f24d802c820 tx=0x7f24d8001e70 comp rx=0 tx=0).stop 2024-04-16T05:42:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 -- 172.21.15.32:0/629803760 shutdown_connections 2024-04-16T05:42:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b230ad0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b230590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/629803760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b20b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 -- 172.21.15.32:0/629803760 >> 172.21.15.32:0/629803760 conn(0x558b6b20c590 msgr2=0x558b6b20aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 -- 172.21.15.32:0/629803760 shutdown_connections 2024-04-16T05:42:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.429+0000 7f24f7dcc740 1 -- 172.21.15.32:0/629803760 wait complete. 2024-04-16T05:42:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 1 -- start start 2024-04-16T05:42:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 1 Processor -- start 2024-04-16T05:42:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b230640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b22dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b22e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01ad900 con 0x558b6b21b360 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01aed70 con 0x558b6b211e60 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01adfd0 con 0x558b6b212780 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24f7dcc740 10 client.0 _ll_register_callbacks cb 0x7f24e0009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b22e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b22e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37746/0 (socket says 172.21.15.32:37746) 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 -- 172.21.15.32:0/3710051633 learned_addr learned my addr 172.21.15.32:0/3710051633 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b230640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24eed9d640 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b22dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 -- 172.21.15.32:0/3710051633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 msgr2=0x558b6b22dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b22dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 -- 172.21.15.32:0/3710051633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 msgr2=0x558b6b230640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b230640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 -- 172.21.15.32:0/3710051633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b6b22e550 con 0x558b6b21b360 2024-04-16T05:42:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24e7fff640 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b22e010 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009850 tx=0x7f24d802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49152bcaf5bba1b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.430+0000 7f24d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24d67fc640 1 -- 172.21.15.32:0/3710051633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8053040 con 0x558b6b21b360 2024-04-16T05:42:03.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24d67fc640 1 -- 172.21.15.32:0/3710051633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f24d802ca30 con 0x558b6b21b360 2024-04-16T05:42:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 1 -- 172.21.15.32:0/3710051633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d802c880 con 0x558b6b21b360 2024-04-16T05:42:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24d67fc640 1 -- 172.21.15.32:0/3710051633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24d8003660 con 0x558b6b21b360 2024-04-16T05:42:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 10 client.17337 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 1 -- 172.21.15.32:0/3710051633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b6b291000 con 0x558b6b21b360 2024-04-16T05:42:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24d67fc640 1 -- 172.21.15.32:0/3710051633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d802c880 con 0x558b6b21b360 2024-04-16T05:42:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24d67fc640 1 -- 172.21.15.32:0/3710051633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24d8034440 con 0x558b6b21b360 2024-04-16T05:42:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 10 client.17337 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 20 client.17337 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558b6b291610 0x558b6b293a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 4 client.17337 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24f7dcc740 1 -- 172.21.15.32:0/3710051633 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b6b17e970 con 0x558b6b291610 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.431+0000 7f24eed9d640 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558b6b291610 0x558b6b293a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24d67fc640 10 client.17337 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24eed9d640 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558b6b291610 0x558b6b293a30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc41615bdfec437 server_cookie=e831f3cd2e362091 in_seq=0 out_seq=0 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24d67fc640 1 -- 172.21.15.32:0/3710051633 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b6b17e970 con 0x558b6b291610 2024-04-16T05:42:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24d67fc640 10 client.17337 handle_command_reply: tid=0 2024-04-16T05:42:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 client.17337 shutdown 2024-04-16T05:42:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 -- 172.21.15.32:0/3710051633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 msgr2=0x558b6b22e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b22e010 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009850 tx=0x7f24d802bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 -- 172.21.15.32:0/3710051633 shutdown_connections 2024-04-16T05:42:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b6b21b360 0x558b6b22e010 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558b6b291610 0x558b6b293a30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b6b212780 0x558b6b22dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 --2- 172.21.15.32:0/3710051633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b6b211e60 0x558b6b230640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24d67fc640 0 client.17337 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 -- 172.21.15.32:0/3710051633 >> 172.21.15.32:0/3710051633 conn(0x558b6b20c590 msgr2=0x558b6b218ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 -- 172.21.15.32:0/3710051633 shutdown_connections 2024-04-16T05:42:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 1 -- 172.21.15.32:0/3710051633 wait complete. 2024-04-16T05:42:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.432+0000 7f24f7dcc740 20 client.17337 trim_cache size 0 max 16384 2024-04-16T05:42:03.445 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:42:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 -- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f24dc0a3d20 msgr2=0x7f24dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f24dc0a3d20 0x7f24dc0a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f24e80ce300 tx=0x7f24e8098710 comp rx=0 tx=0).stop 2024-04-16T05:42:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 -- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f0108110 msgr2=0x7f24f01986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f0108110 0x7f24f01986c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f24e800eaa0 tx=0x7f24e800ef60 comp rx=0 tx=0).stop 2024-04-16T05:42:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 -- 172.21.15.32:0/721834447 shutdown_connections 2024-04-16T05:42:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f24dc0a3d20 0x7f24dc0a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24f0108110 0x7f24f01986c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f006b020 0x7f24f019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 --2- 172.21.15.32:0/721834447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f006a660 0x7f24f019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:03.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 -- 172.21.15.32:0/721834447 >> 172.21.15.32:0/721834447 conn(0x7f24f0077cf0 msgr2=0x7f24f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:03.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 -- 172.21.15.32:0/721834447 shutdown_connections 2024-04-16T05:42:03.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:03.433+0000 7f24f510d640 1 -- 172.21.15.32:0/721834447 wait complete. 2024-04-16T05:42:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:03 smithi134 ceph-mon[36153]: pgmap v216: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:42:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:03 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2373681698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:03.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:03 smithi082 ceph-mon[36195]: pgmap v216: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:42:03.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:03 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2373681698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:04.449 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:42:04.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:42:04.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:42:04.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 -- 172.21.15.32:0/3633212255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 msgr2=0x7fca3806ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3633212255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 0x7fca3806ad10 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fca28009ca0 tx=0x7fca2802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 -- 172.21.15.32:0/3633212255 shutdown_connections 2024-04-16T05:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3633212255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca38107320 0x7fca3810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3633212255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca3806b2e0 0x7fca38106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3633212255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 0x7fca3806ad10 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 -- 172.21.15.32:0/3633212255 >> 172.21.15.32:0/3633212255 conn(0x7fca38077f60 msgr2=0x7fca38078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 -- 172.21.15.32:0/3633212255 shutdown_connections 2024-04-16T05:42:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 -- 172.21.15.32:0/3633212255 wait complete. 2024-04-16T05:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.574+0000 7fca3cf67640 1 Processor -- start 2024-04-16T05:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3cf67640 1 -- start start 2024-04-16T05:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3cf67640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 0x7fca3819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3cf67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca38107320 0x7fca3819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3cf67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca381986b0 0x7fca38198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3cf67640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca381123b0 con 0x7fca3806a930 2024-04-16T05:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3cf67640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca38112230 con 0x7fca38107320 2024-04-16T05:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3cf67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca38112530 con 0x7fca381986b0 2024-04-16T05:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 0x7fca3819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 0x7fca3819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37766/0 (socket says 172.21.15.32:37766) 2024-04-16T05:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 -- 172.21.15.32:0/3263739074 learned_addr learned my addr 172.21.15.32:0/3263739074 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca36d9d640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca381986b0 0x7fca38198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca35d9b640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca38107320 0x7fca3819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 -- 172.21.15.32:0/3263739074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca381986b0 msgr2=0x7fca38198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca381986b0 0x7fca38198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 -- 172.21.15.32:0/3263739074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca38107320 msgr2=0x7fca3819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca38107320 0x7fca3819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 -- 172.21.15.32:0/3263739074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca38199290 con 0x7fca3806a930 2024-04-16T05:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.575+0000 7fca3659c640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 0x7fca3819e560 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fca28005e30 tx=0x7fca28032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.576+0000 7fca2f7fe640 1 -- 172.21.15.32:0/3263739074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28039040 con 0x7fca3806a930 2024-04-16T05:42:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.576+0000 7fca2f7fe640 1 -- 172.21.15.32:0/3263739074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca28044070 con 0x7fca3806a930 2024-04-16T05:42:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.576+0000 7fca3cf67640 1 -- 172.21.15.32:0/3263739074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca381a5370 con 0x7fca3806a930 2024-04-16T05:42:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.576+0000 7fca2f7fe640 1 -- 172.21.15.32:0/3263739074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca280042a0 con 0x7fca3806a930 2024-04-16T05:42:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.576+0000 7fca3cf67640 1 -- 172.21.15.32:0/3263739074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca381a5720 con 0x7fca3806a930 2024-04-16T05:42:04.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.578+0000 7fca2f7fe640 1 -- 172.21.15.32:0/3263739074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca28004440 con 0x7fca3806a930 2024-04-16T05:42:04.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3fc2b740 1 Processor -- start 2024-04-16T05:42:04.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3fc2b740 1 -- start start 2024-04-16T05:42:04.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf79e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf82a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf82fb0 0x55e98bf853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e98bedee90 con 0x55e98bf7a3d0 2024-04-16T05:42:04.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e98be7c4c0 con 0x55e98bf79ab0 2024-04-16T05:42:04.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e98bedae50 con 0x55e98bf82fb0 2024-04-16T05:42:04.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca2f7fe640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca240a3d20 0x7fca240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf82a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf82a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37770/0 (socket says 172.21.15.32:37770) 2024-04-16T05:42:04.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca3659c640 1 -- 172.21.15.32:0/1282789486 learned_addr learned my addr 172.21.15.32:0/1282789486 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.579+0000 7fca36d9d640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca240a3d20 0x7fca240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca35d9b640 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf82fb0 0x55e98bf853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca36d9d640 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf79e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca2f7fe640 1 -- 172.21.15.32:0/3263739074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca280f2e30 con 0x7fca3806a930 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca3659c640 1 -- 172.21.15.32:0/1282789486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf82fb0 msgr2=0x55e98bf853a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca3659c640 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf82fb0 0x55e98bf853a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca3659c640 1 -- 172.21.15.32:0/1282789486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 msgr2=0x55e98bf79e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca3659c640 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf79e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca3659c640 1 -- 172.21.15.32:0/1282789486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98bcb97e0 con 0x55e98bf7a3d0 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca36d9d640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca240a3d20 0x7fca240a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca30005fd0 tx=0x7fca3000ba20 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca3659c640 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf82a70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fca2808cea0 tx=0x7fca2808dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51df2deedf3bc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca2cff9640 1 -- 172.21.15.32:0/1282789486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca280be500 con 0x55e98bf7a3d0 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca2cff9640 1 -- 172.21.15.32:0/1282789486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca280d8110 con 0x55e98bf7a3d0 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.580+0000 7fca2cff9640 1 -- 172.21.15.32:0/1282789486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca280c7810 con 0x55e98bf7a3d0 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1282789486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 msgr2=0x55e98bf82a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf82a70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fca2808cea0 tx=0x7fca2808dee0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1282789486 shutdown_connections 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf82fb0 0x55e98bf853a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf82a70 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/1282789486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf79e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1282789486 >> 172.21.15.32:0/1282789486 conn(0x55e98bf741e0 msgr2=0x55e98bf745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1282789486 shutdown_connections 2024-04-16T05:42:04.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1282789486 wait complete. 2024-04-16T05:42:04.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.581+0000 7fca3fc2b740 1 -- start start 2024-04-16T05:42:04.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3fc2b740 1 Processor -- start 2024-04-16T05:42:04.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf80070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf805b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf82fb0 0x55e98bf80af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e98bf85b60 con 0x55e98bf7a3d0 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e98be67410 con 0x55e98bf82fb0 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e98bf85ce0 con 0x55e98bf79ab0 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3fc2b740 10 client.0 _ll_register_callbacks cb 0x55e98be62b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca35d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf805b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca35d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf805b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37784/0 (socket says 172.21.15.32:37784) 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca35d9b640 1 -- 172.21.15.32:0/2933936015 learned_addr learned my addr 172.21.15.32:0/2933936015 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca3659c640 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf80070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca36d9d640 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf82fb0 0x55e98bf80af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca35d9b640 1 -- 172.21.15.32:0/2933936015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 msgr2=0x55e98bf80070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca35d9b640 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf80070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca35d9b640 1 -- 172.21.15.32:0/2933936015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf82fb0 msgr2=0x55e98bf80af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca35d9b640 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf82fb0 0x55e98bf80af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.582+0000 7fca35d9b640 1 -- 172.21.15.32:0/2933936015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98bd19f70 con 0x55e98bf7a3d0 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca35d9b640 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf805b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fca2000be40 tx=0x7fca20012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9ae14c5a110fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:04.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca1dffb640 1 -- 172.21.15.32:0/2933936015 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20004230 con 0x55e98bf7a3d0 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca1dffb640 1 -- 172.21.15.32:0/2933936015 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca200043d0 con 0x55e98bf7a3d0 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 1 -- 172.21.15.32:0/2933936015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca280d8110 con 0x55e98bf7a3d0 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca1dffb640 1 -- 172.21.15.32:0/2933936015 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca200035f0 con 0x55e98bf7a3d0 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 10 client.17349 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 1 -- 172.21.15.32:0/2933936015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e98bbb1170 con 0x55e98bf7a3d0 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca1dffb640 1 -- 172.21.15.32:0/2933936015 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca280d8110 con 0x55e98bf7a3d0 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca1dffb640 1 -- 172.21.15.32:0/2933936015 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca20010b80 con 0x55e98bf7a3d0 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 10 client.17349 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:04.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 10 client.17349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:04.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 20 client.17349 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:04.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e98bf93150 0x55e98bf955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 4 client.17349 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:04.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.583+0000 7fca3fc2b740 1 -- 172.21.15.32:0/2933936015 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e98beec8b0 con 0x55e98bf93150 2024-04-16T05:42:04.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.584+0000 7fca3659c640 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e98bf93150 0x55e98bf955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.584+0000 7fca3659c640 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e98bf93150 0x55e98bf955c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903de0e8dbfc9dda server_cookie=d154344a991ce5f in_seq=0 out_seq=0 2024-04-16T05:42:04.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.584+0000 7fca1dffb640 10 client.17349 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:04.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca1dffb640 1 -- 172.21.15.32:0/2933936015 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e98beec8b0 con 0x55e98bf93150 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca1dffb640 10 client.17349 handle_command_reply: tid=0 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 client.17349 shutdown 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 -- 172.21.15.32:0/2933936015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 msgr2=0x55e98bf805b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf805b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fca2000be40 tx=0x7fca20012820 comp rx=0 tx=0).stop 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 -- 172.21.15.32:0/2933936015 shutdown_connections 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf82fb0 0x55e98bf80af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf7a3d0 0x55e98bf805b0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e98bf93150 0x55e98bf955c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/2933936015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf80070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca1dffb640 0 client.17349 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:04.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 -- 172.21.15.32:0/2933936015 >> 172.21.15.32:0/2933936015 conn(0x55e98bf741e0 msgr2=0x55e98bf75f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 -- 172.21.15.32:0/2933936015 shutdown_connections 2024-04-16T05:42:04.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 1 -- 172.21.15.32:0/2933936015 wait complete. 2024-04-16T05:42:04.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.585+0000 7fca3fc2b740 20 client.17349 trim_cache size 0 max 16384 2024-04-16T05:42:04.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.602+0000 7fca3fc2b740 1 Processor -- start 2024-04-16T05:42:04.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.602+0000 7fca3fc2b740 1 -- start start 2024-04-16T05:42:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf7a3d0 0x55e98bf8e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf9aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca381b1b80 con 0x55e98bf82fb0 2024-04-16T05:42:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca381af450 con 0x55e98bf79ab0 2024-04-16T05:42:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca381ad2a0 con 0x55e98bf7a3d0 2024-04-16T05:42:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf9aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf9aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37794/0 (socket says 172.21.15.32:37794) 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 -- 172.21.15.32:0/1841074173 learned_addr learned my addr 172.21.15.32:0/1841074173 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca35d9b640 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf7a3d0 0x55e98bf8e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca3659c640 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 -- 172.21.15.32:0/1841074173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf7a3d0 msgr2=0x55e98bf8e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf7a3d0 0x55e98bf8e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 -- 172.21.15.32:0/1841074173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 msgr2=0x55e98bf827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf827a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 -- 172.21.15.32:0/1841074173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98bf78e60 con 0x55e98bf82fb0 2024-04-16T05:42:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca3659c640 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca36d9d640 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf9aed0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fca30013fa0 tx=0x7fca300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ba155d56b96cda server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.603+0000 7fca1ffff640 1 -- 172.21.15.32:0/1841074173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3001f860 con 0x55e98bf82fb0 2024-04-16T05:42:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca1ffff640 1 -- 172.21.15.32:0/1841074173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca30029480 con 0x55e98bf82fb0 2024-04-16T05:42:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca1ffff640 1 -- 172.21.15.32:0/1841074173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30011680 con 0x55e98bf82fb0 2024-04-16T05:42:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1841074173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 msgr2=0x55e98bf9aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf9aed0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fca30013fa0 tx=0x7fca300046c0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1841074173 shutdown_connections 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf9aed0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf7a3d0 0x55e98bf8e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/1841074173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf79ab0 0x55e98bf827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1841074173 >> 172.21.15.32:0/1841074173 conn(0x55e98bf741e0 msgr2=0x55e98bf73250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1841074173 shutdown_connections 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 -- 172.21.15.32:0/1841074173 wait complete. 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 -- start start 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.604+0000 7fca3fc2b740 1 Processor -- start 2024-04-16T05:42:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf8b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf7a3d0 0x55e98bf87e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3fc2b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf88560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca381ad910 con 0x55e98bf82fb0 2024-04-16T05:42:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca381aed80 con 0x55e98bf7a3d0 2024-04-16T05:42:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3fc2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca381adfe0 con 0x55e98bf79ab0 2024-04-16T05:42:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3fc2b740 10 client.0 _ll_register_callbacks cb 0x7fca2000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf88560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf88560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37806/0 (socket says 172.21.15.32:37806) 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 -- 172.21.15.32:0/3353001058 learned_addr learned my addr 172.21.15.32:0/3353001058 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca35d9b640 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf7a3d0 0x55e98bf87e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3659c640 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf8b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 -- 172.21.15.32:0/3353001058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 msgr2=0x55e98bf8b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf8b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 -- 172.21.15.32:0/3353001058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf7a3d0 msgr2=0x55e98bf87e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf7a3d0 0x55e98bf87e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 -- 172.21.15.32:0/3353001058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98bff9750 con 0x55e98bf82fb0 2024-04-16T05:42:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca3659c640 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf8b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca36d9d640 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf88560 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fca3000b8e0 tx=0x7fca30005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f766b8c794435013 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.605+0000 7fca1dffb640 1 -- 172.21.15.32:0/3353001058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30036060 con 0x55e98bf82fb0 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 1 -- 172.21.15.32:0/3353001058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca30029480 con 0x55e98bf82fb0 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca1dffb640 1 -- 172.21.15.32:0/3353001058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fca30004730 con 0x55e98bf82fb0 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca1dffb640 1 -- 172.21.15.32:0/3353001058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca3003d050 con 0x55e98bf82fb0 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 10 client.17355 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 1 -- 172.21.15.32:0/3353001058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e98bffa7e0 con 0x55e98bf82fb0 2024-04-16T05:42:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca1dffb640 1 -- 172.21.15.32:0/3353001058 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca30029480 con 0x55e98bf82fb0 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca1dffb640 1 -- 172.21.15.32:0/3353001058 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca30030030 con 0x55e98bf82fb0 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 10 client.17355 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 20 client.17355 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e98bffaeb0 0x55e98bffd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 4 client.17355 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3fc2b740 1 -- 172.21.15.32:0/3353001058 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e98bee6720 con 0x55e98bffaeb0 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.606+0000 7fca3659c640 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e98bffaeb0 0x55e98bffd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.607+0000 7fca3659c640 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e98bffaeb0 0x55e98bffd2a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9f726267a34e43 server_cookie=e06e19fab7226a41 in_seq=0 out_seq=0 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.607+0000 7fca1dffb640 10 client.17355 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca1dffb640 1 -- 172.21.15.32:0/3353001058 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e98bee6720 con 0x55e98bffaeb0 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca1dffb640 10 client.17355 handle_command_reply: tid=0 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 client.17355 shutdown 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 -- 172.21.15.32:0/3353001058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 msgr2=0x55e98bf88560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf88560 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fca3000b8e0 tx=0x7fca30005270 comp rx=0 tx=0).stop 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 -- 172.21.15.32:0/3353001058 shutdown_connections 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e98bf82fb0 0x55e98bf88560 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e98bf7a3d0 0x55e98bf87e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55e98bffaeb0 0x55e98bffd2a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 --2- 172.21.15.32:0/3353001058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98bf79ab0 0x55e98bf8b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca1dffb640 0 client.17355 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 -- 172.21.15.32:0/3353001058 >> 172.21.15.32:0/3353001058 conn(0x55e98bf741e0 msgr2=0x55e98bf80030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 -- 172.21.15.32:0/3353001058 shutdown_connections 2024-04-16T05:42:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 1 -- 172.21.15.32:0/3353001058 wait complete. 2024-04-16T05:42:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.608+0000 7fca3fc2b740 20 client.17355 trim_cache size 0 max 16384 2024-04-16T05:42:04.618 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:42:04.618 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:42:04.618 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "d88226c3-fb9d-4383-80cb-313f2fb16c66", 2024-04-16T05:42:04.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:42:04.618 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:42:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 -- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca240a3d20 msgr2=0x7fca240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca240a3d20 0x7fca240a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca30005fd0 tx=0x7fca3000ba20 comp rx=0 tx=0).stop 2024-04-16T05:42:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 -- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 msgr2=0x7fca3819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 0x7fca3819e560 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fca28005e30 tx=0x7fca28032f20 comp rx=0 tx=0).stop 2024-04-16T05:42:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 -- 172.21.15.32:0/3263739074 shutdown_connections 2024-04-16T05:42:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca381986b0 0x7fca38198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca38107320 0x7fca3819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fca240a3d20 0x7fca240a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 --2- 172.21.15.32:0/3263739074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca3806a930 0x7fca3819e560 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 -- 172.21.15.32:0/3263739074 >> 172.21.15.32:0/3263739074 conn(0x7fca38077f60 msgr2=0x7fca380fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 -- 172.21.15.32:0/3263739074 shutdown_connections 2024-04-16T05:42:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.609+0000 7fca3cf67640 1 -- 172.21.15.32:0/3263739074 wait complete. 2024-04-16T05:42:04.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:42:04.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:42:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 -- 172.21.15.32:0/1990885180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130101140 msgr2=0x7fc130101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 --2- 172.21.15.32:0/1990885180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130101140 0x7fc130101520 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc120009ca0 tx=0x7fc12002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 -- 172.21.15.32:0/1990885180 shutdown_connections 2024-04-16T05:42:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 --2- 172.21.15.32:0/1990885180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc130106210 0x7fc13010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 --2- 172.21.15.32:0/1990885180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc130101af0 0x7fc130105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 --2- 172.21.15.32:0/1990885180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130101140 0x7fc130101520 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 -- 172.21.15.32:0/1990885180 >> 172.21.15.32:0/1990885180 conn(0x7fc130078000 msgr2=0x7fc130078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 -- 172.21.15.32:0/1990885180 shutdown_connections 2024-04-16T05:42:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 -- 172.21.15.32:0/1990885180 wait complete. 2024-04-16T05:42:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 Processor -- start 2024-04-16T05:42:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.731+0000 7fc1352d9640 1 -- start start 2024-04-16T05:42:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc1352d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc130101140 0x7fc130199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc1352d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc130106210 0x7fc130199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc1352d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13019a210 0x7fc1301a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc1352d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301112c0 con 0x7fc13019a210 2024-04-16T05:42:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc1352d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc130111140 con 0x7fc130101140 2024-04-16T05:42:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc1352d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc130111440 con 0x7fc130106210 2024-04-16T05:42:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13019a210 0x7fc1301a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13019a210 0x7fc1301a47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37812/0 (socket says 172.21.15.32:37812) 2024-04-16T05:42:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 -- 172.21.15.32:0/3383485432 learned_addr learned my addr 172.21.15.32:0/3383485432 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12effd640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc130101140 0x7fc130199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc130106210 0x7fc130199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 -- 172.21.15.32:0/3383485432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc130106210 msgr2=0x7fc130199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc130106210 0x7fc130199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 -- 172.21.15.32:0/3383485432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc130101140 msgr2=0x7fc130199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc130101140 0x7fc130199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 -- 172.21.15.32:0/3383485432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1301a4f90 con 0x7fc13019a210 2024-04-16T05:42:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc12f7fe640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13019a210 0x7fc1301a47c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc12800d950 tx=0x7fc12800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc127fff640 1 -- 172.21.15.32:0/3383485432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128017030 con 0x7fc13019a210 2024-04-16T05:42:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc127fff640 1 -- 172.21.15.32:0/3383485432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc128004020 con 0x7fc13019a210 2024-04-16T05:42:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc1352d9640 1 -- 172.21.15.32:0/3383485432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1301a5280 con 0x7fc13019a210 2024-04-16T05:42:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.732+0000 7fc127fff640 1 -- 172.21.15.32:0/3383485432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128003c00 con 0x7fc13019a210 2024-04-16T05:42:04.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.733+0000 7fc1352d9640 1 -- 172.21.15.32:0/3383485432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1301a5780 con 0x7fc13019a210 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.735+0000 7fc127fff640 1 -- 172.21.15.32:0/3383485432 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc12801a070 con 0x7fc13019a210 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc137fa0740 1 Processor -- start 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc137fa0740 1 -- start start 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec15b00 0x5578cec15ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 0x5578cec25b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc137fa0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5578ceb7b180 con 0x5578cec16420 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc137fa0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5578ceb18360 con 0x5578cec15b00 2024-04-16T05:42:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc137fa0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578ceb77140 con 0x5578cec23760 2024-04-16T05:42:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc12effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc12effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec23220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37818/0 (socket says 172.21.15.32:37818) 2024-04-16T05:42:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc12effd640 1 -- 172.21.15.32:0/599100485 learned_addr learned my addr 172.21.15.32:0/599100485 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc127fff640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec15b00 0x5578cec15ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc12f7fe640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.736+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 0x5578cec25b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc12effd640 1 -- 172.21.15.32:0/599100485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 msgr2=0x5578cec25b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc12effd640 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 0x5578cec25b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc12effd640 1 -- 172.21.15.32:0/599100485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec15b00 msgr2=0x5578cec15ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc12effd640 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec15b00 0x5578cec15ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc12effd640 1 -- 172.21.15.32:0/599100485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ce9577d0 con 0x5578cec16420 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc127fff640 1 -- 172.21.15.32:0/3383485432 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fc1280d0410 con 0x7fc13019a210 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec15b00 0x5578cec15ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc12effd640 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec23220 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc12002c700 tx=0x7fc1200099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8886aad8e05cc550 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc12f7fe640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc1280989e0 tx=0x7fc1280982b0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc1257fa640 1 -- 172.21.15.32:0/599100485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120043070 con 0x5578cec16420 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc1257fa640 1 -- 172.21.15.32:0/599100485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120032c90 con 0x5578cec16420 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.737+0000 7fc1257fa640 1 -- 172.21.15.32:0/599100485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120039040 con 0x5578cec16420 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 -- 172.21.15.32:0/599100485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 msgr2=0x5578cec23220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec23220 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc12002c700 tx=0x7fc1200099f0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 -- 172.21.15.32:0/599100485 shutdown_connections 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 0x5578cec25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec23220 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 --2- 172.21.15.32:0/599100485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec15b00 0x5578cec15ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 -- 172.21.15.32:0/599100485 >> 172.21.15.32:0/599100485 conn(0x5578cec10580 msgr2=0x5578cec10960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 -- 172.21.15.32:0/599100485 shutdown_connections 2024-04-16T05:42:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 -- 172.21.15.32:0/599100485 wait complete. 2024-04-16T05:42:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 -- start start 2024-04-16T05:42:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.738+0000 7fc137fa0740 1 Processor -- start 2024-04-16T05:42:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 0x5578cec20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 0x5578cec20ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec23760 0x5578cec211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5578cec26540 con 0x5578cec23760 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5578ceb342c0 con 0x5578cec16420 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578cec266c0 con 0x5578cec15b00 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 10 client.0 _ll_register_callbacks cb 0x5578ceb1b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec23760 0x5578cec211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec23760 0x5578cec211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37830/0 (socket says 172.21.15.32:37830) 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 -- 172.21.15.32:0/257038224 learned_addr learned my addr 172.21.15.32:0/257038224 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 0x5578cec20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12effd640 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 0x5578cec20ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 -- 172.21.15.32:0/257038224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 msgr2=0x5578cec20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 0x5578cec20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 -- 172.21.15.32:0/257038224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 msgr2=0x5578cec20ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 0x5578cec20ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 -- 172.21.15.32:0/257038224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ce88d3f0 con 0x5578cec23760 2024-04-16T05:42:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec23760 0x5578cec211e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc118011430 tx=0x7fc11800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d98eea1b053fbf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc1167fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc1167fc640 1 -- 172.21.15.32:0/257038224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180140d0 con 0x5578cec23760 2024-04-16T05:42:04.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc1167fc640 1 -- 172.21.15.32:0/257038224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fc118014270 con 0x5578cec23760 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 1 -- 172.21.15.32:0/257038224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc120032c90 con 0x5578cec23760 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc1167fc640 1 -- 172.21.15.32:0/257038224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc118005020 con 0x5578cec23760 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 10 client.17367 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.739+0000 7fc137fa0740 1 -- 172.21.15.32:0/257038224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578ce84d0f0 con 0x5578cec23760 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc1167fc640 1 -- 172.21.15.32:0/257038224 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120032c90 con 0x5578cec23760 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc1167fc640 1 -- 172.21.15.32:0/257038224 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc1180234e0 con 0x5578cec23760 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc137fa0740 10 client.17367 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc137fa0740 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:04.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc137fa0740 20 client.17367 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc137fa0740 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5578cec33990 0x5578cec35de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc137fa0740 4 client.17367 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc137fa0740 1 -- 172.21.15.32:0/257038224 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578ceb88ba0 con 0x5578cec33990 2024-04-16T05:42:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.740+0000 7fc12effd640 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5578cec33990 0x5578cec35de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc1167fc640 10 client.17367 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc12effd640 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5578cec33990 0x5578cec35de0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c2136a9c09440a7 server_cookie=7660a9f72eec89b7 in_seq=0 out_seq=0 2024-04-16T05:42:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc1167fc640 1 -- 172.21.15.32:0/257038224 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578ceb88ba0 con 0x5578cec33990 2024-04-16T05:42:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc1167fc640 10 client.17367 handle_command_reply: tid=0 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 client.17367 shutdown 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 -- 172.21.15.32:0/257038224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec23760 msgr2=0x5578cec211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec23760 0x5578cec211e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc118011430 tx=0x7fc11800c830 comp rx=0 tx=0).stop 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 -- 172.21.15.32:0/257038224 shutdown_connections 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec23760 0x5578cec211e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 0x5578cec20ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5578cec33990 0x5578cec35de0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 --2- 172.21.15.32:0/257038224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 0x5578cec20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc137fa0740 1 -- 172.21.15.32:0/257038224 >> 172.21.15.32:0/257038224 conn(0x5578cec10580 msgr2=0x5578cec122d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.741+0000 7fc1167fc640 0 client.17367 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.742+0000 7fc137fa0740 1 -- 172.21.15.32:0/257038224 shutdown_connections 2024-04-16T05:42:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.742+0000 7fc137fa0740 1 -- 172.21.15.32:0/257038224 wait complete. 2024-04-16T05:42:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.742+0000 7fc137fa0740 20 client.17367 trim_cache size 0 max 16384 2024-04-16T05:42:04.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc137fa0740 1 Processor -- start 2024-04-16T05:42:04.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc137fa0740 1 -- start start 2024-04-16T05:42:04.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 0x5578cec22e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec2f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec23760 0x5578cec3b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc137fa0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301afa70 con 0x5578cec16420 2024-04-16T05:42:04.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc137fa0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301ad340 con 0x5578cec23760 2024-04-16T05:42:04.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc137fa0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301ab190 con 0x5578cec15b00 2024-04-16T05:42:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc12effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec2f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc12effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec2f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37834/0 (socket says 172.21.15.32:37834) 2024-04-16T05:42:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.759+0000 7fc12effd640 1 -- 172.21.15.32:0/2805373434 learned_addr learned my addr 172.21.15.32:0/2805373434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 0x5578cec22e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc12effd640 1 -- 172.21.15.32:0/2805373434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 msgr2=0x5578cec22e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc12effd640 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 0x5578cec22e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc12effd640 1 -- 172.21.15.32:0/2805373434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec23760 msgr2=0x5578cec3b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec23760 0x5578cec3b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc12effd640 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec23760 0x5578cec3b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc12effd640 1 -- 172.21.15.32:0/2805373434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578cec16bb0 con 0x5578cec16420 2024-04-16T05:42:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc12effd640 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec2f470 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc120006010 tx=0x7fc12002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32363cb444bea07b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc124ff9640 1 -- 172.21.15.32:0/2805373434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120039040 con 0x5578cec16420 2024-04-16T05:42:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc124ff9640 1 -- 172.21.15.32:0/2805373434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120032c90 con 0x5578cec16420 2024-04-16T05:42:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.760+0000 7fc124ff9640 1 -- 172.21.15.32:0/2805373434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120043040 con 0x5578cec16420 2024-04-16T05:42:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- 172.21.15.32:0/2805373434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 msgr2=0x5578cec2f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec2f470 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc120006010 tx=0x7fc12002e040 comp rx=0 tx=0).stop 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- 172.21.15.32:0/2805373434 shutdown_connections 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec23760 0x5578cec3b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec16420 0x5578cec2f470 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 --2- 172.21.15.32:0/2805373434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec15b00 0x5578cec22e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- 172.21.15.32:0/2805373434 >> 172.21.15.32:0/2805373434 conn(0x5578cec10580 msgr2=0x5578cec12260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- 172.21.15.32:0/2805373434 shutdown_connections 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- 172.21.15.32:0/2805373434 wait complete. 2024-04-16T05:42:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- start start 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 Processor -- start 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec15b00 0x5578cec29670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 0x5578cec29bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 0x5578cec2a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301ab800 con 0x5578cec15b00 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301acc70 con 0x5578cec16420 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301abed0 con 0x5578cec23760 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc137fa0740 10 client.0 _ll_register_callbacks cb 0x7fc11800cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc12e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec15b00 0x5578cec29670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc12e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec15b00 0x5578cec29670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37836/0 (socket says 172.21.15.32:37836) 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.761+0000 7fc12e7fc640 1 -- 172.21.15.32:0/350680305 learned_addr learned my addr 172.21.15.32:0/350680305 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc12e7fc640 1 -- 172.21.15.32:0/350680305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 msgr2=0x5578cec2a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc12effd640 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 0x5578cec29bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 0x5578cec2a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc12e7fc640 1 -- 172.21.15.32:0/350680305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 msgr2=0x5578cec29bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 0x5578cec29bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc12e7fc640 1 -- 172.21.15.32:0/350680305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578cec99250 con 0x5578cec15b00 2024-04-16T05:42:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc12e7fc640 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec15b00 0x5578cec29670 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc118016010 tx=0x7fc11800c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d91fc4c404e7399 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc1167fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc1167fc640 1 -- 172.21.15.32:0/350680305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118031020 con 0x5578cec15b00 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc1167fc640 1 -- 172.21.15.32:0/350680305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7fc118023b20 con 0x5578cec15b00 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc137fa0740 1 -- 172.21.15.32:0/350680305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc120032c90 con 0x5578cec15b00 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc1167fc640 1 -- 172.21.15.32:0/350680305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc118039050 con 0x5578cec15b00 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc137fa0740 10 client.17373 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc137fa0740 1 -- 172.21.15.32:0/350680305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578cec9a500 con 0x5578cec15b00 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.762+0000 7fc1167fc640 1 -- 172.21.15.32:0/350680305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120032c90 con 0x5578cec15b00 2024-04-16T05:42:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc1167fc640 1 -- 172.21.15.32:0/350680305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc1180231f0 con 0x5578cec15b00 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc137fa0740 10 client.17373 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc137fa0740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc137fa0740 20 client.17373 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc137fa0740 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5578cec9ab60 0x5578cec9cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc137fa0740 4 client.17373 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc137fa0740 1 -- 172.21.15.32:0/350680305 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578ceb82a10 con 0x5578cec9ab60 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc12effd640 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5578cec9ab60 0x5578cec9cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc12effd640 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5578cec9ab60 0x5578cec9cfe0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc72a2b3e26896a5 server_cookie=a54c6a6ee290062f in_seq=0 out_seq=0 2024-04-16T05:42:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc1167fc640 10 client.17373 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc1167fc640 1 -- 172.21.15.32:0/350680305 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5578ceb82a10 con 0x5578cec9ab60 2024-04-16T05:42:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc1167fc640 10 client.17373 handle_command_reply: tid=0 2024-04-16T05:42:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.763+0000 7fc137fa0740 1 client.17373 shutdown 2024-04-16T05:42:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 -- 172.21.15.32:0/350680305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec15b00 msgr2=0x5578cec29670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec15b00 0x5578cec29670 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc118016010 tx=0x7fc11800c4c0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 -- 172.21.15.32:0/350680305 shutdown_connections 2024-04-16T05:42:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5578cec9ab60 0x5578cec9cfe0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cec23760 0x5578cec2a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cec16420 0x5578cec29bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 --2- 172.21.15.32:0/350680305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578cec15b00 0x5578cec29670 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1167fc640 0 client.17373 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 -- 172.21.15.32:0/350680305 >> 172.21.15.32:0/350680305 conn(0x5578cec10580 msgr2=0x5578cec20720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 -- 172.21.15.32:0/350680305 shutdown_connections 2024-04-16T05:42:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 1 -- 172.21.15.32:0/350680305 wait complete. 2024-04-16T05:42:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc137fa0740 20 client.17373 trim_cache size 0 max 16384 2024-04-16T05:42:04.774 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:42:04.774 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:42:04.774 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:42:04.774 INFO:teuthology.orchestra.run.smithi032.stdout: "d88226c3-fb9d-4383-80cb-313f2fb16c66": { 2024-04-16T05:42:04.774 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:42:04.774 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "d88226c3-fb9d-4383-80cb-313f2fb16c66", 2024-04-16T05:42:04.774 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:42:04.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:42:04.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:42:04.775 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:42:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 -- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc11c0a3d20 msgr2=0x7fc11c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc1280989e0 tx=0x7fc1280982b0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 -- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13019a210 msgr2=0x7fc1301a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13019a210 0x7fc1301a47c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc12800d950 tx=0x7fc12800de10 comp rx=0 tx=0).stop 2024-04-16T05:42:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 -- 172.21.15.32:0/3383485432 shutdown_connections 2024-04-16T05:42:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13019a210 0x7fc1301a47c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc130106210 0x7fc130199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 --2- 172.21.15.32:0/3383485432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc130101140 0x7fc130199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 -- 172.21.15.32:0/3383485432 >> 172.21.15.32:0/3383485432 conn(0x7fc130078000 msgr2=0x7fc1300ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 -- 172.21.15.32:0/3383485432 shutdown_connections 2024-04-16T05:42:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:04.764+0000 7fc1352d9640 1 -- 172.21.15.32:0/3383485432 wait complete. 2024-04-16T05:42:04.778 INFO:tasks.cephfs.filesystem:scrub status for tag:d88226c3-fb9d-4383-80cb-313f2fb16c66 - {'path': '/', 'tag': 'd88226c3-fb9d-4383-80cb-313f2fb16c66', 'options': 'recursive,force'} 2024-04-16T05:42:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:05 smithi032 ceph-mon[32351]: pgmap v217: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:42:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:05 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:42:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:05 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:05 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:42:05.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:05 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:05.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:05 smithi134 ceph-mon[36153]: pgmap v217: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:42:05.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:05 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:42:05.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:05 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:05.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:05 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:42:05.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:05 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:05.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:05 smithi082 ceph-mon[36195]: pgmap v217: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:42:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:05 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:42:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:05 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:05 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:42:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:05 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:06.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:06 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:42:06.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:06 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:42:06.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:06 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:42:07.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:07 smithi032 ceph-mon[32351]: pgmap v218: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:42:07.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:07 smithi032 ceph-mon[32351]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T05:42:07.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:07 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:42:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:07 smithi134 ceph-mon[36153]: pgmap v218: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:42:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:07 smithi134 ceph-mon[36153]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T05:42:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:07 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:42:07.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:07 smithi082 ceph-mon[36195]: pgmap v218: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:42:07.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:07 smithi082 ceph-mon[36195]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T05:42:07.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:07 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:42:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:08 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:42:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:08 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:42:08.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:08 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:42:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:08 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:42:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:08 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:42:08.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:08 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:42:08.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:08 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:42:08.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:08 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:42:08.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:08 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:42:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:09 smithi134 ceph-mon[36153]: pgmap v220: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-16T05:42:10.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:09 smithi082 ceph-mon[36195]: pgmap v220: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-16T05:42:10.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:09 smithi032 ceph-mon[32351]: pgmap v220: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-16T05:42:12.035 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:11 smithi032 ceph-mon[32351]: pgmap v221: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-16T05:42:12.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:11 smithi134 ceph-mon[36153]: pgmap v221: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-16T05:42:12.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:11 smithi082 ceph-mon[36195]: pgmap v221: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-16T05:42:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:13 smithi134 ceph-mon[36153]: pgmap v222: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:42:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:13 smithi134 ceph-mon[36153]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T05:42:14.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:13 smithi082 ceph-mon[36195]: pgmap v222: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:42:14.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:13 smithi082 ceph-mon[36195]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T05:42:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:13 smithi032 ceph-mon[32351]: pgmap v222: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:42:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:13 smithi032 ceph-mon[32351]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T05:42:16.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:15 smithi134 ceph-mon[36153]: pgmap v224: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:42:16.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:15 smithi082 ceph-mon[36195]: pgmap v224: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:42:16.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:15 smithi032 ceph-mon[32351]: pgmap v224: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-16T05:42:18.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:17 smithi134 ceph-mon[36153]: pgmap v225: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 206 B/s wr, 7 op/s 2024-04-16T05:42:18.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:17 smithi082 ceph-mon[36195]: pgmap v225: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 206 B/s wr, 7 op/s 2024-04-16T05:42:18.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:17 smithi032 ceph-mon[32351]: pgmap v225: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 206 B/s wr, 7 op/s 2024-04-16T05:42:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:19 smithi134 ceph-mon[36153]: pgmap v226: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:42:20.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:19 smithi082 ceph-mon[36195]: pgmap v226: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:42:20.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:19 smithi032 ceph-mon[32351]: pgmap v226: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:42:22.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:21 smithi134 ceph-mon[36153]: pgmap v227: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:42:22.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:21 smithi082 ceph-mon[36195]: pgmap v227: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:42:22.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:21 smithi032 ceph-mon[32351]: pgmap v227: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:42:24.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:23 smithi134 ceph-mon[36153]: pgmap v228: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:42:24.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:23 smithi082 ceph-mon[36195]: pgmap v228: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:42:24.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:23 smithi032 ceph-mon[32351]: pgmap v228: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:42:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:25 smithi134 ceph-mon[36153]: pgmap v229: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:42:26.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:25 smithi082 ceph-mon[36195]: pgmap v229: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:42:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:25 smithi032 ceph-mon[32351]: pgmap v229: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:42:28.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:27 smithi134 ceph-mon[36153]: pgmap v230: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:28.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:27 smithi082 ceph-mon[36195]: pgmap v230: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:28.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:27 smithi032 ceph-mon[32351]: pgmap v230: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:30.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:29 smithi134 ceph-mon[36153]: pgmap v231: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:30.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:29 smithi082 ceph-mon[36195]: pgmap v231: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:30.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:29 smithi032 ceph-mon[32351]: pgmap v231: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:32.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:31 smithi134 ceph-mon[36153]: pgmap v232: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:32.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:31 smithi082 ceph-mon[36195]: pgmap v232: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:32.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:31 smithi032 ceph-mon[32351]: pgmap v232: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:33 smithi134 ceph-mon[36153]: pgmap v233: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:34.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:33 smithi082 ceph-mon[36195]: pgmap v233: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:34.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:33 smithi032 ceph-mon[32351]: pgmap v233: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:34.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:42:34.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:42:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 -- 172.21.15.32:0/2222241802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 msgr2=0x7f44ec103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 --2- 172.21.15.32:0/2222241802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 0x7f44ec103660 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009900 tx=0x7f44dc02bd60 comp rx=0 tx=0).stop 2024-04-16T05:42:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 -- 172.21.15.32:0/2222241802 shutdown_connections 2024-04-16T05:42:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 --2- 172.21.15.32:0/2222241802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44ec10c5c0 0x7f44ec10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 --2- 172.21.15.32:0/2222241802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44ec103ba0 0x7f44ec10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 --2- 172.21.15.32:0/2222241802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 0x7f44ec103660 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 -- 172.21.15.32:0/2222241802 >> 172.21.15.32:0/2222241802 conn(0x7f44ec078000 msgr2=0x7f44ec078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 -- 172.21.15.32:0/2222241802 shutdown_connections 2024-04-16T05:42:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 -- 172.21.15.32:0/2222241802 wait complete. 2024-04-16T05:42:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.900+0000 7f44f1f67640 1 Processor -- start 2024-04-16T05:42:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44f1f67640 1 -- start start 2024-04-16T05:42:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44f1f67640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 0x7f44ec100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44f1f67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44ec103ba0 0x7f44ec101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44f1f67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44ec10c5c0 0x7f44ec101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44f1f67640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1123f0 con 0x7f44ec103280 2024-04-16T05:42:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44f1f67640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec112270 con 0x7f44ec103ba0 2024-04-16T05:42:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44f1f67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec112570 con 0x7f44ec10c5c0 2024-04-16T05:42:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 0x7f44ec100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 0x7f44ec100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39232/0 (socket says 172.21.15.32:39232) 2024-04-16T05:42:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eb7fe640 1 -- 172.21.15.32:0/3538722926 learned_addr learned my addr 172.21.15.32:0/3538722926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44ebfff640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44ec10c5c0 0x7f44ec101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eaffd640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44ec103ba0 0x7f44ec101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eb7fe640 1 -- 172.21.15.32:0/3538722926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44ec10c5c0 msgr2=0x7f44ec101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44ec10c5c0 0x7f44ec101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eb7fe640 1 -- 172.21.15.32:0/3538722926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44ec103ba0 msgr2=0x7f44ec101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44ec103ba0 0x7f44ec101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eb7fe640 1 -- 172.21.15.32:0/3538722926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec1a6df0 con 0x7f44ec103280 2024-04-16T05:42:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44eaffd640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44ec103ba0 0x7f44ec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.901+0000 7f44ebfff640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44ec10c5c0 0x7f44ec101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.902+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 0x7f44ec100ed0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009a20 tx=0x7f44dc02df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.902+0000 7f44e8ff9640 1 -- 172.21.15.32:0/3538722926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc043020 con 0x7f44ec103280 2024-04-16T05:42:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.902+0000 7f44e8ff9640 1 -- 172.21.15.32:0/3538722926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc0054e0 con 0x7f44ec103280 2024-04-16T05:42:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.902+0000 7f44f1f67640 1 -- 172.21.15.32:0/3538722926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec1a7080 con 0x7f44ec103280 2024-04-16T05:42:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.902+0000 7f44e8ff9640 1 -- 172.21.15.32:0/3538722926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc003ce0 con 0x7f44ec103280 2024-04-16T05:42:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.902+0000 7f44f1f67640 1 -- 172.21.15.32:0/3538722926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec1a7620 con 0x7f44ec103280 2024-04-16T05:42:34.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.904+0000 7f44e8ff9640 1 -- 172.21.15.32:0/3538722926 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44dc046020 con 0x7f44ec103280 2024-04-16T05:42:34.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.904+0000 7f44f4c2f740 1 Processor -- start 2024-04-16T05:42:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.904+0000 7f44f4c2f740 1 -- start start 2024-04-16T05:42:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396ee50 0x558ff396f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3977e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff397a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558ff38d4180 con 0x558ff3978350 2024-04-16T05:42:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558ff3871360 con 0x558ff396f770 2024-04-16T05:42:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558ff38d0140 con 0x558ff396ee50 2024-04-16T05:42:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44e8ff9640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44d40a3da0 0x7f44d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff397a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff397a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39236/0 (socket says 172.21.15.32:39236) 2024-04-16T05:42:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 -- 172.21.15.32:0/3050516492 learned_addr learned my addr 172.21.15.32:0/3050516492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3977e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44ebfff640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44d40a3da0 0x7f44d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44ebfff640 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396ee50 0x558ff396f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44e8ff9640 1 -- 172.21.15.32:0/3538722926 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f44dc0f2d20 con 0x7f44ec103280 2024-04-16T05:42:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 -- 172.21.15.32:0/3050516492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396ee50 msgr2=0x558ff396f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396ee50 0x558ff396f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 -- 172.21.15.32:0/3050516492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 msgr2=0x558ff3977e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3977e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 -- 172.21.15.32:0/3050516492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff36b07d0 con 0x558ff3978350 2024-04-16T05:42:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44ebfff640 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396ee50 0x558ff396f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44eaffd640 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff397a740 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f44e000cc60 tx=0x7f44e000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b859ea894d21ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44ebfff640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44d40a3da0 0x7f44d40a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009fd0 tx=0x7f44e4009540 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.905+0000 7f44d1ffb640 1 -- 172.21.15.32:0/3050516492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0014070 con 0x558ff3978350 2024-04-16T05:42:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44d1ffb640 1 -- 172.21.15.32:0/3050516492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e00047a0 con 0x558ff3978350 2024-04-16T05:42:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44d1ffb640 1 -- 172.21.15.32:0/3050516492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0010720 con 0x558ff3978350 2024-04-16T05:42:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 -- 172.21.15.32:0/3050516492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 msgr2=0x558ff397a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff397a740 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f44e000cc60 tx=0x7f44e000e720 comp rx=0 tx=0).stop 2024-04-16T05:42:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 -- 172.21.15.32:0/3050516492 shutdown_connections 2024-04-16T05:42:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff397a740 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3977e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/3050516492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396ee50 0x558ff396f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 -- 172.21.15.32:0/3050516492 >> 172.21.15.32:0/3050516492 conn(0x558ff3969580 msgr2=0x558ff3969960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 -- 172.21.15.32:0/3050516492 shutdown_connections 2024-04-16T05:42:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.906+0000 7f44f4c2f740 1 -- 172.21.15.32:0/3050516492 wait complete. 2024-04-16T05:42:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 1 -- start start 2024-04-16T05:42:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 1 Processor -- start 2024-04-16T05:42:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff3975330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3975870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff3978350 0x558ff3975db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558ff397b140 con 0x558ff396ee50 2024-04-16T05:42:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558ff388d2c0 con 0x558ff396f770 2024-04-16T05:42:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558ff397b2c0 con 0x558ff3978350 2024-04-16T05:42:34.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44f4c2f740 10 client.0 _ll_register_callbacks cb 0x558ff3874090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff3975330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff3975330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39248/0 (socket says 172.21.15.32:39248) 2024-04-16T05:42:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44eb7fe640 1 -- 172.21.15.32:0/1046391390 learned_addr learned my addr 172.21.15.32:0/1046391390 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44eaffd640 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3975870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44eb7fe640 1 -- 172.21.15.32:0/1046391390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff3978350 msgr2=0x558ff3975db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff3978350 0x558ff3975db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44eb7fe640 1 -- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 msgr2=0x558ff3975870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.907+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3975870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44eb7fe640 1 -- 172.21.15.32:0/1046391390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff35e63f0 con 0x558ff396ee50 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44eaffd640 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3975870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff3975330 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0bea00 tx=0x7f44dc0466a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac552c2f66474f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44caffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44caffd640 1 -- 172.21.15.32:0/1046391390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc0be3d0 con 0x558ff396ee50 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44caffd640 1 -- 172.21.15.32:0/1046391390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f44dc03ac80 con 0x558ff396ee50 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1046391390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e00047a0 con 0x558ff396ee50 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44caffd640 1 -- 172.21.15.32:0/1046391390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44dc0044a0 con 0x558ff396ee50 2024-04-16T05:42:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 10 client.17385 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1046391390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558ff35a60f0 con 0x558ff396ee50 2024-04-16T05:42:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44caffd640 1 -- 172.21.15.32:0/1046391390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e00047a0 con 0x558ff396ee50 2024-04-16T05:42:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44caffd640 1 -- 172.21.15.32:0/1046391390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44dc046310 con 0x558ff396ee50 2024-04-16T05:42:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 10 client.17385 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 20 client.17385 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558ff3988600 0x558ff398aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 4 client.17385 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.908+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1046391390 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff38e1ba0 con 0x558ff3988600 2024-04-16T05:42:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.909+0000 7f44eaffd640 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558ff3988600 0x558ff398aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.909+0000 7f44eaffd640 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558ff3988600 0x558ff398aa90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce9a21e897f29af server_cookie=fa0b2b54fe6ad830 in_seq=0 out_seq=0 2024-04-16T05:42:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.909+0000 7f44caffd640 10 client.17385 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44caffd640 1 -- 172.21.15.32:0/1046391390 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ff38e1ba0 con 0x558ff3988600 2024-04-16T05:42:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44caffd640 10 client.17385 handle_command_reply: tid=0 2024-04-16T05:42:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 client.17385 shutdown 2024-04-16T05:42:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1046391390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 msgr2=0x558ff3975330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff3975330 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0bea00 tx=0x7f44dc0466a0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1046391390 shutdown_connections 2024-04-16T05:42:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558ff3988600 0x558ff398aa90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff3978350 0x558ff3975db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396f770 0x558ff3975870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44caffd640 0 client.17385 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1046391390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff3975330 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1046391390 >> 172.21.15.32:0/1046391390 conn(0x558ff3969580 msgr2=0x558ff396b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:34.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1046391390 shutdown_connections 2024-04-16T05:42:34.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.910+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1046391390 wait complete. 2024-04-16T05:42:34.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.911+0000 7f44f4c2f740 20 client.17385 trim_cache size 0 max 16384 2024-04-16T05:42:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44f4c2f740 1 Processor -- start 2024-04-16T05:42:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44f4c2f740 1 -- start start 2024-04-16T05:42:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396ee50 0x558ff3977a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396f770 0x558ff3975bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff39903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1b3e80 con 0x558ff3978350 2024-04-16T05:42:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1b1750 con 0x558ff396ee50 2024-04-16T05:42:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1af5c0 con 0x558ff396f770 2024-04-16T05:42:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.928+0000 7f44ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff39903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff39903a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39260/0 (socket says 172.21.15.32:39260) 2024-04-16T05:42:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44ebfff640 1 -- 172.21.15.32:0/1202997752 learned_addr learned my addr 172.21.15.32:0/1202997752 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44ebfff640 1 -- 172.21.15.32:0/1202997752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396f770 msgr2=0x558ff3975bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44ebfff640 1 --2- 172.21.15.32:0/1202997752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396f770 0x558ff3975bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44ebfff640 1 -- 172.21.15.32:0/1202997752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396ee50 msgr2=0x558ff3977a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44ebfff640 1 --2- 172.21.15.32:0/1202997752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396ee50 0x558ff3977a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44ebfff640 1 -- 172.21.15.32:0/1202997752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff396a150 con 0x558ff3978350 2024-04-16T05:42:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44ebfff640 1 --2- 172.21.15.32:0/1202997752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff39903a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f44e4012f80 tx=0x7f44e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e5efedf43fe4f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44d17fa640 1 -- 172.21.15.32:0/1202997752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4010d60 con 0x558ff3978350 2024-04-16T05:42:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44d17fa640 1 -- 172.21.15.32:0/1202997752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e4005020 con 0x558ff3978350 2024-04-16T05:42:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.929+0000 7f44d17fa640 1 -- 172.21.15.32:0/1202997752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40286e0 con 0x558ff3978350 2024-04-16T05:42:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1202997752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 msgr2=0x558ff39903a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1202997752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff39903a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f44e4012f80 tx=0x7f44e4004ab0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1202997752 shutdown_connections 2024-04-16T05:42:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1202997752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff3978350 0x558ff39903a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1202997752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396f770 0x558ff3975bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1202997752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff396ee50 0x558ff3977a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1202997752 >> 172.21.15.32:0/1202997752 conn(0x558ff3969580 msgr2=0x558ff396a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1202997752 shutdown_connections 2024-04-16T05:42:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1202997752 wait complete. 2024-04-16T05:42:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- start start 2024-04-16T05:42:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 Processor -- start 2024-04-16T05:42:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff397e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396f770 0x558ff397ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff3978350 0x558ff39edc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1afc10 con 0x558ff396ee50 2024-04-16T05:42:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1b1080 con 0x558ff3978350 2024-04-16T05:42:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.930+0000 7f44f4c2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1b02e0 con 0x558ff396f770 2024-04-16T05:42:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44f4c2f740 10 client.0 _ll_register_callbacks cb 0x7f44dc004310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff397e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff397e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39272/0 (socket says 172.21.15.32:39272) 2024-04-16T05:42:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 -- 172.21.15.32:0/1890978312 learned_addr learned my addr 172.21.15.32:0/1890978312 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 -- 172.21.15.32:0/1890978312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396f770 msgr2=0x558ff397ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396f770 0x558ff397ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 -- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff3978350 msgr2=0x558ff39edc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44ebfff640 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff3978350 0x558ff39edc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff3978350 0x558ff39edc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 -- 172.21.15.32:0/1890978312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff39ee210 con 0x558ff396ee50 2024-04-16T05:42:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44ebfff640 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff3978350 0x558ff39edc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44eb7fe640 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff397e710 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0bb440 tx=0x7f44dc0be3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e019e0e78595316d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44caffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44caffd640 1 -- 172.21.15.32:0/1890978312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc03ce60 con 0x558ff396ee50 2024-04-16T05:42:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44caffd640 1 -- 172.21.15.32:0/1890978312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f44dc0becd0 con 0x558ff396ee50 2024-04-16T05:42:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1890978312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e4005020 con 0x558ff396ee50 2024-04-16T05:42:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44caffd640 1 -- 172.21.15.32:0/1890978312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44dc005690 con 0x558ff396ee50 2024-04-16T05:42:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44f4c2f740 10 client.17391 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.931+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1890978312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558ff39ef500 con 0x558ff396ee50 2024-04-16T05:42:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44caffd640 1 -- 172.21.15.32:0/1890978312 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e4005020 con 0x558ff396ee50 2024-04-16T05:42:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44caffd640 1 -- 172.21.15.32:0/1890978312 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44dc0ddba0 con 0x558ff396ee50 2024-04-16T05:42:34.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44f4c2f740 10 client.17391 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44f4c2f740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44f4c2f740 20 client.17391 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558ff39efa40 0x558ff39f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44f4c2f740 4 client.17391 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1890978312 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ff38dba10 con 0x558ff39efa40 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44eaffd640 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558ff39efa40 0x558ff39f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44caffd640 10 client.17391 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44eaffd640 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558ff39efa40 0x558ff39f1e30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7de86ce8585bd5 server_cookie=df393bd3d02c3cbe in_seq=0 out_seq=0 2024-04-16T05:42:34.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44caffd640 1 -- 172.21.15.32:0/1890978312 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558ff38dba10 con 0x558ff39efa40 2024-04-16T05:42:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44caffd640 10 client.17391 handle_command_reply: tid=0 2024-04-16T05:42:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.932+0000 7f44f4c2f740 1 client.17391 shutdown 2024-04-16T05:42:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1890978312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 msgr2=0x558ff397e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff397e710 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0bb440 tx=0x7f44dc0be3d0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1890978312 shutdown_connections 2024-04-16T05:42:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff3978350 0x558ff39edc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558ff39efa40 0x558ff39f1e30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ff396f770 0x558ff397ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 --2- 172.21.15.32:0/1890978312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ff396ee50 0x558ff397e710 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44caffd640 0 client.17391 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1890978312 >> 172.21.15.32:0/1890978312 conn(0x558ff3969580 msgr2=0x558ff396afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1890978312 shutdown_connections 2024-04-16T05:42:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 1 -- 172.21.15.32:0/1890978312 wait complete. 2024-04-16T05:42:34.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f4c2f740 20 client.17391 trim_cache size 0 max 16384 2024-04-16T05:42:34.943 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:42:34.943 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:42:34.943 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:42:34.943 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:42:34.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 -- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44d40a3da0 msgr2=0x7f44d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44d40a3da0 0x7f44d40a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009fd0 tx=0x7f44e4009540 comp rx=0 tx=0).stop 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 -- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 msgr2=0x7f44ec100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 0x7f44ec100ed0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009a20 tx=0x7f44dc02df60 comp rx=0 tx=0).stop 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 -- 172.21.15.32:0/3538722926 shutdown_connections 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44ec10c5c0 0x7f44ec101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44ec103ba0 0x7f44ec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44d40a3da0 0x7f44d40a6260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 --2- 172.21.15.32:0/3538722926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec103280 0x7f44ec100ed0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.933+0000 7f44f1f67640 1 -- 172.21.15.32:0/3538722926 >> 172.21.15.32:0/3538722926 conn(0x7f44ec078000 msgr2=0x7f44ec0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.934+0000 7f44f1f67640 1 -- 172.21.15.32:0/3538722926 shutdown_connections 2024-04-16T05:42:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:34.934+0000 7f44f1f67640 1 -- 172.21.15.32:0/3538722926 wait complete. 2024-04-16T05:42:34.946 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:42:34.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:42:35.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 -- 172.21.15.32:0/1273485794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0106730 msgr2=0x7f10c0106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 --2- 172.21.15.32:0/1273485794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0106730 0x7f10c0106b10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009900 tx=0x7f10b002bcd0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 -- 172.21.15.32:0/1273485794 shutdown_connections 2024-04-16T05:42:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 --2- 172.21.15.32:0/1273485794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10c0101310 0x7f10c010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 --2- 172.21.15.32:0/1273485794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10c0100950 0x7f10c0100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 --2- 172.21.15.32:0/1273485794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0106730 0x7f10c0106b10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 -- 172.21.15.32:0/1273485794 >> 172.21.15.32:0/1273485794 conn(0x7f10c00761e0 msgr2=0x7f10c00765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 -- 172.21.15.32:0/1273485794 shutdown_connections 2024-04-16T05:42:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 -- 172.21.15.32:0/1273485794 wait complete. 2024-04-16T05:42:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 Processor -- start 2024-04-16T05:42:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.054+0000 7f10c680d640 1 -- start start 2024-04-16T05:42:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10c680d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0100950 0x7f10c019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10c680d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10c0101310 0x7f10c019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10c680d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10c0106730 0x7f10c01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10c680d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10c01101a0 con 0x7f10c0100950 2024-04-16T05:42:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10c680d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0110020 con 0x7f10c0101310 2024-04-16T05:42:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10c680d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0110320 con 0x7f10c0106730 2024-04-16T05:42:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0100950 0x7f10c019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0100950 0x7f10c019e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39298/0 (socket says 172.21.15.32:39298) 2024-04-16T05:42:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 -- 172.21.15.32:0/375717444 learned_addr learned my addr 172.21.15.32:0/375717444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bf7fe640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10c0101310 0x7f10c019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 -- 172.21.15.32:0/375717444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10c0106730 msgr2=0x7f10c01986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10c0106730 0x7f10c01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 -- 172.21.15.32:0/375717444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10c0101310 msgr2=0x7f10c019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10c0101310 0x7f10c019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 -- 172.21.15.32:0/375717444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c0198f60 con 0x7f10c0100950 2024-04-16T05:42:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bf7fe640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10c0101310 0x7f10c019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bffff640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0100950 0x7f10c019e5e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f10b0005940 tx=0x7f10b002cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bd7fa640 1 -- 172.21.15.32:0/375717444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0041070 con 0x7f10c0100950 2024-04-16T05:42:35.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bd7fa640 1 -- 172.21.15.32:0/375717444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b0004450 con 0x7f10c0100950 2024-04-16T05:42:35.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c01991f0 con 0x7f10c0100950 2024-04-16T05:42:35.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10bd7fa640 1 -- 172.21.15.32:0/375717444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0034440 con 0x7f10c0100950 2024-04-16T05:42:35.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.055+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01a5380 con 0x7f10c0100950 2024-04-16T05:42:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.057+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c0101dc0 con 0x7f10c0100950 2024-04-16T05:42:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.058+0000 7f10bd7fa640 1 -- 172.21.15.32:0/375717444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10b0044070 con 0x7f10c0100950 2024-04-16T05:42:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.059+0000 7f10bd7fa640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f10a80a3d20 0x7f10a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.059+0000 7f10bf7fe640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f10a80a3d20 0x7f10a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.059+0000 7f10bd7fa640 1 -- 172.21.15.32:0/375717444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f10b00f2920 con 0x7f10c0100950 2024-04-16T05:42:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.059+0000 7f10bf7fe640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f10a80a3d20 0x7f10a80a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005d00 tx=0x7f10b4005c70 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.063+0000 7f10bd7fa640 1 -- 172.21.15.32:0/375717444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10b00bed40 con 0x7f10c0100950 2024-04-16T05:42:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.289+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10c0069270 con 0x7f10c0100950 2024-04-16T05:42:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.289+0000 7f10bd7fa640 1 -- 172.21.15.32:0/375717444 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f10b00ba030 con 0x7f10c0100950 2024-04-16T05:42:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:42:35.288 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:42:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:42:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f10a80a3d20 msgr2=0x7f10a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f10a80a3d20 0x7f10a80a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005d00 tx=0x7f10b4005c70 comp rx=0 tx=0).stop 2024-04-16T05:42:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0100950 msgr2=0x7f10c019e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0100950 0x7f10c019e5e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f10b0005940 tx=0x7f10b002cd10 comp rx=0 tx=0).stop 2024-04-16T05:42:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 shutdown_connections 2024-04-16T05:42:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10c0106730 0x7f10c01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10c0101310 0x7f10c019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f10a80a3d20 0x7f10a80a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.291+0000 7f10c680d640 1 --2- 172.21.15.32:0/375717444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c0100950 0x7f10c019e5e0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.292+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 >> 172.21.15.32:0/375717444 conn(0x7f10c00761e0 msgr2=0x7f10c00ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.292+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 shutdown_connections 2024-04-16T05:42:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.292+0000 7f10c680d640 1 -- 172.21.15.32:0/375717444 wait complete. 2024-04-16T05:42:35.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:42:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.410+0000 7f4e7ff75640 1 -- 172.21.15.32:0/1478893526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 msgr2=0x7f4e7810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.410+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/1478893526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 0x7f4e7810ef00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009ca0 tx=0x7f4e7002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.410+0000 7f4e7ff75640 1 -- 172.21.15.32:0/1478893526 shutdown_connections 2024-04-16T05:42:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.410+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/1478893526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 0x7f4e7810ef00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.410+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/1478893526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e78101120 0x7f4e78106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.410+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/1478893526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78100770 0x7f4e78100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.410+0000 7f4e7ff75640 1 -- 172.21.15.32:0/1478893526 >> 172.21.15.32:0/1478893526 conn(0x7f4e78077450 msgr2=0x7f4e780fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.410+0000 7f4e7ff75640 1 -- 172.21.15.32:0/1478893526 shutdown_connections 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 -- 172.21.15.32:0/1478893526 wait complete. 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 Processor -- start 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 -- start start 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e78100770 0x7f4e7819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 0x7f4e7819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78198690 0x7f4e78198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781123d0 con 0x7f4e78107330 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e78112250 con 0x7f4e78100770 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7ff75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e78112550 con 0x7f4e78198690 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7d510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 0x7f4e7819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7d510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 0x7f4e7819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39328/0 (socket says 172.21.15.32:39328) 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7d510640 1 -- 172.21.15.32:0/3017066027 learned_addr learned my addr 172.21.15.32:0/3017066027 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.411+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e78100770 0x7f4e7819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7e512640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78198690 0x7f4e78198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7d510640 1 -- 172.21.15.32:0/3017066027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78198690 msgr2=0x7f4e78198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7d510640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78198690 0x7f4e78198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7d510640 1 -- 172.21.15.32:0/3017066027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e78100770 msgr2=0x7f4e7819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7d510640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e78100770 0x7f4e7819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7d510640 1 -- 172.21.15.32:0/3017066027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e78199290 con 0x7f4e78107330 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7e512640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78198690 0x7f4e78198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e78100770 0x7f4e7819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7d510640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 0x7f4e7819eb10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800d770 tx=0x7f4e6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e6effd640 1 -- 172.21.15.32:0/3017066027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68017070 con 0x7f4e78107330 2024-04-16T05:42:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e6effd640 1 -- 172.21.15.32:0/3017066027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e68009c30 con 0x7f4e78107330 2024-04-16T05:42:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e6effd640 1 -- 172.21.15.32:0/3017066027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800f620 con 0x7f4e78107330 2024-04-16T05:42:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7ff75640 1 -- 172.21.15.32:0/3017066027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e781a5370 con 0x7f4e78107330 2024-04-16T05:42:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.412+0000 7f4e7ff75640 1 -- 172.21.15.32:0/3017066027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e781a5720 con 0x7f4e78107330 2024-04-16T05:42:35.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.414+0000 7f4e6effd640 1 -- 172.21.15.32:0/3017066027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e6801a070 con 0x7f4e78107330 2024-04-16T05:42:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.414+0000 7f4e82c39740 1 Processor -- start 2024-04-16T05:42:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e82c39740 1 -- start start 2024-04-16T05:42:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 0x564b8d0e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e82c39740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564b8d0440e0 con 0x564b8d0e8360 2024-04-16T05:42:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e82c39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564b8cfe1520 con 0x564b8d0df780 2024-04-16T05:42:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e82c39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564b8d0400a0 con 0x564b8d0dee60 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7d510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7e512640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7d510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0ea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39332/0 (socket says 172.21.15.32:39332) 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7e512640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0df240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:54276/0 (socket says 172.21.15.32:54276) 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7d510640 1 -- 172.21.15.32:0/113222207 learned_addr learned my addr 172.21.15.32:0/113222207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/113222207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 0x564b8d0e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e6effd640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e640a3d20 0x7f4e640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7d510640 1 -- 172.21.15.32:0/113222207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 msgr2=0x564b8d0df240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7d510640 1 --2- 172.21.15.32:0/113222207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0df240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7d510640 1 -- 172.21.15.32:0/113222207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 msgr2=0x564b8d0e7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7d510640 1 --2- 172.21.15.32:0/113222207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 0x564b8d0e7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7e512640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e640a3d20 0x7f4e640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.415+0000 7f4e7d510640 1 -- 172.21.15.32:0/113222207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b8d0098d0 con 0x564b8d0e8360 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.416+0000 7f4e6effd640 1 -- 172.21.15.32:0/3017066027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f4e680d0750 con 0x7f4e78107330 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.416+0000 7f4e7d510640 1 --2- 172.21.15.32:0/113222207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0ea750 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4e68098760 tx=0x7f4e68023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ce837e66fd1aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.416+0000 7f4e63fff640 1 -- 172.21.15.32:0/113222207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68021030 con 0x564b8d0e8360 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.416+0000 7f4e7e512640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e640a3d20 0x7f4e640a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009430 tx=0x7f4e700095c0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.416+0000 7f4e63fff640 1 -- 172.21.15.32:0/113222207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e68009de0 con 0x564b8d0e8360 2024-04-16T05:42:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.416+0000 7f4e63fff640 1 -- 172.21.15.32:0/113222207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680a4f70 con 0x564b8d0e8360 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 -- 172.21.15.32:0/113222207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 msgr2=0x564b8d0ea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 --2- 172.21.15.32:0/113222207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0ea750 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4e68098760 tx=0x7f4e68023000 comp rx=0 tx=0).stop 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 -- 172.21.15.32:0/113222207 shutdown_connections 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 --2- 172.21.15.32:0/113222207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0ea750 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 --2- 172.21.15.32:0/113222207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 0x564b8d0e7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 --2- 172.21.15.32:0/113222207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 -- 172.21.15.32:0/113222207 >> 172.21.15.32:0/113222207 conn(0x564b8d0d9590 msgr2=0x564b8d0d9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 -- 172.21.15.32:0/113222207 shutdown_connections 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 -- 172.21.15.32:0/113222207 wait complete. 2024-04-16T05:42:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 -- start start 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 Processor -- start 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 0x564b8d0e5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.417+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e82c39740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564b8d0eafa0 con 0x564b8d0e8360 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e82c39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564b8cffd4d0 con 0x564b8d0df780 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e82c39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564b8d0eb120 con 0x564b8d0dee60 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e82c39740 10 client.0 _ll_register_callbacks cb 0x564b8cfe22c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0e5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39338/0 (socket says 172.21.15.32:39338) 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 -- 172.21.15.32:0/1430730528 learned_addr learned my addr 172.21.15.32:0/1430730528 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7d510640 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 0x564b8d0e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 -- 172.21.15.32:0/1430730528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 msgr2=0x564b8d0e5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 -- 172.21.15.32:0/1430730528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 msgr2=0x564b8d0e5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 0x564b8d0e5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 -- 172.21.15.32:0/1430730528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b8cd563f0 con 0x564b8d0e8360 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e7e512640 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0e5d80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4e70036000 tx=0x7f4e70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b137b2ecbc1d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e617fa640 1 -- 172.21.15.32:0/1430730528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70050020 con 0x564b8d0e8360 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e617fa640 1 -- 172.21.15.32:0/1430730528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f4e70004240 con 0x564b8d0e8360 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.418+0000 7f4e82c39740 1 -- 172.21.15.32:0/1430730528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e68009de0 con 0x564b8d0e8360 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e617fa640 1 -- 172.21.15.32:0/1430730528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e70039040 con 0x564b8d0e8360 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e82c39740 10 client.17409 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e82c39740 1 -- 172.21.15.32:0/1430730528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564b8ce207d0 con 0x564b8d0e8360 2024-04-16T05:42:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e617fa640 1 -- 172.21.15.32:0/1430730528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e68009de0 con 0x564b8d0e8360 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e617fa640 1 -- 172.21.15.32:0/1430730528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e7003f040 con 0x564b8d0e8360 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e82c39740 10 client.17409 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e82c39740 10 client.17409 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e82c39740 20 client.17409 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e82c39740 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x564b8d0f8200 0x564b8d0fa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e82c39740 4 client.17409 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e82c39740 1 -- 172.21.15.32:0/1430730528 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b8d051b00 con 0x564b8d0f8200 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.419+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x564b8d0f8200 0x564b8d0fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.420+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x564b8d0f8200 0x564b8d0fa630 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34e8ddc1f4cb19a6 server_cookie=d31f73fe1f787b0 in_seq=0 out_seq=0 2024-04-16T05:42:35.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.420+0000 7f4e617fa640 10 client.17409 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e617fa640 1 -- 172.21.15.32:0/1430730528 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b8d051b00 con 0x564b8d0f8200 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e617fa640 10 client.17409 handle_command_reply: tid=0 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 client.17409 shutdown 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 -- 172.21.15.32:0/1430730528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 msgr2=0x564b8d0e5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0e5d80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4e70036000 tx=0x7f4e70004a40 comp rx=0 tx=0).stop 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 -- 172.21.15.32:0/1430730528 shutdown_connections 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x564b8d0f8200 0x564b8d0fa630 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0e8360 0x564b8d0e5d80 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0df780 0x564b8d0e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 --2- 172.21.15.32:0/1430730528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e617fa640 0 client.17409 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:42:35.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 -- 172.21.15.32:0/1430730528 >> 172.21.15.32:0/1430730528 conn(0x564b8d0d9590 msgr2=0x564b8d0db2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 -- 172.21.15.32:0/1430730528 shutdown_connections 2024-04-16T05:42:35.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 1 -- 172.21.15.32:0/1430730528 wait complete. 2024-04-16T05:42:35.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.421+0000 7f4e82c39740 20 client.17409 trim_cache size 0 max 16384 2024-04-16T05:42:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e82c39740 1 Processor -- start 2024-04-16T05:42:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e82c39740 1 -- start start 2024-04-16T05:42:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0e8360 0x564b8d0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e82c39740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781b1da0 con 0x564b8d0df780 2024-04-16T05:42:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e82c39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781af670 con 0x564b8d0e8360 2024-04-16T05:42:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e82c39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781ad4e0 con 0x564b8d0dee60 2024-04-16T05:42:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.434+0000 7f4e7d510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7d510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39348/0 (socket says 172.21.15.32:39348) 2024-04-16T05:42:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7d510640 1 -- 172.21.15.32:0/2158726795 learned_addr learned my addr 172.21.15.32:0/2158726795 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7d510640 1 -- 172.21.15.32:0/2158726795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 msgr2=0x564b8d0e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7d510640 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7d510640 1 -- 172.21.15.32:0/2158726795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0e8360 msgr2=0x564b8d0fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7d510640 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0e8360 0x564b8d0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7d510640 1 -- 172.21.15.32:0/2158726795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b8d0d3b40 con 0x564b8d0df780 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e7d510640 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0f3f50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4e68098cd0 tx=0x7f4e68005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0187b752e43f4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e637fe640 1 -- 172.21.15.32:0/2158726795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680b5070 con 0x564b8d0df780 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e637fe640 1 -- 172.21.15.32:0/2158726795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6809c300 con 0x564b8d0df780 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.435+0000 7f4e637fe640 1 -- 172.21.15.32:0/2158726795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68021040 con 0x564b8d0df780 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- 172.21.15.32:0/2158726795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 msgr2=0x564b8d0f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0f3f50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4e68098cd0 tx=0x7f4e68005b90 comp rx=0 tx=0).stop 2024-04-16T05:42:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- 172.21.15.32:0/2158726795 shutdown_connections 2024-04-16T05:42:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0e8360 0x564b8d0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0f3f50 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2158726795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0dee60 0x564b8d0e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- 172.21.15.32:0/2158726795 >> 172.21.15.32:0/2158726795 conn(0x564b8d0d9590 msgr2=0x564b8d0d8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- 172.21.15.32:0/2158726795 shutdown_connections 2024-04-16T05:42:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- 172.21.15.32:0/2158726795 wait complete. 2024-04-16T05:42:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- start start 2024-04-16T05:42:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 Processor -- start 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0dee60 0x564b8d0ec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0e8360 0x564b8d15d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781adb30 con 0x564b8d0df780 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781aefa0 con 0x564b8d0dee60 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781ae200 con 0x564b8d0e8360 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e82c39740 10 client.0 _ll_register_callbacks cb 0x7f4e70009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.436+0000 7f4e7d510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7d510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0ecbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39356/0 (socket says 172.21.15.32:39356) 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7d510640 1 -- 172.21.15.32:0/2577432378 learned_addr learned my addr 172.21.15.32:0/2577432378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0dee60 0x564b8d0ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7d510640 1 -- 172.21.15.32:0/2577432378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0e8360 msgr2=0x564b8d15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7e512640 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0e8360 0x564b8d15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7d510640 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0e8360 0x564b8d15d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7d510640 1 -- 172.21.15.32:0/2577432378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0dee60 msgr2=0x564b8d0ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7d510640 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0dee60 0x564b8d0ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7d510640 1 -- 172.21.15.32:0/2577432378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b8d15dd60 con 0x564b8d0df780 2024-04-16T05:42:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0dee60 0x564b8d0ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e7d510640 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0ecbc0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005860 tx=0x7f4e680986b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b2db2315d8a73d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e617fa640 1 -- 172.21.15.32:0/2577432378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68021070 con 0x564b8d0df780 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e617fa640 1 -- 172.21.15.32:0/2577432378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f4e680cdd30 con 0x564b8d0df780 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e82c39740 1 -- 172.21.15.32:0/2577432378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b8d04b970 con 0x564b8d0df780 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e617fa640 1 -- 172.21.15.32:0/2577432378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e680b5070 con 0x564b8d0df780 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e82c39740 10 client.17415 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e82c39740 1 -- 172.21.15.32:0/2577432378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564b8d15efb0 con 0x564b8d0df780 2024-04-16T05:42:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.437+0000 7f4e617fa640 1 -- 172.21.15.32:0/2577432378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564b8d04b970 con 0x564b8d0df780 2024-04-16T05:42:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e617fa640 1 -- 172.21.15.32:0/2577432378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e6809c560 con 0x564b8d0df780 2024-04-16T05:42:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e82c39740 10 client.17415 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e82c39740 10 client.17415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:42:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e82c39740 20 client.17415 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x564b8d15f250 0x564b8d161670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e82c39740 4 client.17415 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e82c39740 1 -- 172.21.15.32:0/2577432378 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b8cfc76b0 con 0x564b8d15f250 2024-04-16T05:42:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x564b8d15f250 0x564b8d161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e7dd11640 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x564b8d15f250 0x564b8d161670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcc290f6606f7db0 server_cookie=dd938f3d0ef7540d in_seq=0 out_seq=0 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e617fa640 10 client.17415 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e617fa640 1 -- 172.21.15.32:0/2577432378 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b8cfc76b0 con 0x564b8d15f250 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.438+0000 7f4e617fa640 10 client.17415 handle_command_reply: tid=0 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 client.17415 shutdown 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 -- 172.21.15.32:0/2577432378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 msgr2=0x564b8d0ecbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0ecbc0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005860 tx=0x7f4e680986b0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 -- 172.21.15.32:0/2577432378 shutdown_connections 2024-04-16T05:42:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x564b8d15f250 0x564b8d161670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b8d0e8360 0x564b8d15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e617fa640 0 client.17415 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:42:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564b8d0df780 0x564b8d0ecbc0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 --2- 172.21.15.32:0/2577432378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b8d0dee60 0x564b8d0ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 -- 172.21.15.32:0/2577432378 >> 172.21.15.32:0/2577432378 conn(0x564b8d0d9590 msgr2=0x564b8d0e52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 -- 172.21.15.32:0/2577432378 shutdown_connections 2024-04-16T05:42:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 1 -- 172.21.15.32:0/2577432378 wait complete. 2024-04-16T05:42:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e82c39740 20 client.17415 trim_cache size 0 max 16384 2024-04-16T05:42:35.449 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:42:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 -- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e640a3d20 msgr2=0x7f4e640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e640a3d20 0x7f4e640a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009430 tx=0x7f4e700095c0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 -- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 msgr2=0x7f4e7819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 0x7f4e7819eb10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800d770 tx=0x7f4e6800dc30 comp rx=0 tx=0).stop 2024-04-16T05:42:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 -- 172.21.15.32:0/3017066027 shutdown_connections 2024-04-16T05:42:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78198690 0x7f4e78198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f4e640a3d20 0x7f4e640a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e78107330 0x7f4e7819eb10 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 --2- 172.21.15.32:0/3017066027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e78100770 0x7f4e7819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.439+0000 7f4e7ff75640 1 -- 172.21.15.32:0/3017066027 >> 172.21.15.32:0/3017066027 conn(0x7f4e78077450 msgr2=0x7f4e78105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.440+0000 7f4e7ff75640 1 -- 172.21.15.32:0/3017066027 shutdown_connections 2024-04-16T05:42:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.440+0000 7f4e7ff75640 1 -- 172.21.15.32:0/3017066027 wait complete. 2024-04-16T05:42:35.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:42:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 -- 172.21.15.32:0/2590404124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101ae0 msgr2=0x7fd104105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/2590404124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101ae0 0x7fd104105b20 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009900 tx=0x7fd0f402bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 -- 172.21.15.32:0/2590404124 shutdown_connections 2024-04-16T05:42:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/2590404124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd104106200 0x7fd10410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/2590404124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101ae0 0x7fd104105b20 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/2590404124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd104101130 0x7fd104101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 -- 172.21.15.32:0/2590404124 >> 172.21.15.32:0/2590404124 conn(0x7fd1040780b0 msgr2=0x7fd1040784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 -- 172.21.15.32:0/2590404124 shutdown_connections 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 -- 172.21.15.32:0/2590404124 wait complete. 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 Processor -- start 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 -- start start 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101130 0x7fd10419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd104106200 0x7fd10419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1041986d0 0x7fd104198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.562+0000 7fd10b2fb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041123d0 con 0x7fd104101130 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd10b2fb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd104112250 con 0x7fd104106200 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd10b2fb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd104112550 con 0x7fd1041986d0 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101130 0x7fd10419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101130 0x7fd10419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39382/0 (socket says 172.21.15.32:39382) 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 -- 172.21.15.32:0/326177627 learned_addr learned my addr 172.21.15.32:0/326177627 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd108896640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd104106200 0x7fd10419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 -- 172.21.15.32:0/326177627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1041986d0 msgr2=0x7fd104198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1041986d0 0x7fd104198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 -- 172.21.15.32:0/326177627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd104106200 msgr2=0x7fd10419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd104106200 0x7fd10419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 -- 172.21.15.32:0/326177627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1041992d0 con 0x7fd104101130 2024-04-16T05:42:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd108896640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd104106200 0x7fd10419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd109097640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101130 0x7fd10419e5c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd10000d950 tx=0x7fd10000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/326177627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100017070 con 0x7fd104101130 2024-04-16T05:42:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/326177627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd100009c30 con 0x7fd104101130 2024-04-16T05:42:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd10b2fb640 1 -- 172.21.15.32:0/326177627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1041a5360 con 0x7fd104101130 2024-04-16T05:42:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/326177627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10000f6b0 con 0x7fd104101130 2024-04-16T05:42:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.563+0000 7fd10b2fb640 1 -- 172.21.15.32:0/326177627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a5710 con 0x7fd104101130 2024-04-16T05:42:35.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.565+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/326177627 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd1000109d0 con 0x7fd104101130 2024-04-16T05:42:35.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd10dfbd740 1 Processor -- start 2024-04-16T05:42:35.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd10dfbd740 1 -- start start 2024-04-16T05:42:35.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af83e60 0x56059af84240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af8ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56059aee90e0 con 0x56059af8d360 2024-04-16T05:42:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56059ae86520 con 0x56059af84780 2024-04-16T05:42:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56059aee50a0 con 0x56059af83e60 2024-04-16T05:42:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd0fa7fc640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0f00a3d20 0x7fd0f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd108896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd108896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39394/0 (socket says 172.21.15.32:39394) 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd109898640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0f00a3d20 0x7fd0f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd108896640 1 -- 172.21.15.32:0/2379397509 learned_addr learned my addr 172.21.15.32:0/2379397509 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/326177627 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7fd1000d0870 con 0x7fd104101130 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.566+0000 7fd109898640 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af83e60 0x56059af84240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd109097640 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af8ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd108896640 1 -- 172.21.15.32:0/2379397509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af83e60 msgr2=0x56059af84240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd108896640 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af83e60 0x56059af84240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd108896640 1 -- 172.21.15.32:0/2379397509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 msgr2=0x56059af8ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd108896640 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af8ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd108896640 1 -- 172.21.15.32:0/2379397509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059aeae8d0 con 0x56059af8d360 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd109097640 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af8ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd108896640 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8f750 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4000c00 tx=0x7fd0f402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7f13c257daabd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd109898640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0f00a3d20 0x7fd0f00a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0092f0 tx=0x7fd0fc009420 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd0ef7fe640 1 -- 172.21.15.32:0/2379397509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f402cb40 con 0x56059af8d360 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd0ef7fe640 1 -- 172.21.15.32:0/2379397509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f402cce0 con 0x56059af8d360 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.567+0000 7fd0ef7fe640 1 -- 172.21.15.32:0/2379397509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4004920 con 0x56059af8d360 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2379397509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 msgr2=0x56059af8f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8f750 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4000c00 tx=0x7fd0f402c5d0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2379397509 shutdown_connections 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8f750 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af8ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/2379397509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af83e60 0x56059af84240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2379397509 >> 172.21.15.32:0/2379397509 conn(0x56059af7e590 msgr2=0x56059af7e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2379397509 shutdown_connections 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2379397509 wait complete. 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 -- start start 2024-04-16T05:42:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.568+0000 7fd10dfbd740 1 Processor -- start 2024-04-16T05:42:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 0x56059af8a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af84780 0x56059af8a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56059af8ffa0 con 0x56059af8d360 2024-04-16T05:42:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56059aea24d0 con 0x56059af83e60 2024-04-16T05:42:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56059af90120 con 0x56059af84780 2024-04-16T05:42:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd10dfbd740 10 client.0 _ll_register_callbacks cb 0x56059ae872c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39398/0 (socket says 172.21.15.32:39398) 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 -- 172.21.15.32:0/4094808573 learned_addr learned my addr 172.21.15.32:0/4094808573 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109097640 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 0x56059af8a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd108896640 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af84780 0x56059af8a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 -- 172.21.15.32:0/4094808573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af84780 msgr2=0x56059af8a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af84780 0x56059af8a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 -- 172.21.15.32:0/4094808573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 msgr2=0x56059af8a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 0x56059af8a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 -- 172.21.15.32:0/4094808573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059abfb3f0 con 0x56059af8d360 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd109898640 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8ae70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc012220 tx=0x7fd0fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af0eed193da9a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd0ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd0ecff9640 1 -- 172.21.15.32:0/4094808573 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc004d20 con 0x56059af8d360 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd0ecff9640 1 -- 172.21.15.32:0/4094808573 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7fd0fc005020 con 0x56059af8d360 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.569+0000 7fd10dfbd740 1 -- 172.21.15.32:0/4094808573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56059aef6b00 con 0x56059af8d360 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd0ecff9640 1 -- 172.21.15.32:0/4094808573 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc012310 con 0x56059af8d360 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd10dfbd740 10 client.17427 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd10dfbd740 1 -- 172.21.15.32:0/4094808573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56059acc57d0 con 0x56059af8d360 2024-04-16T05:42:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd0ecff9640 1 -- 172.21.15.32:0/4094808573 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56059aef6b00 con 0x56059af8d360 2024-04-16T05:42:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd0ecff9640 1 -- 172.21.15.32:0/4094808573 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0fc01a070 con 0x56059af8d360 2024-04-16T05:42:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd10dfbd740 10 client.17427 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd10dfbd740 10 client.17427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:42:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd10dfbd740 20 client.17427 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56059af9d3a0 0x56059af9f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd10dfbd740 4 client.17427 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd10dfbd740 1 -- 172.21.15.32:0/4094808573 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56059aef0970 con 0x56059af9d3a0 2024-04-16T05:42:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.570+0000 7fd109097640 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56059af9d3a0 0x56059af9f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd109097640 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56059af9d3a0 0x56059af9f7d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b1e99f43fa264b6 server_cookie=612ec45705fef214 in_seq=0 out_seq=0 2024-04-16T05:42:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd0ecff9640 10 client.17427 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:42:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd0ecff9640 1 -- 172.21.15.32:0/4094808573 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56059aef0970 con 0x56059af9d3a0 2024-04-16T05:42:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd0ecff9640 10 client.17427 handle_command_reply: tid=0 2024-04-16T05:42:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 client.17427 shutdown 2024-04-16T05:42:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 -- 172.21.15.32:0/4094808573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 msgr2=0x56059af8ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8ae70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc012220 tx=0x7fd0fc004a40 comp rx=0 tx=0).stop 2024-04-16T05:42:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 -- 172.21.15.32:0/4094808573 shutdown_connections 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56059af9d3a0 0x56059af9f7d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af8d360 0x56059af8ae70 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd0ecff9640 0 client.17427 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af84780 0x56059af8a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/4094808573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 0x56059af8a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.571+0000 7fd10dfbd740 1 -- 172.21.15.32:0/4094808573 >> 172.21.15.32:0/4094808573 conn(0x56059af7e590 msgr2=0x56059af802e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.572+0000 7fd10dfbd740 1 -- 172.21.15.32:0/4094808573 shutdown_connections 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.572+0000 7fd10dfbd740 1 -- 172.21.15.32:0/4094808573 wait complete. 2024-04-16T05:42:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.572+0000 7fd10dfbd740 20 client.17427 trim_cache size 0 max 16384 2024-04-16T05:42:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.585+0000 7fd10dfbd740 1 Processor -- start 2024-04-16T05:42:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.585+0000 7fd10dfbd740 1 -- start start 2024-04-16T05:42:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.585+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 0x56059af98f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.585+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af84780 0x56059af795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.585+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059afa5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.585+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041b1c10 con 0x56059af84780 2024-04-16T05:42:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.585+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041af4e0 con 0x56059af83e60 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.585+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ad350 con 0x56059af8d360 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af84780 0x56059af795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af84780 0x56059af795c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39400/0 (socket says 172.21.15.32:39400) 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 -- 172.21.15.32:0/2327457085 learned_addr learned my addr 172.21.15.32:0/2327457085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd109898640 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059afa5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 -- 172.21.15.32:0/2327457085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 msgr2=0x56059afa5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059afa5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 -- 172.21.15.32:0/2327457085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 msgr2=0x56059af98f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd109097640 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 0x56059af98f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 0x56059af98f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 -- 172.21.15.32:0/2327457085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059af893a0 con 0x56059af84780 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd108896640 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af84780 0x56059af795c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402c820 tx=0x7fd0f402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197e9175a305c9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd0eeffd640 1 -- 172.21.15.32:0/2327457085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f404e070 con 0x56059af84780 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd0eeffd640 1 -- 172.21.15.32:0/2327457085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4005520 con 0x56059af84780 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.586+0000 7fd0eeffd640 1 -- 172.21.15.32:0/2327457085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4004c50 con 0x56059af84780 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2327457085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af84780 msgr2=0x56059af795c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af84780 0x56059af795c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402c820 tx=0x7fd0f402ca10 comp rx=0 tx=0).stop 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2327457085 shutdown_connections 2024-04-16T05:42:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059afa5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af84780 0x56059af795c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/2327457085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af83e60 0x56059af98f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2327457085 >> 172.21.15.32:0/2327457085 conn(0x56059af7e590 msgr2=0x56059af80270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2327457085 shutdown_connections 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 -- 172.21.15.32:0/2327457085 wait complete. 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 -- start start 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.587+0000 7fd10dfbd740 1 Processor -- start 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af83e60 0x56059af87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd10dfbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059af87b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ad9a0 con 0x56059af83e60 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041aee10 con 0x56059af84780 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd10dfbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ae070 con 0x56059af8d360 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd10dfbd740 10 client.0 _ll_register_callbacks cb 0x7fd0fc009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af83e60 0x56059af87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af83e60 0x56059af87080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39410/0 (socket says 172.21.15.32:39410) 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 -- 172.21.15.32:0/3353715040 learned_addr learned my addr 172.21.15.32:0/3353715040 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd108896640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109898640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059af87b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 -- 172.21.15.32:0/3353715040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 msgr2=0x56059af87b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059af87b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 -- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 msgr2=0x56059af875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 -- 172.21.15.32:0/3353715040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059b002900 con 0x56059af83e60 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109898640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059af87b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd108896640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd109097640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af83e60 0x56059af87080 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd1000988e0 tx=0x7fd100005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34e4fda0782fbd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd0ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd0ecff9640 1 -- 172.21.15.32:0/3353715040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1000b4020 con 0x56059af83e60 2024-04-16T05:42:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.588+0000 7fd0ecff9640 1 -- 172.21.15.32:0/3353715040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7fd100098af0 con 0x56059af83e60 2024-04-16T05:42:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 1 -- 172.21.15.32:0/3353715040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f4005520 con 0x56059af83e60 2024-04-16T05:42:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd0ecff9640 1 -- 172.21.15.32:0/3353715040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd100020040 con 0x56059af83e60 2024-04-16T05:42:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 10 client.17433 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 1 -- 172.21.15.32:0/3353715040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56059b003be0 con 0x56059af83e60 2024-04-16T05:42:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd0ecff9640 1 -- 172.21.15.32:0/3353715040 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4005520 con 0x56059af83e60 2024-04-16T05:42:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd0ecff9640 1 -- 172.21.15.32:0/3353715040 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd10009b8d0 con 0x56059af83e60 2024-04-16T05:42:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 10 client.17433 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 10 client.17433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:42:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 20 client.17433 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56059b004360 0x56059b006780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 4 client.17433 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd10dfbd740 1 -- 172.21.15.32:0/3353715040 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56059ae6c6b0 con 0x56059b004360 2024-04-16T05:42:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.589+0000 7fd108896640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56059b004360 0x56059b006780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd0ecff9640 10 client.17433 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd108896640 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56059b004360 0x56059b006780 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4fdede5a226f1dd server_cookie=81f999354b8eb92f in_seq=0 out_seq=0 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd0ecff9640 1 -- 172.21.15.32:0/3353715040 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56059ae6c6b0 con 0x56059b004360 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd0ecff9640 10 client.17433 handle_command_reply: tid=0 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 client.17433 shutdown 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 -- 172.21.15.32:0/3353715040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af83e60 msgr2=0x56059af87080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af83e60 0x56059af87080 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd1000988e0 tx=0x7fd100005d30 comp rx=0 tx=0).stop 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 -- 172.21.15.32:0/3353715040 shutdown_connections 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56059b004360 0x56059b006780 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56059af8d360 0x56059af87b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd0ecff9640 0 client.17433 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:42:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56059af84780 0x56059af875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 --2- 172.21.15.32:0/3353715040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56059af83e60 0x56059af87080 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 -- 172.21.15.32:0/3353715040 >> 172.21.15.32:0/3353715040 conn(0x56059af7e590 msgr2=0x56059af7cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 -- 172.21.15.32:0/3353715040 shutdown_connections 2024-04-16T05:42:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 1 -- 172.21.15.32:0/3353715040 wait complete. 2024-04-16T05:42:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.590+0000 7fd10dfbd740 20 client.17433 trim_cache size 0 max 16384 2024-04-16T05:42:35.599 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:42:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 -- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0f00a3d20 msgr2=0x7fd0f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0f00a3d20 0x7fd0f00a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0092f0 tx=0x7fd0fc009420 comp rx=0 tx=0).stop 2024-04-16T05:42:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 -- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101130 msgr2=0x7fd10419e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101130 0x7fd10419e5c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd10000d950 tx=0x7fd10000de10 comp rx=0 tx=0).stop 2024-04-16T05:42:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 -- 172.21.15.32:0/326177627 shutdown_connections 2024-04-16T05:42:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1041986d0 0x7fd104198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd104106200 0x7fd10419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd0f00a3d20 0x7fd0f00a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 --2- 172.21.15.32:0/326177627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104101130 0x7fd10419e5c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 -- 172.21.15.32:0/326177627 >> 172.21.15.32:0/326177627 conn(0x7fd1040780b0 msgr2=0x7fd1040fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 -- 172.21.15.32:0/326177627 shutdown_connections 2024-04-16T05:42:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.591+0000 7fd10b2fb640 1 -- 172.21.15.32:0/326177627 wait complete. 2024-04-16T05:42:35.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:42:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 -- 172.21.15.32:0/3493142461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 msgr2=0x7f0b30102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 --2- 172.21.15.32:0/3493142461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 0x7f0b30102630 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009900 tx=0x7f0b2c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:42:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 -- 172.21.15.32:0/3493142461 shutdown_connections 2024-04-16T05:42:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 --2- 172.21.15.32:0/3493142461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30107320 0x7f0b3010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 --2- 172.21.15.32:0/3493142461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b30102c00 0x7f0b30106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 --2- 172.21.15.32:0/3493142461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 0x7f0b30102630 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 -- 172.21.15.32:0/3493142461 >> 172.21.15.32:0/3493142461 conn(0x7f0b30079490 msgr2=0x7f0b300798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 -- 172.21.15.32:0/3493142461 shutdown_connections 2024-04-16T05:42:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.711+0000 7f0b38315640 1 -- 172.21.15.32:0/3493142461 wait complete. 2024-04-16T05:42:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b38315640 1 Processor -- start 2024-04-16T05:42:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b38315640 1 -- start start 2024-04-16T05:42:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b38315640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 0x7f0b3019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b38315640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30107320 0x7f0b3019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b38315640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b30198650 0x7f0b30198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b38315640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301123b0 con 0x7f0b30102250 2024-04-16T05:42:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b38315640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112230 con 0x7f0b30198650 2024-04-16T05:42:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b38315640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112530 con 0x7f0b30107320 2024-04-16T05:42:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 0x7f0b3019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 0x7f0b3019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39422/0 (socket says 172.21.15.32:39422) 2024-04-16T05:42:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 -- 172.21.15.32:0/3591051607 learned_addr learned my addr 172.21.15.32:0/3591051607 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b358b0640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30107320 0x7f0b3019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 -- 172.21.15.32:0/3591051607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30107320 msgr2=0x7f0b3019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30107320 0x7f0b3019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 -- 172.21.15.32:0/3591051607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b30198650 msgr2=0x7f0b30198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b30198650 0x7f0b30198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 -- 172.21.15.32:0/3591051607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b30199250 con 0x7f0b30102250 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b358b0640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30107320 0x7f0b3019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.712+0000 7f0b360b1640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 0x7f0b3019e590 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006010 tx=0x7f0b2c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.713+0000 7f0b277fe640 1 -- 172.21.15.32:0/3591051607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c041070 con 0x7f0b30102250 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.713+0000 7f0b277fe640 1 -- 172.21.15.32:0/3591051607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b2c004450 con 0x7f0b30102250 2024-04-16T05:42:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.713+0000 7f0b38315640 1 -- 172.21.15.32:0/3591051607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b301994e0 con 0x7f0b30102250 2024-04-16T05:42:35.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.713+0000 7f0b277fe640 1 -- 172.21.15.32:0/3591051607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c034440 con 0x7f0b30102250 2024-04-16T05:42:35.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.713+0000 7f0b38315640 1 -- 172.21.15.32:0/3591051607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b301a5620 con 0x7f0b30102250 2024-04-16T05:42:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.715+0000 7f0b277fe640 1 -- 172.21.15.32:0/3591051607 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b2c044070 con 0x7f0b30102250 2024-04-16T05:42:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.715+0000 7f0b3afde740 1 Processor -- start 2024-04-16T05:42:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b3afde740 1 -- start start 2024-04-16T05:42:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b277fe640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd3fe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd406e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd409750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b3afde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5564dd3630e0 con 0x5564dd3fde60 2024-04-16T05:42:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b3afde740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5564dd300520 con 0x5564dd3fe780 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b3afde740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5564dd35f0a0 con 0x5564dd407360 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b360b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd3fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b368b2640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b360b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd3fe240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39438/0 (socket says 172.21.15.32:39438) 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b360b1640 1 -- 172.21.15.32:0/1177654954 learned_addr learned my addr 172.21.15.32:0/1177654954 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b277fe640 1 -- 172.21.15.32:0/3591051607 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f0b2c0f27a0 con 0x7f0b30102250 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b368b2640 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd409750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b358b0640 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd406e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b360b1640 1 -- 172.21.15.32:0/1177654954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 msgr2=0x5564dd409750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b360b1640 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd409750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b360b1640 1 -- 172.21.15.32:0/1177654954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 msgr2=0x5564dd406e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b360b1640 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd406e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b360b1640 1 -- 172.21.15.32:0/1177654954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564dd3288d0 con 0x5564dd3fde60 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.716+0000 7f0b358b0640 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd406e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.717+0000 7f0b368b2640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009370 tx=0x7f0b28009890 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.717+0000 7f0b360b1640 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd3fe240 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0bae30 tx=0x7f0b2c003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8fe2834281d7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.717+0000 7f0b24bfe640 1 -- 172.21.15.32:0/1177654954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c04b040 con 0x5564dd3fde60 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.717+0000 7f0b24bfe640 1 -- 172.21.15.32:0/1177654954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b2c04be80 con 0x5564dd3fde60 2024-04-16T05:42:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.717+0000 7f0b24bfe640 1 -- 172.21.15.32:0/1177654954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0be460 con 0x5564dd3fde60 2024-04-16T05:42:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.717+0000 7f0b3afde740 1 -- 172.21.15.32:0/1177654954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 msgr2=0x5564dd3fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.717+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd3fe240 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0bae30 tx=0x7f0b2c003990 comp rx=0 tx=0).stop 2024-04-16T05:42:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 -- 172.21.15.32:0/1177654954 shutdown_connections 2024-04-16T05:42:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd409750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd406e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1177654954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd3fe240 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 -- 172.21.15.32:0/1177654954 >> 172.21.15.32:0/1177654954 conn(0x5564dd3f8590 msgr2=0x5564dd3f8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 -- 172.21.15.32:0/1177654954 shutdown_connections 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 -- 172.21.15.32:0/1177654954 wait complete. 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 -- start start 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 Processor -- start 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd404290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd4047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd404d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5564dd409f10 con 0x5564dd3fde60 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5564dd31c4d0 con 0x5564dd3fe780 2024-04-16T05:42:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5564dd40a090 con 0x5564dd407360 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b3afde740 10 client.0 _ll_register_callbacks cb 0x5564dd3012c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b360b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd404290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b360b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd404290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39450/0 (socket says 172.21.15.32:39450) 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.718+0000 7f0b360b1640 1 -- 172.21.15.32:0/849745996 learned_addr learned my addr 172.21.15.32:0/849745996 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b358b0640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd4047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b368b2640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd404d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b360b1640 1 -- 172.21.15.32:0/849745996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 msgr2=0x5564dd404d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b360b1640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd404d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b360b1640 1 -- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 msgr2=0x5564dd4047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b360b1640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd4047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b360b1640 1 -- 172.21.15.32:0/849745996 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564dd0753f0 con 0x5564dd3fde60 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b358b0640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd4047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b368b2640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd404d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b360b1640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd404290 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c02e040 tx=0x7f0b2c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432fcd0e61c0bb8f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b19ffb640 1 -- 172.21.15.32:0/849745996 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0d7060 con 0x5564dd3fde60 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b19ffb640 1 -- 172.21.15.32:0/849745996 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f0b2c0f2340 con 0x5564dd3fde60 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b3afde740 1 -- 172.21.15.32:0/849745996 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b2c04be80 con 0x5564dd3fde60 2024-04-16T05:42:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b19ffb640 1 -- 172.21.15.32:0/849745996 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b2c04b050 con 0x5564dd3fde60 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b3afde740 10 client.17445 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b3afde740 1 -- 172.21.15.32:0/849745996 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564dd13f7d0 con 0x5564dd3fde60 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b19ffb640 1 -- 172.21.15.32:0/849745996 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b2c04be80 con 0x5564dd3fde60 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.719+0000 7f0b19ffb640 1 -- 172.21.15.32:0/849745996 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0b2c0c6c40 con 0x5564dd3fde60 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b3afde740 10 client.17445 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b3afde740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b3afde740 20 client.17445 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b3afde740 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5564dd416d90 0x5564dd419180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b3afde740 4 client.17445 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b3afde740 1 -- 172.21.15.32:0/849745996 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564dd370b00 con 0x5564dd416d90 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b358b0640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5564dd416d90 0x5564dd419180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b19ffb640 10 client.17445 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.720+0000 7f0b358b0640 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5564dd416d90 0x5564dd419180 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b62b0a2d2ea901b4 server_cookie=7c83c5146bf23d64 in_seq=0 out_seq=0 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b19ffb640 1 -- 172.21.15.32:0/849745996 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564dd370b00 con 0x5564dd416d90 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b19ffb640 10 client.17445 handle_command_reply: tid=0 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 client.17445 shutdown 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 -- 172.21.15.32:0/849745996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 msgr2=0x5564dd404290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd404290 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c02e040 tx=0x7f0b2c003820 comp rx=0 tx=0).stop 2024-04-16T05:42:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 -- 172.21.15.32:0/849745996 shutdown_connections 2024-04-16T05:42:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5564dd416d90 0x5564dd419180 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd407360 0x5564dd404d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd4047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b19ffb640 0 client.17445 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 --2- 172.21.15.32:0/849745996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd3fde60 0x5564dd404290 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 -- 172.21.15.32:0/849745996 >> 172.21.15.32:0/849745996 conn(0x5564dd3f8590 msgr2=0x5564dd3fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 -- 172.21.15.32:0/849745996 shutdown_connections 2024-04-16T05:42:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 1 -- 172.21.15.32:0/849745996 wait complete. 2024-04-16T05:42:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.721+0000 7f0b3afde740 20 client.17445 trim_cache size 0 max 16384 2024-04-16T05:42:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b3afde740 1 Processor -- start 2024-04-16T05:42:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b3afde740 1 -- start start 2024-04-16T05:42:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fde60 0x5564dd4069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd403500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd412c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b3afde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301b1cb0 con 0x5564dd407360 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b3afde740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301af580 con 0x5564dd3fe780 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b3afde740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301ad3f0 con 0x5564dd3fde60 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd412c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd412c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39464/0 (socket says 172.21.15.32:39464) 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 -- 172.21.15.32:0/2820704329 learned_addr learned my addr 172.21.15.32:0/2820704329 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b360b1640 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fde60 0x5564dd4069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b358b0640 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd403500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 -- 172.21.15.32:0/2820704329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fde60 msgr2=0x5564dd4069c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fde60 0x5564dd4069c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 -- 172.21.15.32:0/2820704329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 msgr2=0x5564dd403500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd403500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 -- 172.21.15.32:0/2820704329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564dd3f83c0 con 0x5564dd407360 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.735+0000 7f0b368b2640 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd412c70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009da0 tx=0x7f0b28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81da861f5bcef2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b1bfff640 1 -- 172.21.15.32:0/2820704329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28011dc0 con 0x5564dd407360 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b1bfff640 1 -- 172.21.15.32:0/2820704329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b2801f480 con 0x5564dd407360 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b1bfff640 1 -- 172.21.15.32:0/2820704329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28005260 con 0x5564dd407360 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 -- 172.21.15.32:0/2820704329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 msgr2=0x5564dd412c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd412c70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009da0 tx=0x7f0b28017000 comp rx=0 tx=0).stop 2024-04-16T05:42:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 -- 172.21.15.32:0/2820704329 shutdown_connections 2024-04-16T05:42:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd412c70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fe780 0x5564dd403500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 --2- 172.21.15.32:0/2820704329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fde60 0x5564dd4069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 -- 172.21.15.32:0/2820704329 >> 172.21.15.32:0/2820704329 conn(0x5564dd3f8590 msgr2=0x5564dd3f7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 -- 172.21.15.32:0/2820704329 shutdown_connections 2024-04-16T05:42:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 -- 172.21.15.32:0/2820704329 wait complete. 2024-04-16T05:42:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.736+0000 7f0b3afde740 1 -- start start 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b3afde740 1 Processor -- start 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fde60 0x5564dd40f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fe780 0x5564dd40a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b3afde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd40ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b3afde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301ada40 con 0x5564dd407360 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b3afde740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301aeeb0 con 0x5564dd3fde60 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b3afde740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301ae110 con 0x5564dd3fe780 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b3afde740 10 client.0 _ll_register_callbacks cb 0x7f0b2c0da420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd40ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd40ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39466/0 (socket says 172.21.15.32:39466) 2024-04-16T05:42:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 -- 172.21.15.32:0/1664121131 learned_addr learned my addr 172.21.15.32:0/1664121131 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b358b0640 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fe780 0x5564dd40a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b360b1640 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fde60 0x5564dd40f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 -- 172.21.15.32:0/1664121131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fe780 msgr2=0x5564dd40a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fe780 0x5564dd40a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 -- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fde60 msgr2=0x5564dd40f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fde60 0x5564dd40f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 -- 172.21.15.32:0/1664121131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564dd40b6f0 con 0x5564dd407360 2024-04-16T05:42:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b360b1640 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fde60 0x5564dd40f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b368b2640 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd40ae70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f0b28005ba0 tx=0x7f0b28020f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102fed7cb461adbe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.737+0000 7f0b19ffb640 1 -- 172.21.15.32:0/1664121131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2802f070 con 0x5564dd407360 2024-04-16T05:42:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b19ffb640 1 -- 172.21.15.32:0/1664121131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f0b2801f840 con 0x5564dd407360 2024-04-16T05:42:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 1 -- 172.21.15.32:0/1664121131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b2801f480 con 0x5564dd407360 2024-04-16T05:42:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b19ffb640 1 -- 172.21.15.32:0/1664121131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b2803b050 con 0x5564dd407360 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 10 client.17451 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 1 -- 172.21.15.32:0/1664121131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564dd47d2c0 con 0x5564dd407360 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b19ffb640 1 -- 172.21.15.32:0/1664121131 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b2801f480 con 0x5564dd407360 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b19ffb640 1 -- 172.21.15.32:0/1664121131 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0b28034440 con 0x5564dd407360 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 10 client.17451 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 20 client.17451 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5564dd47d9e0 0x5564dd47fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 4 client.17451 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b3afde740 1 -- 172.21.15.32:0/1664121131 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564dd36a970 con 0x5564dd47d9e0 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.738+0000 7f0b360b1640 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5564dd47d9e0 0x5564dd47fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b19ffb640 10 client.17451 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b360b1640 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5564dd47d9e0 0x5564dd47fe00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0de268f4e2923a server_cookie=75a404d56fe46721 in_seq=0 out_seq=0 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b19ffb640 1 -- 172.21.15.32:0/1664121131 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564dd36a970 con 0x5564dd47d9e0 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b19ffb640 10 client.17451 handle_command_reply: tid=0 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 client.17451 shutdown 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 -- 172.21.15.32:0/1664121131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 msgr2=0x5564dd40ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd40ae70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f0b28005ba0 tx=0x7f0b28020f90 comp rx=0 tx=0).stop 2024-04-16T05:42:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 -- 172.21.15.32:0/1664121131 shutdown_connections 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564dd407360 0x5564dd40ae70 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5564dd47d9e0 0x5564dd47fe00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564dd3fe780 0x5564dd40a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b19ffb640 0 client.17451 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 --2- 172.21.15.32:0/1664121131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564dd3fde60 0x5564dd40f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 -- 172.21.15.32:0/1664121131 >> 172.21.15.32:0/1664121131 conn(0x5564dd3f8590 msgr2=0x5564dd3f6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 -- 172.21.15.32:0/1664121131 shutdown_connections 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 1 -- 172.21.15.32:0/1664121131 wait complete. 2024-04-16T05:42:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.739+0000 7f0b3afde740 20 client.17451 trim_cache size 0 max 16384 2024-04-16T05:42:35.749 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:42:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 -- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0b1c0a3d20 msgr2=0x7f0b1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009370 tx=0x7f0b28009890 comp rx=0 tx=0).stop 2024-04-16T05:42:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 -- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 msgr2=0x7f0b3019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 0x7f0b3019e590 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006010 tx=0x7f0b2c02cc70 comp rx=0 tx=0).stop 2024-04-16T05:42:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 -- 172.21.15.32:0/3591051607 shutdown_connections 2024-04-16T05:42:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b30198650 0x7f0b30198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30107320 0x7f0b3019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 --2- 172.21.15.32:0/3591051607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b30102250 0x7f0b3019e590 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:35.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 -- 172.21.15.32:0/3591051607 >> 172.21.15.32:0/3591051607 conn(0x7f0b30079490 msgr2=0x7f0b300ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:35.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 -- 172.21.15.32:0/3591051607 shutdown_connections 2024-04-16T05:42:35.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:35.740+0000 7f0b38315640 1 -- 172.21.15.32:0/3591051607 wait complete. 2024-04-16T05:42:35.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:35 smithi032 ceph-mon[32351]: pgmap v234: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:35.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:35 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/375717444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:35 smithi134 ceph-mon[36153]: pgmap v234: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:35 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/375717444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:36.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:35 smithi082 ceph-mon[36195]: pgmap v234: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:36.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:35 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/375717444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:42:36.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:42:36.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:42:36.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:42:36.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/948899525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103280 msgr2=0x7fa8cc103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/948899525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103280 0x7fa8cc103660 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009900 tx=0x7fa8bc02bd60 comp rx=0 tx=0).stop 2024-04-16T05:42:36.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/948899525 shutdown_connections 2024-04-16T05:42:36.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/948899525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8cc10c5c0 0x7fa8cc10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/948899525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8cc103ba0 0x7fa8cc10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/948899525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103280 0x7fa8cc103660 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/948899525 >> 172.21.15.32:0/948899525 conn(0x7fa8cc078000 msgr2=0x7fa8cc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/948899525 shutdown_connections 2024-04-16T05:42:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.878+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/948899525 wait complete. 2024-04-16T05:42:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8d1f7c640 1 Processor -- start 2024-04-16T05:42:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8d1f7c640 1 -- start start 2024-04-16T05:42:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8d1f7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8cc103280 0x7fa8cc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8d1f7c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103ba0 0x7fa8cc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8d1f7c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8cc10c5c0 0x7fa8cc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8d1f7c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1123d0 con 0x7fa8cc103ba0 2024-04-16T05:42:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8d1f7c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc112250 con 0x7fa8cc10c5c0 2024-04-16T05:42:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8d1f7c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc112550 con 0x7fa8cc103280 2024-04-16T05:42:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103ba0 0x7fa8cc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103ba0 0x7fa8cc19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39476/0 (socket says 172.21.15.32:39476) 2024-04-16T05:42:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8caffd640 1 -- 172.21.15.32:0/14346155 learned_addr learned my addr 172.21.15.32:0/14346155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8cc103280 0x7fa8cc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8cc10c5c0 0x7fa8cc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8caffd640 1 -- 172.21.15.32:0/14346155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8cc103280 msgr2=0x7fa8cc19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8caffd640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8cc103280 0x7fa8cc19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8caffd640 1 -- 172.21.15.32:0/14346155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8cc10c5c0 msgr2=0x7fa8cc1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8caffd640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8cc10c5c0 0x7fa8cc1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.879+0000 7fa8caffd640 1 -- 172.21.15.32:0/14346155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc198fb0 con 0x7fa8cc103ba0 2024-04-16T05:42:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.880+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8cc10c5c0 0x7fa8cc1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.880+0000 7fa8caffd640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103ba0 0x7fa8cc19eb70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000d950 tx=0x7fa8c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.880+0000 7fa8c8ff9640 1 -- 172.21.15.32:0/14346155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0017070 con 0x7fa8cc103ba0 2024-04-16T05:42:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.880+0000 7fa8c8ff9640 1 -- 172.21.15.32:0/14346155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c0009c30 con 0x7fa8cc103ba0 2024-04-16T05:42:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.880+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/14346155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc1992a0 con 0x7fa8cc103ba0 2024-04-16T05:42:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.880+0000 7fa8c8ff9640 1 -- 172.21.15.32:0/14346155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000f6b0 con 0x7fa8cc103ba0 2024-04-16T05:42:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.880+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/14346155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a55b0 con 0x7fa8cc103ba0 2024-04-16T05:42:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.882+0000 7fa8c8ff9640 1 -- 172.21.15.32:0/14346155 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa8c00109d0 con 0x7fa8cc103ba0 2024-04-16T05:42:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8d4b66740 1 Processor -- start 2024-04-16T05:42:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8d4b66740 1 -- start start 2024-04-16T05:42:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2416870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416e40 0x55efc2423e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55efc237be90 con 0x55efc2416490 2024-04-16T05:42:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55efc23194c0 con 0x55efc2416e40 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55efc2377e50 con 0x55efc24243c0 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8c8ff9640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8b40a3d20 0x7fa8b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2416870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2416870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39484/0 (socket says 172.21.15.32:39484) 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8cbfff640 1 -- 172.21.15.32:0/2630985790 learned_addr learned my addr 172.21.15.32:0/2630985790 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8b40a3d20 0x7fa8b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8caffd640 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416e40 0x55efc2423e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.883+0000 7fa8c8ff9640 1 -- 172.21.15.32:0/14346155 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7fa8c00d0820 con 0x7fa8cc103ba0 2024-04-16T05:42:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8cbfff640 1 -- 172.21.15.32:0/2630985790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 msgr2=0x55efc24267b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24267b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8cbfff640 1 -- 172.21.15.32:0/2630985790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416e40 msgr2=0x55efc2423e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416e40 0x55efc2423e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8cbfff640 1 -- 172.21.15.32:0/2630985790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc21567e0 con 0x55efc2416490 2024-04-16T05:42:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2416870 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4012d90 tx=0x7fa8c40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaddc5a360410b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8b40a3d20 0x7fa8b40a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4010ca0 tx=0x7fa8c401d000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8b1ffb640 1 -- 172.21.15.32:0/2630985790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4012580 con 0x55efc2416490 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8b1ffb640 1 -- 172.21.15.32:0/2630985790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c4012870 con 0x55efc2416490 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.884+0000 7fa8b1ffb640 1 -- 172.21.15.32:0/2630985790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4005460 con 0x55efc2416490 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2630985790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 msgr2=0x55efc2416870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2416870 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4012d90 tx=0x7fa8c40107a0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2630985790 shutdown_connections 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416e40 0x55efc2423e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2630985790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2416870 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2630985790 >> 172.21.15.32:0/2630985790 conn(0x55efc24111e0 msgr2=0x55efc24115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2630985790 shutdown_connections 2024-04-16T05:42:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2630985790 wait complete. 2024-04-16T05:42:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 -- start start 2024-04-16T05:42:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.885+0000 7fa8d4b66740 1 Processor -- start 2024-04-16T05:42:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2421480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 0x55efc24219c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2421f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55efc24270c0 con 0x55efc2416490 2024-04-16T05:42:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55efc2304410 con 0x55efc24243c0 2024-04-16T05:42:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55efc2427240 con 0x55efc2416e40 2024-04-16T05:42:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8d4b66740 10 client.0 _ll_register_callbacks cb 0x55efc22ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2421480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2421480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39498/0 (socket says 172.21.15.32:39498) 2024-04-16T05:42:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 -- 172.21.15.32:0/21200170 learned_addr learned my addr 172.21.15.32:0/21200170 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2421f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8caffd640 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 0x55efc24219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 -- 172.21.15.32:0/21200170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 msgr2=0x55efc24219c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 0x55efc24219c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 -- 172.21.15.32:0/21200170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 msgr2=0x55efc2421f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2421f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 -- 172.21.15.32:0/21200170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc21b6f70 con 0x55efc2416490 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2421480 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc006010 tx=0x7fa8bc0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a895e3696f7586 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.886+0000 7fa8aaffd640 1 -- 172.21.15.32:0/21200170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc044070 con 0x55efc2416490 2024-04-16T05:42:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8aaffd640 1 -- 172.21.15.32:0/21200170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7fa8bc0046c0 con 0x55efc2416490 2024-04-16T05:42:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 1 -- 172.21.15.32:0/21200170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c4012870 con 0x55efc2416490 2024-04-16T05:42:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8aaffd640 1 -- 172.21.15.32:0/21200170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc051050 con 0x55efc2416490 2024-04-16T05:42:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 10 client.17463 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 1 -- 172.21.15.32:0/21200170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55efc204e170 con 0x55efc2416490 2024-04-16T05:42:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8aaffd640 1 -- 172.21.15.32:0/21200170 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c4012870 con 0x55efc2416490 2024-04-16T05:42:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8aaffd640 1 -- 172.21.15.32:0/21200170 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa8bc0035f0 con 0x55efc2416490 2024-04-16T05:42:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 10 client.17463 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 20 client.17463 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55efc2434710 0x55efc2436b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 4 client.17463 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8d4b66740 1 -- 172.21.15.32:0/21200170 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc23898b0 con 0x55efc2434710 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.887+0000 7fa8caffd640 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55efc2434710 0x55efc2436b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.888+0000 7fa8aaffd640 10 client.17463 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.888+0000 7fa8caffd640 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55efc2434710 0x55efc2436b40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b9e037d97886b6 server_cookie=3a950e13b54b0d2c in_seq=0 out_seq=0 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.888+0000 7fa8aaffd640 1 -- 172.21.15.32:0/21200170 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55efc23898b0 con 0x55efc2434710 2024-04-16T05:42:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.888+0000 7fa8aaffd640 10 client.17463 handle_command_reply: tid=0 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.888+0000 7fa8d4b66740 1 client.17463 shutdown 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 -- 172.21.15.32:0/21200170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 msgr2=0x55efc2421480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2421480 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc006010 tx=0x7fa8bc0045f0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 -- 172.21.15.32:0/21200170 shutdown_connections 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2421f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55efc2434710 0x55efc2436b40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 0x55efc24219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8aaffd640 0 client.17463 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/21200170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc2421480 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 -- 172.21.15.32:0/21200170 >> 172.21.15.32:0/21200170 conn(0x55efc24111e0 msgr2=0x55efc2412f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 -- 172.21.15.32:0/21200170 shutdown_connections 2024-04-16T05:42:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 1 -- 172.21.15.32:0/21200170 wait complete. 2024-04-16T05:42:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.889+0000 7fa8d4b66740 20 client.17463 trim_cache size 0 max 16384 2024-04-16T05:42:36.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8d4b66740 1 Processor -- start 2024-04-16T05:42:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8d4b66740 1 -- start start 2024-04-16T05:42:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416490 0x55efc2423bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416e40 0x55efc242ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1afb70 con 0x55efc2416e40 2024-04-16T05:42:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ad440 con 0x55efc2416490 2024-04-16T05:42:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ab2b0 con 0x55efc24243c0 2024-04-16T05:42:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416e40 0x55efc242ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416e40 0x55efc242ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39504/0 (socket says 172.21.15.32:39504) 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 -- 172.21.15.32:0/1085365872 learned_addr learned my addr 172.21.15.32:0/1085365872 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 -- 172.21.15.32:0/1085365872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 msgr2=0x55efc24100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 -- 172.21.15.32:0/1085365872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416490 msgr2=0x55efc2423bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416490 0x55efc2423bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416490 0x55efc2423bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 -- 172.21.15.32:0/1085365872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc2416be0 con 0x55efc2416e40 2024-04-16T05:42:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.907+0000 7fa8caffd640 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416e40 0x55efc242ffc0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009610 tx=0x7fa8c001e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886f39545a7ee5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8b17fa640 1 -- 172.21.15.32:0/1085365872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0020040 con 0x55efc2416e40 2024-04-16T05:42:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8b17fa640 1 -- 172.21.15.32:0/1085365872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c00a5040 con 0x55efc2416e40 2024-04-16T05:42:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8b17fa640 1 -- 172.21.15.32:0/1085365872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0098bd0 con 0x55efc2416e40 2024-04-16T05:42:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 -- 172.21.15.32:0/1085365872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416e40 msgr2=0x55efc242ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416e40 0x55efc242ffc0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009610 tx=0x7fa8c001e770 comp rx=0 tx=0).stop 2024-04-16T05:42:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 -- 172.21.15.32:0/1085365872 shutdown_connections 2024-04-16T05:42:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc24243c0 0x55efc24100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416e40 0x55efc242ffc0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/1085365872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc2416490 0x55efc2423bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 -- 172.21.15.32:0/1085365872 >> 172.21.15.32:0/1085365872 conn(0x55efc24111e0 msgr2=0x55efc240f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 -- 172.21.15.32:0/1085365872 shutdown_connections 2024-04-16T05:42:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.908+0000 7fa8d4b66740 1 -- 172.21.15.32:0/1085365872 wait complete. 2024-04-16T05:42:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 1 -- start start 2024-04-16T05:42:36.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 1 Processor -- start 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc24294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 0x55efc24299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2429f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ab900 con 0x55efc2416490 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1acd70 con 0x55efc24243c0 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1abfd0 con 0x55efc2416e40 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8d4b66740 10 client.0 _ll_register_callbacks cb 0x7fa8bc005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc24294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc24294a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39516/0 (socket says 172.21.15.32:39516) 2024-04-16T05:42:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cb7fe640 1 -- 172.21.15.32:0/2169574055 learned_addr learned my addr 172.21.15.32:0/2169574055 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2429f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cb7fe640 1 -- 172.21.15.32:0/2169574055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 msgr2=0x55efc24299e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 0x55efc24299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cb7fe640 1 -- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 msgr2=0x55efc2429f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2429f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cb7fe640 1 -- 172.21.15.32:0/2169574055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc249ac90 con 0x55efc2416490 2024-04-16T05:42:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.909+0000 7fa8cbfff640 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2429f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8cb7fe640 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc24294a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009900 tx=0x7fa8bc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b542b7c1895b596 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8aaffd640 1 -- 172.21.15.32:0/2169574055 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc044030 con 0x55efc2416490 2024-04-16T05:42:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8aaffd640 1 -- 172.21.15.32:0/2169574055 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7fa8bc005070 con 0x55efc2416490 2024-04-16T05:42:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2169574055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c00a5040 con 0x55efc2416490 2024-04-16T05:42:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8aaffd640 1 -- 172.21.15.32:0/2169574055 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc05d050 con 0x55efc2416490 2024-04-16T05:42:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 10 client.17469 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2169574055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55efc249be80 con 0x55efc2416490 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8aaffd640 1 -- 172.21.15.32:0/2169574055 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c00a5040 con 0x55efc2416490 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8aaffd640 1 -- 172.21.15.32:0/2169574055 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa8bc004b80 con 0x55efc2416490 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 10 client.17469 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 20 client.17469 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55efc2434710 0x55efc249e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 4 client.17469 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:42:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.910+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2169574055 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55efc2383720 con 0x55efc2434710 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.911+0000 7fa8caffd640 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55efc2434710 0x55efc249e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.911+0000 7fa8caffd640 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55efc2434710 0x55efc249e5e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9572176a14f8677a server_cookie=2b15e691a951ca0d in_seq=0 out_seq=0 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.911+0000 7fa8aaffd640 10 client.17469 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8aaffd640 1 -- 172.21.15.32:0/2169574055 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55efc2383720 con 0x55efc2434710 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8aaffd640 10 client.17469 handle_command_reply: tid=0 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 client.17469 shutdown 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2169574055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 msgr2=0x55efc24294a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc24294a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009900 tx=0x7fa8bc005850 comp rx=0 tx=0).stop 2024-04-16T05:42:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2169574055 shutdown_connections 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc24243c0 0x55efc2429f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55efc2434710 0x55efc249e5e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55efc2416e40 0x55efc24299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 --2- 172.21.15.32:0/2169574055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55efc2416490 0x55efc24294a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2169574055 >> 172.21.15.32:0/2169574055 conn(0x55efc24111e0 msgr2=0x55efc2410250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8aaffd640 0 client.17469 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2169574055 shutdown_connections 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.912+0000 7fa8d4b66740 1 -- 172.21.15.32:0/2169574055 wait complete. 2024-04-16T05:42:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d4b66740 20 client.17469 trim_cache size 0 max 16384 2024-04-16T05:42:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:42:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:42:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "fcbda044-c1bc-48d5-80c6-9fe2d05aa69b", 2024-04-16T05:42:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:42:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:42:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8b40a3d20 msgr2=0x7fa8b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8b40a3d20 0x7fa8b40a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4010ca0 tx=0x7fa8c401d000 comp rx=0 tx=0).stop 2024-04-16T05:42:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103ba0 msgr2=0x7fa8cc19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103ba0 0x7fa8cc19eb70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000d950 tx=0x7fa8c000de10 comp rx=0 tx=0).stop 2024-04-16T05:42:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/14346155 shutdown_connections 2024-04-16T05:42:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8cc10c5c0 0x7fa8cc1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8b40a3d20 0x7fa8b40a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8cc103ba0 0x7fa8cc19eb70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 --2- 172.21.15.32:0/14346155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8cc103280 0x7fa8cc19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:36.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/14346155 >> 172.21.15.32:0/14346155 conn(0x7fa8cc078000 msgr2=0x7fa8cc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:36.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/14346155 shutdown_connections 2024-04-16T05:42:36.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:36.913+0000 7fa8d1f7c640 1 -- 172.21.15.32:0/14346155 wait complete. 2024-04-16T05:42:36.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:42:36.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:42:37.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 -- 172.21.15.32:0/3334232046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4101bf0 msgr2=0x7f43e410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 --2- 172.21.15.32:0/3334232046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4101bf0 0x7f43e410dd30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009cd0 tx=0x7f43dc02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 -- 172.21.15.32:0/3334232046 shutdown_connections 2024-04-16T05:42:37.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 --2- 172.21.15.32:0/3334232046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4101bf0 0x7f43e410dd30 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 --2- 172.21.15.32:0/3334232046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43e4101250 0x7f43e41016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 --2- 172.21.15.32:0/3334232046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e41070e0 0x7f43e41074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 -- 172.21.15.32:0/3334232046 >> 172.21.15.32:0/3334232046 conn(0x7f43e4076d00 msgr2=0x7f43e4077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:37.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 -- 172.21.15.32:0/3334232046 shutdown_connections 2024-04-16T05:42:37.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.035+0000 7f43e9d59640 1 -- 172.21.15.32:0/3334232046 wait complete. 2024-04-16T05:42:37.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e9d59640 1 Processor -- start 2024-04-16T05:42:37.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e9d59640 1 -- start start 2024-04-16T05:42:37.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e9d59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e4101250 0x7f43e419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e9d59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43e4101bf0 0x7f43e419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e9d59640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e41070e0 0x7f43e4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e9d59640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41112c0 con 0x7f43e41070e0 2024-04-16T05:42:37.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e9d59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4111140 con 0x7f43e4101250 2024-04-16T05:42:37.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e9d59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4111440 con 0x7f43e4101bf0 2024-04-16T05:42:37.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e41070e0 0x7f43e4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e41070e0 0x7f43e4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39528/0 (socket says 172.21.15.32:39528) 2024-04-16T05:42:37.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e3fff640 1 -- 172.21.15.32:0/170029824 learned_addr learned my addr 172.21.15.32:0/170029824 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:37.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43e4101bf0 0x7f43e419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e37fe640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e4101250 0x7f43e419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e3fff640 1 -- 172.21.15.32:0/170029824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43e4101bf0 msgr2=0x7f43e419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.036+0000 7f43e3fff640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43e4101bf0 0x7f43e419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e3fff640 1 -- 172.21.15.32:0/170029824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e4101250 msgr2=0x7f43e419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e3fff640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e4101250 0x7f43e419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e3fff640 1 -- 172.21.15.32:0/170029824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e4198fe0 con 0x7f43e41070e0 2024-04-16T05:42:37.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e3fff640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e41070e0 0x7f43e4198720 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0348f0 tx=0x7f43dc032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:37.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e0ff9640 1 -- 172.21.15.32:0/170029824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc041020 con 0x7f43e41070e0 2024-04-16T05:42:37.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e0ff9640 1 -- 172.21.15.32:0/170029824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc034cd0 con 0x7f43e41070e0 2024-04-16T05:42:37.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e9d59640 1 -- 172.21.15.32:0/170029824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e4199270 con 0x7f43e41070e0 2024-04-16T05:42:37.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e0ff9640 1 -- 172.21.15.32:0/170029824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc039040 con 0x7f43e41070e0 2024-04-16T05:42:37.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.037+0000 7f43e9d59640 1 -- 172.21.15.32:0/170029824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e41a53b0 con 0x7f43e41070e0 2024-04-16T05:42:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.039+0000 7f43e0ff9640 1 -- 172.21.15.32:0/170029824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43dc044020 con 0x7f43e41070e0 2024-04-16T05:42:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43eca22740 1 Processor -- start 2024-04-16T05:42:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43eca22740 1 -- start start 2024-04-16T05:42:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43eca22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43eca22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43eca22740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4e1350 0x55556d4e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43eca22740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55556d43d180 con 0x55556d4e1350 2024-04-16T05:42:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43eca22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55556d3da360 con 0x55556d4d7e50 2024-04-16T05:42:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43eca22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55556d439140 con 0x55556d4d8770 2024-04-16T05:42:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4e1350 0x55556d4e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43e0ff9640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f43cc0a3d20 0x7f43cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4e1350 0x55556d4e3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39536/0 (socket says 172.21.15.32:39536) 2024-04-16T05:42:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43e2ffd640 1 -- 172.21.15.32:0/2122612328 learned_addr learned my addr 172.21.15.32:0/2122612328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43e3fff640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f43cc0a3d20 0x7f43cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.040+0000 7f43e37fe640 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e2ffd640 1 -- 172.21.15.32:0/2122612328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 msgr2=0x55556d4e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e0ff9640 1 -- 172.21.15.32:0/170029824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f43dc0f2940 con 0x7f43e41070e0 2024-04-16T05:42:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e2ffd640 1 -- 172.21.15.32:0/2122612328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 msgr2=0x55556d4d8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4d8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e2ffd640 1 -- 172.21.15.32:0/2122612328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556d2197d0 con 0x55556d4e1350 2024-04-16T05:42:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e37fe640 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:42:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4e1350 0x55556d4e3740 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f43d80127c0 tx=0x7f43d800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34db1568e349a4de server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43e3fff640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f43cc0a3d20 0x7f43cc0a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0f0300 tx=0x7f43dc0900a0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43c9ffb640 1 -- 172.21.15.32:0/2122612328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8012e00 con 0x55556d4e1350 2024-04-16T05:42:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43c9ffb640 1 -- 172.21.15.32:0/2122612328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d8010c50 con 0x55556d4e1350 2024-04-16T05:42:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.041+0000 7f43c9ffb640 1 -- 172.21.15.32:0/2122612328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d801eb30 con 0x55556d4e1350 2024-04-16T05:42:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 -- 172.21.15.32:0/2122612328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4e1350 msgr2=0x55556d4e3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4e1350 0x55556d4e3740 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f43d80127c0 tx=0x7f43d800b7c0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 -- 172.21.15.32:0/2122612328 shutdown_connections 2024-04-16T05:42:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4e1350 0x55556d4e3740 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 --2- 172.21.15.32:0/2122612328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4d8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 -- 172.21.15.32:0/2122612328 >> 172.21.15.32:0/2122612328 conn(0x55556d4d2580 msgr2=0x55556d4d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 -- 172.21.15.32:0/2122612328 shutdown_connections 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 -- 172.21.15.32:0/2122612328 wait complete. 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 -- start start 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.042+0000 7f43eca22740 1 Processor -- start 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4de3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4e1350 0x55556d4dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55556d4e4140 con 0x55556d4d7e50 2024-04-16T05:42:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55556d3f62c0 con 0x55556d4e1350 2024-04-16T05:42:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55556d4e42c0 con 0x55556d4d8770 2024-04-16T05:42:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 10 client.0 _ll_register_callbacks cb 0x55556d3dd090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4de3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4de3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39552/0 (socket says 172.21.15.32:39552) 2024-04-16T05:42:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 -- 172.21.15.32:0/3708218035 learned_addr learned my addr 172.21.15.32:0/3708218035 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4e1350 0x55556d4dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e37fe640 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 -- 172.21.15.32:0/3708218035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 msgr2=0x55556d4de930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4de930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 -- 172.21.15.32:0/3708218035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4e1350 msgr2=0x55556d4dee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4e1350 0x55556d4dee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 -- 172.21.15.32:0/3708218035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556d14f3f0 con 0x55556d4d7e50 2024-04-16T05:42:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4de3f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f43d801e6b0 tx=0x7f43d801e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f73c01812c3db90 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43c2ffd640 1 -- 172.21.15.32:0/3708218035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8030060 con 0x55556d4d7e50 2024-04-16T05:42:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 1 -- 172.21.15.32:0/3708218035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d8010c50 con 0x55556d4d7e50 2024-04-16T05:42:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43c2ffd640 1 -- 172.21.15.32:0/3708218035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f43d8010740 con 0x55556d4d7e50 2024-04-16T05:42:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43c2ffd640 1 -- 172.21.15.32:0/3708218035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43d8024040 con 0x55556d4d7e50 2024-04-16T05:42:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 10 client.17481 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.043+0000 7f43eca22740 1 -- 172.21.15.32:0/3708218035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55556d10f0f0 con 0x55556d4d7e50 2024-04-16T05:42:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43c2ffd640 1 -- 172.21.15.32:0/3708218035 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d8010c50 con 0x55556d4d7e50 2024-04-16T05:42:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43c2ffd640 1 -- 172.21.15.32:0/3708218035 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f43d802a030 con 0x55556d4d7e50 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43eca22740 10 client.17481 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43eca22740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43eca22740 20 client.17481 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43eca22740 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55556d4f16b0 0x55556d4f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43eca22740 4 client.17481 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43eca22740 1 -- 172.21.15.32:0/3708218035 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55556d44aba0 con 0x55556d4f16b0 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.044+0000 7f43e37fe640 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55556d4f16b0 0x55556d4f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43c2ffd640 10 client.17481 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43e37fe640 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55556d4f16b0 0x55556d4f3aa0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b680ac9f998a4cff server_cookie=dfc2a093c6719ccb in_seq=0 out_seq=0 2024-04-16T05:42:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43c2ffd640 1 -- 172.21.15.32:0/3708218035 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55556d44aba0 con 0x55556d4f16b0 2024-04-16T05:42:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43c2ffd640 10 client.17481 handle_command_reply: tid=0 2024-04-16T05:42:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 client.17481 shutdown 2024-04-16T05:42:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 -- 172.21.15.32:0/3708218035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 msgr2=0x55556d4de3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4de3f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f43d801e6b0 tx=0x7f43d801e710 comp rx=0 tx=0).stop 2024-04-16T05:42:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 -- 172.21.15.32:0/3708218035 shutdown_connections 2024-04-16T05:42:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4e1350 0x55556d4dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55556d4f16b0 0x55556d4f3aa0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4d8770 0x55556d4de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43c2ffd640 0 client.17481 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 --2- 172.21.15.32:0/3708218035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4de3f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 -- 172.21.15.32:0/3708218035 >> 172.21.15.32:0/3708218035 conn(0x55556d4d2580 msgr2=0x55556d4d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 -- 172.21.15.32:0/3708218035 shutdown_connections 2024-04-16T05:42:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.045+0000 7f43eca22740 1 -- 172.21.15.32:0/3708218035 wait complete. 2024-04-16T05:42:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.046+0000 7f43eca22740 20 client.17481 trim_cache size 0 max 16384 2024-04-16T05:42:37.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.063+0000 7f43eca22740 1 Processor -- start 2024-04-16T05:42:37.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.063+0000 7f43eca22740 1 -- start start 2024-04-16T05:42:37.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43eca22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43eca22740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d8770 0x55556d4dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43eca22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 0x55556d4d40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43eca22740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41b1d70 con 0x55556d4d8770 2024-04-16T05:42:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43eca22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41af640 con 0x55556d4d7e50 2024-04-16T05:42:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43eca22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41ad4b0 con 0x55556d4e1350 2024-04-16T05:42:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d8770 0x55556d4dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d8770 0x55556d4dec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39556/0 (socket says 172.21.15.32:39556) 2024-04-16T05:42:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 -- 172.21.15.32:0/3375792905 learned_addr learned my addr 172.21.15.32:0/3375792905 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 -- 172.21.15.32:0/3375792905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 msgr2=0x55556d4d40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 0x55556d4d40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 -- 172.21.15.32:0/3375792905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 msgr2=0x55556d4ed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4ed030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 -- 172.21.15.32:0/3375792905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556d4d37b0 con 0x55556d4d8770 2024-04-16T05:42:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43e37fe640 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d8770 0x55556d4dec90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f43d4000c00 tx=0x7f43d400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d550676bce3859c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43c97fa640 1 -- 172.21.15.32:0/3375792905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40140d0 con 0x55556d4d8770 2024-04-16T05:42:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43c97fa640 1 -- 172.21.15.32:0/3375792905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4014270 con 0x55556d4d8770 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.064+0000 7f43c97fa640 1 -- 172.21.15.32:0/3375792905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4004730 con 0x55556d4d8770 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 -- 172.21.15.32:0/3375792905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d8770 msgr2=0x55556d4dec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d8770 0x55556d4dec90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f43d4000c00 tx=0x7f43d400b650 comp rx=0 tx=0).stop 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 -- 172.21.15.32:0/3375792905 shutdown_connections 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 0x55556d4d40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d8770 0x55556d4dec90 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 --2- 172.21.15.32:0/3375792905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d7e50 0x55556d4ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 -- 172.21.15.32:0/3375792905 >> 172.21.15.32:0/3375792905 conn(0x55556d4d2580 msgr2=0x55556d4d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 -- 172.21.15.32:0/3375792905 shutdown_connections 2024-04-16T05:42:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 -- 172.21.15.32:0/3375792905 wait complete. 2024-04-16T05:42:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.065+0000 7f43eca22740 1 -- start start 2024-04-16T05:42:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43eca22740 1 Processor -- start 2024-04-16T05:42:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43eca22740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43eca22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d8770 0x55556d4e9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43eca22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 0x55556d4e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43eca22740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41adb00 con 0x55556d4d7e50 2024-04-16T05:42:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43eca22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41aef70 con 0x55556d4d8770 2024-04-16T05:42:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43eca22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41ae1d0 con 0x55556d4e1350 2024-04-16T05:42:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43eca22740 10 client.0 _ll_register_callbacks cb 0x7f43d800be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:42:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4e91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39564/0 (socket says 172.21.15.32:39564) 2024-04-16T05:42:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 -- 172.21.15.32:0/2914124448 learned_addr learned my addr 172.21.15.32:0/2914124448 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:42:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 0x55556d4e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e37fe640 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d8770 0x55556d4e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 -- 172.21.15.32:0/2914124448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 msgr2=0x55556d4e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 0x55556d4e6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 -- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d8770 msgr2=0x55556d4e9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d8770 0x55556d4e9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e2ffd640 1 -- 172.21.15.32:0/2914124448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556d4e6e00 con 0x55556d4d7e50 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.066+0000 7f43e37fe640 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d8770 0x55556d4e9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43e2ffd640 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4e91e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f43d801f730 tx=0x7f43d800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48f385bfb412b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:42:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43c2ffd640 1 -- 172.21.15.32:0/2914124448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8030040 con 0x55556d4d7e50 2024-04-16T05:42:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43c2ffd640 1 -- 172.21.15.32:0/2914124448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9438+0+0 (secure 0 0 0) 0x7f43d8010740 con 0x55556d4d7e50 2024-04-16T05:42:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 1 -- 172.21.15.32:0/2914124448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d4014270 con 0x55556d4d7e50 2024-04-16T05:42:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43c2ffd640 1 -- 172.21.15.32:0/2914124448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43d8044050 con 0x55556d4d7e50 2024-04-16T05:42:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 10 client.17487 fetch_fsmap learned FSMap version 25 2024-04-16T05:42:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 1 -- 172.21.15.32:0/2914124448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55556d557490 con 0x55556d4d7e50 2024-04-16T05:42:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43c2ffd640 1 -- 172.21.15.32:0/2914124448 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4014270 con 0x55556d4d7e50 2024-04-16T05:42:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43c2ffd640 1 -- 172.21.15.32:0/2914124448 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f43d803a030 con 0x55556d4d7e50 2024-04-16T05:42:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 10 client.17487 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:42:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:42:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 20 client.17487 populate_metadata read hostname 'smithi032' 2024-04-16T05:42:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55556d557ac0 0x55556d559ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:42:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 4 client.17487 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:42:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.067+0000 7f43eca22740 1 -- 172.21.15.32:0/2914124448 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55556d444a10 con 0x55556d557ac0 2024-04-16T05:42:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43e37fe640 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55556d557ac0 0x55556d559ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:42:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43e37fe640 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55556d557ac0 0x55556d559ee0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8ae2eca848619b server_cookie=c652fc894b3a7685 in_seq=0 out_seq=0 2024-04-16T05:42:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43c2ffd640 10 client.17487 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43c2ffd640 1 -- 172.21.15.32:0/2914124448 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55556d444a10 con 0x55556d557ac0 2024-04-16T05:42:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43c2ffd640 10 client.17487 handle_command_reply: tid=0 2024-04-16T05:42:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43eca22740 1 client.17487 shutdown 2024-04-16T05:42:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43eca22740 1 -- 172.21.15.32:0/2914124448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 msgr2=0x55556d4e91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43eca22740 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4e91e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f43d801f730 tx=0x7f43d800b3b0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43eca22740 1 -- 172.21.15.32:0/2914124448 shutdown_connections 2024-04-16T05:42:37.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.068+0000 7f43eca22740 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55556d557ac0 0x55556d559ee0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43eca22740 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55556d4e1350 0x55556d4e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43eca22740 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55556d4d8770 0x55556d4e9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43c2ffd640 0 client.17487 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:42:37.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43eca22740 1 --2- 172.21.15.32:0/2914124448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556d4d7e50 0x55556d4e91e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43eca22740 1 -- 172.21.15.32:0/2914124448 >> 172.21.15.32:0/2914124448 conn(0x55556d4d2580 msgr2=0x55556d4d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:37.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43eca22740 1 -- 172.21.15.32:0/2914124448 shutdown_connections 2024-04-16T05:42:37.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43eca22740 1 -- 172.21.15.32:0/2914124448 wait complete. 2024-04-16T05:42:37.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43eca22740 20 client.17487 trim_cache size 0 max 16384 2024-04-16T05:42:37.080 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:42:37.080 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:42:37.080 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:42:37.081 INFO:teuthology.orchestra.run.smithi032.stdout: "fcbda044-c1bc-48d5-80c6-9fe2d05aa69b": { 2024-04-16T05:42:37.081 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:42:37.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "fcbda044-c1bc-48d5-80c6-9fe2d05aa69b", 2024-04-16T05:42:37.081 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:42:37.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:42:37.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:42:37.081 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:42:37.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 -- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f43cc0a3d20 msgr2=0x7f43cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f43cc0a3d20 0x7f43cc0a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0f0300 tx=0x7f43dc0900a0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 -- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e41070e0 msgr2=0x7f43e4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:42:37.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e41070e0 0x7f43e4198720 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0348f0 tx=0x7f43dc032ec0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 -- 172.21.15.32:0/170029824 shutdown_connections 2024-04-16T05:42:37.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f43cc0a3d20 0x7f43cc0a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e41070e0 0x7f43e4198720 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43e4101bf0 0x7f43e419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 --2- 172.21.15.32:0/170029824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e4101250 0x7f43e419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:42:37.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 -- 172.21.15.32:0/170029824 >> 172.21.15.32:0/170029824 conn(0x7f43e4076d00 msgr2=0x7f43e4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:42:37.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 -- 172.21.15.32:0/170029824 shutdown_connections 2024-04-16T05:42:37.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:42:37.069+0000 7f43e9d59640 1 -- 172.21.15.32:0/170029824 wait complete. 2024-04-16T05:42:37.085 INFO:tasks.cephfs.filesystem:scrub status for tag:fcbda044-c1bc-48d5-80c6-9fe2d05aa69b - {'path': '/', 'tag': 'fcbda044-c1bc-48d5-80c6-9fe2d05aa69b', 'options': 'recursive,force'} 2024-04-16T05:42:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:37 smithi134 ceph-mon[36153]: pgmap v235: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:37 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:42:38.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:37 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:38.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:37 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:42:38.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:37 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:38.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:37 smithi082 ceph-mon[36195]: pgmap v235: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:38.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:37 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:42:38.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:37 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:38.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:37 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:42:38.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:37 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:38.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:37 smithi032 ceph-mon[32351]: pgmap v235: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:38.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:37 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:42:38.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:37 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:38.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:37 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:42:38.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:37 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:42:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:39 smithi134 ceph-mon[36153]: pgmap v236: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:40.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:39 smithi082 ceph-mon[36195]: pgmap v236: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:40.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:39 smithi032 ceph-mon[32351]: pgmap v236: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:40 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:42:41.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:40 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:42:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:40 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:42:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:41 smithi134 ceph-mon[36153]: pgmap v237: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:42.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:41 smithi032 ceph-mon[32351]: pgmap v237: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:42.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:41 smithi082 ceph-mon[36195]: pgmap v237: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:43 smithi134 ceph-mon[36153]: pgmap v238: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:44.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:43 smithi082 ceph-mon[36195]: pgmap v238: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:43 smithi032 ceph-mon[32351]: pgmap v238: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:45 smithi134 ceph-mon[36153]: pgmap v239: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:46.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:45 smithi082 ceph-mon[36195]: pgmap v239: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:46.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:45 smithi032 ceph-mon[32351]: pgmap v239: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:48.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:47 smithi134 ceph-mon[36153]: pgmap v240: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:48.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:47 smithi082 ceph-mon[36195]: pgmap v240: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:48.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:47 smithi032 ceph-mon[32351]: pgmap v240: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:42:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:49 smithi134 ceph-mon[36153]: pgmap v241: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:50.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:49 smithi082 ceph-mon[36195]: pgmap v241: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:50.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:49 smithi032 ceph-mon[32351]: pgmap v241: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:42:52.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:51 smithi134 ceph-mon[36153]: pgmap v242: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:52.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:51 smithi082 ceph-mon[36195]: pgmap v242: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:52.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:51 smithi032 ceph-mon[32351]: pgmap v242: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:53 smithi134 ceph-mon[36153]: pgmap v243: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:54.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:53 smithi082 ceph-mon[36195]: pgmap v243: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:54.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:53 smithi032 ceph-mon[32351]: pgmap v243: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:42:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:55 smithi134 ceph-mon[36153]: pgmap v244: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:56.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:55 smithi082 ceph-mon[36195]: pgmap v244: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:56.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:55 smithi032 ceph-mon[32351]: pgmap v244: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:57 smithi134 ceph-mon[36153]: pgmap v245: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:58.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:57 smithi082 ceph-mon[36195]: pgmap v245: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:42:58.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:57 smithi032 ceph-mon[32351]: pgmap v245: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:42:59 smithi134 ceph-mon[36153]: pgmap v246: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:00.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:42:59 smithi082 ceph-mon[36195]: pgmap v246: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:00.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:42:59 smithi032 ceph-mon[32351]: pgmap v246: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:01 smithi134 ceph-mon[36153]: pgmap v247: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:02.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:01 smithi082 ceph-mon[36195]: pgmap v247: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:02.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:01 smithi032 ceph-mon[32351]: pgmap v247: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:04.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:03 smithi134 ceph-mon[36153]: pgmap v248: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:43:04.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:03 smithi082 ceph-mon[36195]: pgmap v248: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:43:04.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:03 smithi032 ceph-mon[32351]: pgmap v248: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:43:06.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:05 smithi134 ceph-mon[36153]: pgmap v249: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:43:06.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:05 smithi082 ceph-mon[36195]: pgmap v249: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:43:06.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:05 smithi032 ceph-mon[32351]: pgmap v249: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:43:07.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:43:07.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:43:07.117 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:06 smithi032 ceph-mon[32351]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T05:43:07.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:06 smithi134 ceph-mon[36153]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T05:43:07.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:06 smithi082 ceph-mon[36195]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T05:43:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 -- 172.21.15.32:0/1407345808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934106210 msgr2=0x7f693410dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 --2- 172.21.15.32:0/1407345808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934106210 0x7f693410dde0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f692c009900 tx=0x7f692c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:07.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 -- 172.21.15.32:0/1407345808 shutdown_connections 2024-04-16T05:43:07.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 --2- 172.21.15.32:0/1407345808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934106210 0x7f693410dde0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 --2- 172.21.15.32:0/1407345808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6934101af0 0x7f6934105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 --2- 172.21.15.32:0/1407345808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6934101140 0x7f6934101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 -- 172.21.15.32:0/1407345808 >> 172.21.15.32:0/1407345808 conn(0x7f6934078000 msgr2=0x7f6934078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 -- 172.21.15.32:0/1407345808 shutdown_connections 2024-04-16T05:43:07.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 -- 172.21.15.32:0/1407345808 wait complete. 2024-04-16T05:43:07.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.209+0000 7f693a149640 1 Processor -- start 2024-04-16T05:43:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f693a149640 1 -- start start 2024-04-16T05:43:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f693a149640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934101140 0x7f693419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f693a149640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6934106210 0x7f693419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f693a149640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69341986a0 0x7f6934198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f693a149640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69341112d0 con 0x7f6934101140 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f693a149640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6934111150 con 0x7f6934106210 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f693a149640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6934111450 con 0x7f69341986a0 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934101140 0x7f693419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934101140 0x7f693419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34248/0 (socket says 172.21.15.32:34248) 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 -- 172.21.15.32:0/3764572772 learned_addr learned my addr 172.21.15.32:0/3764572772 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f6932ffd640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6934106210 0x7f693419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f6933fff640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69341986a0 0x7f6934198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 -- 172.21.15.32:0/3764572772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69341986a0 msgr2=0x7f6934198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69341986a0 0x7f6934198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 -- 172.21.15.32:0/3764572772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6934106210 msgr2=0x7f693419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6934106210 0x7f693419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 -- 172.21.15.32:0/3764572772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69341992a0 con 0x7f6934101140 2024-04-16T05:43:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f6932ffd640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6934106210 0x7f693419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f6933fff640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69341986a0 0x7f6934198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.210+0000 7f69337fe640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934101140 0x7f693419e5e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f692400eb90 tx=0x7f692400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.211+0000 7f6930ff9640 1 -- 172.21.15.32:0/3764572772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924004d20 con 0x7f6934101140 2024-04-16T05:43:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.211+0000 7f6930ff9640 1 -- 172.21.15.32:0/3764572772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692400ee90 con 0x7f6934101140 2024-04-16T05:43:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.211+0000 7f693a149640 1 -- 172.21.15.32:0/3764572772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69341a5380 con 0x7f6934101140 2024-04-16T05:43:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.211+0000 7f6930ff9640 1 -- 172.21.15.32:0/3764572772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924010830 con 0x7f6934101140 2024-04-16T05:43:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.211+0000 7f693a149640 1 -- 172.21.15.32:0/3764572772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69341a5730 con 0x7f6934101140 2024-04-16T05:43:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.213+0000 7f6930ff9640 1 -- 172.21.15.32:0/3764572772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6924005020 con 0x7f6934101140 2024-04-16T05:43:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.213+0000 7f693ce0b740 1 Processor -- start 2024-04-16T05:43:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.213+0000 7f693ce0b740 1 -- start start 2024-04-16T05:43:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.213+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 0x558d75f3f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 0x558d75f47e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f693ce0b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558d75ea4180 con 0x558d75f48350 2024-04-16T05:43:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f693ce0b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558d75e41360 con 0x558d75f3ee50 2024-04-16T05:43:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f693ce0b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d75ea0140 con 0x558d75f3f770 2024-04-16T05:43:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6930ff9640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f69200a3d20 0x7f69200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34258/0 (socket says 172.21.15.32:34258) 2024-04-16T05:43:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 -- 172.21.15.32:0/1782942243 learned_addr learned my addr 172.21.15.32:0/1782942243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6933fff640 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 0x558d75f3f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f69337fe640 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 0x558d75f47e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6933fff640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f69200a3d20 0x7f69200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6930ff9640 1 -- 172.21.15.32:0/3764572772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f69240d13f0 con 0x7f6934101140 2024-04-16T05:43:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 -- 172.21.15.32:0/1782942243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 msgr2=0x558d75f47e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 0x558d75f47e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 -- 172.21.15.32:0/1782942243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 msgr2=0x558d75f3f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 0x558d75f3f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 -- 172.21.15.32:0/1782942243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d75c807d0 con 0x558d75f48350 2024-04-16T05:43:07.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6932ffd640 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4a740 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f691c009b60 tx=0x7f691c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25007b49f13b2aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.214+0000 7f6933fff640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f69200a3d20 0x7f69200a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f692c02c170 tx=0x7f692c005fd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f6929ffb640 1 -- 172.21.15.32:0/1782942243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c016070 con 0x558d75f48350 2024-04-16T05:43:07.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f6929ffb640 1 -- 172.21.15.32:0/1782942243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f691c009e50 con 0x558d75f48350 2024-04-16T05:43:07.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f6929ffb640 1 -- 172.21.15.32:0/1782942243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c011470 con 0x558d75f48350 2024-04-16T05:43:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 -- 172.21.15.32:0/1782942243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 msgr2=0x558d75f4a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4a740 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f691c009b60 tx=0x7f691c00e6c0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 -- 172.21.15.32:0/1782942243 shutdown_connections 2024-04-16T05:43:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4a740 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 0x558d75f47e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 --2- 172.21.15.32:0/1782942243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 0x558d75f3f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 -- 172.21.15.32:0/1782942243 >> 172.21.15.32:0/1782942243 conn(0x558d75f39580 msgr2=0x558d75f39960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 -- 172.21.15.32:0/1782942243 shutdown_connections 2024-04-16T05:43:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.215+0000 7f693ce0b740 1 -- 172.21.15.32:0/1782942243 wait complete. 2024-04-16T05:43:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 1 -- start start 2024-04-16T05:43:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 1 Processor -- start 2024-04-16T05:43:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 0x558d75f453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 0x558d75f45920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f45e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558d75f4b140 con 0x558d75f48350 2024-04-16T05:43:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558d75e5d2c0 con 0x558d75f3ee50 2024-04-16T05:43:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d75f4b2c0 con 0x558d75f3f770 2024-04-16T05:43:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f693ce0b740 10 client.0 _ll_register_callbacks cb 0x558d75e44090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f6933fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f45e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f6933fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f45e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34274/0 (socket says 172.21.15.32:34274) 2024-04-16T05:43:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f6933fff640 1 -- 172.21.15.32:0/3528466802 learned_addr learned my addr 172.21.15.32:0/3528466802 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f6932ffd640 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 0x558d75f45920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.216+0000 7f69337fe640 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 0x558d75f453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6933fff640 1 -- 172.21.15.32:0/3528466802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 msgr2=0x558d75f45920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6933fff640 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 0x558d75f45920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6933fff640 1 -- 172.21.15.32:0/3528466802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 msgr2=0x558d75f453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6933fff640 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 0x558d75f453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6933fff640 1 -- 172.21.15.32:0/3528466802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d75bb63f0 con 0x558d75f48350 2024-04-16T05:43:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6933fff640 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f45e60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f692c0042d0 tx=0x7f692c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ceffd17abe3d6de server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6912ffd640 1 -- 172.21.15.32:0/3528466802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c003b70 con 0x558d75f48350 2024-04-16T05:43:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6912ffd640 1 -- 172.21.15.32:0/3528466802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f692c03a040 con 0x558d75f48350 2024-04-16T05:43:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f693ce0b740 1 -- 172.21.15.32:0/3528466802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691c009e50 con 0x558d75f48350 2024-04-16T05:43:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6912ffd640 1 -- 172.21.15.32:0/3528466802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f692c05a050 con 0x558d75f48350 2024-04-16T05:43:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f693ce0b740 10 client.17499 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f693ce0b740 1 -- 172.21.15.32:0/3528466802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d75b760f0 con 0x558d75f48350 2024-04-16T05:43:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6912ffd640 1 -- 172.21.15.32:0/3528466802 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f691c009e50 con 0x558d75f48350 2024-04-16T05:43:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.217+0000 7f6912ffd640 1 -- 172.21.15.32:0/3528466802 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f692c034440 con 0x558d75f48350 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f693ce0b740 10 client.17499 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f693ce0b740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f693ce0b740 20 client.17499 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f693ce0b740 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558d75f586e0 0x558d75f5ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f693ce0b740 4 client.17499 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f693ce0b740 1 -- 172.21.15.32:0/3528466802 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d75eb1ba0 con 0x558d75f586e0 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f69337fe640 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558d75f586e0 0x558d75f5ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f6912ffd640 10 client.17499 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.218+0000 7f69337fe640 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558d75f586e0 0x558d75f5ab30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78efe1bdb60e3a2 server_cookie=bd645038940f2af0 in_seq=0 out_seq=0 2024-04-16T05:43:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f6912ffd640 1 -- 172.21.15.32:0/3528466802 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d75eb1ba0 con 0x558d75f586e0 2024-04-16T05:43:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f6912ffd640 10 client.17499 handle_command_reply: tid=0 2024-04-16T05:43:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 client.17499 shutdown 2024-04-16T05:43:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 -- 172.21.15.32:0/3528466802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 msgr2=0x558d75f45e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f45e60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f692c0042d0 tx=0x7f692c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 -- 172.21.15.32:0/3528466802 shutdown_connections 2024-04-16T05:43:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f45e60 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558d75f586e0 0x558d75f5ab30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3f770 0x558d75f45920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 --2- 172.21.15.32:0/3528466802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3ee50 0x558d75f453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f6912ffd640 0 client.17499 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 -- 172.21.15.32:0/3528466802 >> 172.21.15.32:0/3528466802 conn(0x558d75f39580 msgr2=0x558d75f3b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 -- 172.21.15.32:0/3528466802 shutdown_connections 2024-04-16T05:43:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 1 -- 172.21.15.32:0/3528466802 wait complete. 2024-04-16T05:43:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.219+0000 7f693ce0b740 20 client.17499 trim_cache size 0 max 16384 2024-04-16T05:43:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.237+0000 7f693ce0b740 1 Processor -- start 2024-04-16T05:43:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.237+0000 7f693ce0b740 1 -- start start 2024-04-16T05:43:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f47b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f54000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f60400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f693ce0b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69341b1e30 con 0x558d75f48350 2024-04-16T05:43:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f693ce0b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f69341af700 con 0x558d75f3f770 2024-04-16T05:43:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f693ce0b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69341ad570 con 0x558d75f3ee50 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f60400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f60400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34290/0 (socket says 172.21.15.32:34290) 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 -- 172.21.15.32:0/1101385410 learned_addr learned my addr 172.21.15.32:0/1101385410 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6932ffd640 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f54000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f69337fe640 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f47b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 -- 172.21.15.32:0/1101385410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 msgr2=0x558d75f47b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f47b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 -- 172.21.15.32:0/1101385410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 msgr2=0x558d75f54000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f54000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 -- 172.21.15.32:0/1101385410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d75f34e40 con 0x558d75f48350 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f69337fe640 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f47b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f6933fff640 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f60400 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f692c0092a0 tx=0x7f692c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fb59c6e5991eac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f69297fa640 1 -- 172.21.15.32:0/1101385410 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c02cc30 con 0x558d75f48350 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f69297fa640 1 -- 172.21.15.32:0/1101385410 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c003ea0 con 0x558d75f48350 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.238+0000 7f69297fa640 1 -- 172.21.15.32:0/1101385410 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c043740 con 0x558d75f48350 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 -- 172.21.15.32:0/1101385410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 msgr2=0x558d75f60400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f60400 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f692c0092a0 tx=0x7f692c005c00 comp rx=0 tx=0).stop 2024-04-16T05:43:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 -- 172.21.15.32:0/1101385410 shutdown_connections 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f60400 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f54000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 --2- 172.21.15.32:0/1101385410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f47b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 -- 172.21.15.32:0/1101385410 >> 172.21.15.32:0/1101385410 conn(0x558d75f39580 msgr2=0x558d75f38760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 -- 172.21.15.32:0/1101385410 shutdown_connections 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 -- 172.21.15.32:0/1101385410 wait complete. 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.239+0000 7f693ce0b740 1 -- start start 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f693ce0b740 1 Processor -- start 2024-04-16T05:43:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f4df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f4e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f693ce0b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f693ce0b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69341adbc0 con 0x558d75f48350 2024-04-16T05:43:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f693ce0b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f69341af030 con 0x558d75f3f770 2024-04-16T05:43:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f693ce0b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69341ae290 con 0x558d75f3ee50 2024-04-16T05:43:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f693ce0b740 10 client.0 _ll_register_callbacks cb 0x7f692c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6933fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6933fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34300/0 (socket says 172.21.15.32:34300) 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6933fff640 1 -- 172.21.15.32:0/2780393293 learned_addr learned my addr 172.21.15.32:0/2780393293 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6932ffd640 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f4e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f69337fe640 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f4df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6933fff640 1 -- 172.21.15.32:0/2780393293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 msgr2=0x558d75f4df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6933fff640 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f4df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6933fff640 1 -- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 msgr2=0x558d75f4e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6933fff640 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f4e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6933fff640 1 -- 172.21.15.32:0/2780393293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d75fbdc80 con 0x558d75f48350 2024-04-16T05:43:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.240+0000 7f6932ffd640 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f4e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f6933fff640 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4e980 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f692c060040 tx=0x7f692c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17df146e8df0cd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f6912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f6912ffd640 1 -- 172.21.15.32:0/2780393293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c034650 con 0x558d75f48350 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f6912ffd640 1 -- 172.21.15.32:0/2780393293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f692c0039c0 con 0x558d75f48350 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 1 -- 172.21.15.32:0/2780393293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f692c003ea0 con 0x558d75f48350 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f6912ffd640 1 -- 172.21.15.32:0/2780393293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f692c003bc0 con 0x558d75f48350 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 10 client.17505 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 1 -- 172.21.15.32:0/2780393293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d75fbeea0 con 0x558d75f48350 2024-04-16T05:43:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f6912ffd640 1 -- 172.21.15.32:0/2780393293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c003ea0 con 0x558d75f48350 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f6912ffd640 1 -- 172.21.15.32:0/2780393293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f692c05e550 con 0x558d75f48350 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 10 client.17505 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 20 client.17505 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558d75fbf500 0x558d75fc1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 4 client.17505 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.241+0000 7f693ce0b740 1 -- 172.21.15.32:0/2780393293 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d75eaba10 con 0x558d75fbf500 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.242+0000 7f69337fe640 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558d75fbf500 0x558d75fc1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.242+0000 7f69337fe640 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558d75fbf500 0x558d75fc1920 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af85a0c81e02315 server_cookie=3aa7010ce9d0b438 in_seq=0 out_seq=0 2024-04-16T05:43:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.242+0000 7f6912ffd640 10 client.17505 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.242+0000 7f6912ffd640 1 -- 172.21.15.32:0/2780393293 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558d75eaba10 con 0x558d75fbf500 2024-04-16T05:43:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.242+0000 7f6912ffd640 10 client.17505 handle_command_reply: tid=0 2024-04-16T05:43:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.242+0000 7f693ce0b740 1 client.17505 shutdown 2024-04-16T05:43:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 -- 172.21.15.32:0/2780393293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 msgr2=0x558d75f4e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4e980 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f692c060040 tx=0x7f692c040000 comp rx=0 tx=0).stop 2024-04-16T05:43:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 -- 172.21.15.32:0/2780393293 shutdown_connections 2024-04-16T05:43:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d75f48350 0x558d75f4e980 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558d75f3f770 0x558d75f4e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x558d75fbf500 0x558d75fc1920 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 --2- 172.21.15.32:0/2780393293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d75f3ee50 0x558d75f4df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f6912ffd640 0 client.17505 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 -- 172.21.15.32:0/2780393293 >> 172.21.15.32:0/2780393293 conn(0x558d75f39580 msgr2=0x558d75f45380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 -- 172.21.15.32:0/2780393293 shutdown_connections 2024-04-16T05:43:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 1 -- 172.21.15.32:0/2780393293 wait complete. 2024-04-16T05:43:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693ce0b740 20 client.17505 trim_cache size 0 max 16384 2024-04-16T05:43:07.252 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:43:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:43:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:43:07.252 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:43:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 -- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f69200a3d20 msgr2=0x7f69200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f69200a3d20 0x7f69200a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f692c02c170 tx=0x7f692c005fd0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 -- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934101140 msgr2=0x7f693419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934101140 0x7f693419e5e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f692400eb90 tx=0x7f692400c6a0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 -- 172.21.15.32:0/3764572772 shutdown_connections 2024-04-16T05:43:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69341986a0 0x7f6934198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6934106210 0x7f693419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f69200a3d20 0x7f69200a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 --2- 172.21.15.32:0/3764572772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934101140 0x7f693419e5e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 -- 172.21.15.32:0/3764572772 >> 172.21.15.32:0/3764572772 conn(0x7f6934078000 msgr2=0x7f6934104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 -- 172.21.15.32:0/3764572772 shutdown_connections 2024-04-16T05:43:07.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.243+0000 7f693a149640 1 -- 172.21.15.32:0/3764572772 wait complete. 2024-04-16T05:43:07.255 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:43:07.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:43:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.400+0000 7fda8e635640 1 -- 172.21.15.32:0/3572554492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 msgr2=0x7fda8810ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.400+0000 7fda8e635640 1 --2- 172.21.15.32:0/3572554492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 0x7fda8810ea60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fda80009ca0 tx=0x7fda8002c180 comp rx=0 tx=0).stop 2024-04-16T05:43:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.400+0000 7fda8e635640 1 -- 172.21.15.32:0/3572554492 shutdown_connections 2024-04-16T05:43:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.400+0000 7fda8e635640 1 --2- 172.21.15.32:0/3572554492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 0x7fda8810ea60 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.400+0000 7fda8e635640 1 --2- 172.21.15.32:0/3572554492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda88103b80 0x7fda8810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.400+0000 7fda8e635640 1 --2- 172.21.15.32:0/3572554492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda88103260 0x7fda88103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.400+0000 7fda8e635640 1 -- 172.21.15.32:0/3572554492 >> 172.21.15.32:0/3572554492 conn(0x7fda88077280 msgr2=0x7fda88077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.401+0000 7fda8e635640 1 -- 172.21.15.32:0/3572554492 shutdown_connections 2024-04-16T05:43:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.401+0000 7fda8e635640 1 -- 172.21.15.32:0/3572554492 wait complete. 2024-04-16T05:43:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.401+0000 7fda8e635640 1 Processor -- start 2024-04-16T05:43:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.401+0000 7fda8e635640 1 -- start start 2024-04-16T05:43:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.401+0000 7fda8e635640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda88103260 0x7fda8819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.401+0000 7fda8e635640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda88103b80 0x7fda8819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8e635640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 0x7fda881986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8e635640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda881123a0 con 0x7fda8810c630 2024-04-16T05:43:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8e635640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fda88112220 con 0x7fda88103260 2024-04-16T05:43:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8e635640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda88112520 con 0x7fda88103b80 2024-04-16T05:43:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 0x7fda881986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 0x7fda881986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34308/0 (socket says 172.21.15.32:34308) 2024-04-16T05:43:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 -- 172.21.15.32:0/2892308282 learned_addr learned my addr 172.21.15.32:0/2892308282 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda87fff640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda88103260 0x7fda8819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda877fe640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda88103b80 0x7fda8819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 -- 172.21.15.32:0/2892308282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda88103b80 msgr2=0x7fda8819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda88103b80 0x7fda8819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 -- 172.21.15.32:0/2892308282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda88103260 msgr2=0x7fda8819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda88103260 0x7fda8819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 -- 172.21.15.32:0/2892308282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda88198f60 con 0x7fda8810c630 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8cbd2640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 0x7fda881986a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fda80004af0 tx=0x7fda80035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda857fa640 1 -- 172.21.15.32:0/2892308282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80039040 con 0x7fda8810c630 2024-04-16T05:43:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda857fa640 1 -- 172.21.15.32:0/2892308282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda80044070 con 0x7fda8810c630 2024-04-16T05:43:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda857fa640 1 -- 172.21.15.32:0/2892308282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda800328d0 con 0x7fda8810c630 2024-04-16T05:43:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda881991f0 con 0x7fda8810c630 2024-04-16T05:43:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.402+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda881a5380 con 0x7fda8810c630 2024-04-16T05:43:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.403+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8806aea0 con 0x7fda8810c630 2024-04-16T05:43:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.404+0000 7fda857fa640 1 -- 172.21.15.32:0/2892308282 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fda80003b70 con 0x7fda8810c630 2024-04-16T05:43:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.405+0000 7fda857fa640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fda700a3d20 0x7fda700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.405+0000 7fda87fff640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fda700a3d20 0x7fda700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.405+0000 7fda857fa640 1 -- 172.21.15.32:0/2892308282 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7fda800f3640 con 0x7fda8810c630 2024-04-16T05:43:07.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.406+0000 7fda87fff640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fda700a3d20 0x7fda700a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda7800bbe0 tx=0x7fda7800b390 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.409+0000 7fda857fa640 1 -- 172.21.15.32:0/2892308282 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda800bf9a0 con 0x7fda8810c630 2024-04-16T05:43:07.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.634+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda88069270 con 0x7fda8810c630 2024-04-16T05:43:07.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.634+0000 7fda857fa640 1 -- 172.21.15.32:0/2892308282 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7fda88069270 con 0x7fda8810c630 2024-04-16T05:43:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:43:07.633 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:43:07.634 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:43:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fda700a3d20 msgr2=0x7fda700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fda700a3d20 0x7fda700a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda7800bbe0 tx=0x7fda7800b390 comp rx=0 tx=0).stop 2024-04-16T05:43:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 msgr2=0x7fda881986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 0x7fda881986a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fda80004af0 tx=0x7fda80035d80 comp rx=0 tx=0).stop 2024-04-16T05:43:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 shutdown_connections 2024-04-16T05:43:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fda700a3d20 0x7fda700a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8810c630 0x7fda881986a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda88103b80 0x7fda8819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 --2- 172.21.15.32:0/2892308282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda88103260 0x7fda8819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 >> 172.21.15.32:0/2892308282 conn(0x7fda88077280 msgr2=0x7fda880fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 shutdown_connections 2024-04-16T05:43:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.636+0000 7fda8e635640 1 -- 172.21.15.32:0/2892308282 wait complete. 2024-04-16T05:43:07.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:43:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 -- 172.21.15.32:0/2493753985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec101ae0 msgr2=0x7feeec105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 --2- 172.21.15.32:0/2493753985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec101ae0 0x7feeec105b20 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7feee0009900 tx=0x7feee002bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 -- 172.21.15.32:0/2493753985 shutdown_connections 2024-04-16T05:43:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 --2- 172.21.15.32:0/2493753985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeec106200 0x7feeec10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 --2- 172.21.15.32:0/2493753985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec101ae0 0x7feeec105b20 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 --2- 172.21.15.32:0/2493753985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeec101130 0x7feeec101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 -- 172.21.15.32:0/2493753985 >> 172.21.15.32:0/2493753985 conn(0x7feeec0780b0 msgr2=0x7feeec0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 -- 172.21.15.32:0/2493753985 shutdown_connections 2024-04-16T05:43:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.755+0000 7feef0e0d640 1 -- 172.21.15.32:0/2493753985 wait complete. 2024-04-16T05:43:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feef0e0d640 1 Processor -- start 2024-04-16T05:43:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feef0e0d640 1 -- start start 2024-04-16T05:43:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feef0e0d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeec101130 0x7feeec19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feef0e0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeec106200 0x7feeec19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feef0e0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec198730 0x7feeec198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feef0e0d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1123d0 con 0x7feeec198730 2024-04-16T05:43:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feef0e0d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feeec112250 con 0x7feeec106200 2024-04-16T05:43:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feef0e0d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feeec112550 con 0x7feeec101130 2024-04-16T05:43:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec198730 0x7feeec198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec198730 0x7feeec198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34324/0 (socket says 172.21.15.32:34324) 2024-04-16T05:43:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 -- 172.21.15.32:0/3817025763 learned_addr learned my addr 172.21.15.32:0/3817025763 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeea59c640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeec101130 0x7feeec19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feee9d9b640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeec106200 0x7feeec19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 -- 172.21.15.32:0/3817025763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeec101130 msgr2=0x7feeec19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeec101130 0x7feeec19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 -- 172.21.15.32:0/3817025763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeec106200 msgr2=0x7feeec19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeec106200 0x7feeec19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 -- 172.21.15.32:0/3817025763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec199330 con 0x7feeec198730 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.756+0000 7feeead9d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec198730 0x7feeec198bb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7feee400eb10 tx=0x7feee400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.757+0000 7feedb7fe640 1 -- 172.21.15.32:0/3817025763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4004b20 con 0x7feeec198730 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.757+0000 7feedb7fe640 1 -- 172.21.15.32:0/3817025763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee4004cc0 con 0x7feeec198730 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.757+0000 7feef0e0d640 1 -- 172.21.15.32:0/3817025763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec1a53c0 con 0x7feeec198730 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.757+0000 7feedb7fe640 1 -- 172.21.15.32:0/3817025763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4010760 con 0x7feeec198730 2024-04-16T05:43:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.757+0000 7feef0e0d640 1 -- 172.21.15.32:0/3817025763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec1a5770 con 0x7feeec198730 2024-04-16T05:43:07.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.759+0000 7feedb7fe640 1 -- 172.21.15.32:0/3817025763 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feee4005020 con 0x7feeec198730 2024-04-16T05:43:07.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.759+0000 7feef39fc740 1 Processor -- start 2024-04-16T05:43:07.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.759+0000 7feef39fc740 1 -- start start 2024-04-16T05:43:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feef39fc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 0x55c1045c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feef39fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feef39fc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feef39fc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c10452a0e0 con 0x55c1045d2770 2024-04-16T05:43:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feef39fc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c1044c7520 con 0x55c1045c9270 2024-04-16T05:43:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feef39fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1045260a0 con 0x55c1045c9b90 2024-04-16T05:43:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34328/0 (socket says 172.21.15.32:34328) 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 -- 172.21.15.32:0/2371307276 learned_addr learned my addr 172.21.15.32:0/2371307276 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feedb7fe640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feed40a3d20 0x7feed40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 0x55c1045c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feeea59c640 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feeead9d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feed40a3d20 0x7feed40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 -- 172.21.15.32:0/2371307276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 msgr2=0x55c1045d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feedb7fe640 1 -- 172.21.15.32:0/3817025763 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7feee40d1260 con 0x7feeec198730 2024-04-16T05:43:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 -- 172.21.15.32:0/2371307276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 msgr2=0x55c1045c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 0x55c1045c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 -- 172.21.15.32:0/2371307276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1044ef8d0 con 0x55c1045d2770 2024-04-16T05:43:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feee9d9b640 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d4b60 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7feee002c820 tx=0x7feee0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aec1a3a19adeede server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.760+0000 7feeead9d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feed40a3d20 0x7feed40a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feee409cb10 tx=0x7feee400efd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feed8ff9640 1 -- 172.21.15.32:0/2371307276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee002cb20 con 0x55c1045d2770 2024-04-16T05:43:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feed8ff9640 1 -- 172.21.15.32:0/2371307276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee002ccc0 con 0x55c1045d2770 2024-04-16T05:43:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feed8ff9640 1 -- 172.21.15.32:0/2371307276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0004920 con 0x55c1045d2770 2024-04-16T05:43:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 -- 172.21.15.32:0/2371307276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 msgr2=0x55c1045d4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d4b60 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7feee002c820 tx=0x7feee0000bc0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 -- 172.21.15.32:0/2371307276 shutdown_connections 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d4b60 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 --2- 172.21.15.32:0/2371307276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 0x55c1045c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 -- 172.21.15.32:0/2371307276 >> 172.21.15.32:0/2371307276 conn(0x55c1045c39a0 msgr2=0x55c1045c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 -- 172.21.15.32:0/2371307276 shutdown_connections 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.761+0000 7feef39fc740 1 -- 172.21.15.32:0/2371307276 wait complete. 2024-04-16T05:43:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 1 -- start start 2024-04-16T05:43:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 1 Processor -- start 2024-04-16T05:43:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 0x55c1045cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c1045d54a0 con 0x55c1045d2770 2024-04-16T05:43:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c1044e34d0 con 0x55c1045c9270 2024-04-16T05:43:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1045d5620 con 0x55c1045c9b90 2024-04-16T05:43:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feef39fc740 10 client.0 _ll_register_callbacks cb 0x55c1044c82c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feee9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feee9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34336/0 (socket says 172.21.15.32:34336) 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feee9d9b640 1 -- 172.21.15.32:0/3708558392 learned_addr learned my addr 172.21.15.32:0/3708558392 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feee9d9b640 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 0x55c1045cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feeea59c640 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feee9d9b640 1 -- 172.21.15.32:0/3708558392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 msgr2=0x55c1045cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feee9d9b640 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.762+0000 7feee9d9b640 1 -- 172.21.15.32:0/3708558392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 msgr2=0x55c1045cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feee9d9b640 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 0x55c1045cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feee9d9b640 1 -- 172.21.15.32:0/3708558392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10423c3f0 con 0x55c1045d2770 2024-04-16T05:43:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feee9d9b640 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d0260 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7feee002c170 tx=0x7feee002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838e1a17be72745 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feed1ffb640 1 -- 172.21.15.32:0/3708558392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0004960 con 0x55c1045d2770 2024-04-16T05:43:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feed1ffb640 1 -- 172.21.15.32:0/3708558392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7feee0004020 con 0x55c1045d2770 2024-04-16T05:43:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 1 -- 172.21.15.32:0/3708558392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee002ccc0 con 0x55c1045d2770 2024-04-16T05:43:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feed1ffb640 1 -- 172.21.15.32:0/3708558392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feee0004220 con 0x55c1045d2770 2024-04-16T05:43:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 10 client.17523 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 1 -- 172.21.15.32:0/3708558392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1043067d0 con 0x55c1045d2770 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feed1ffb640 1 -- 172.21.15.32:0/3708558392 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee002ccc0 con 0x55c1045d2770 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feed1ffb640 1 -- 172.21.15.32:0/3708558392 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feee0003a60 con 0x55c1045d2770 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 10 client.17523 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 10 client.17523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 20 client.17523 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55c1045e25b0 0x55c1045e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 4 client.17523 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:07.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.763+0000 7feef39fc740 1 -- 172.21.15.32:0/3708558392 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c104537b00 con 0x55c1045e25b0 2024-04-16T05:43:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.764+0000 7feeea59c640 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55c1045e25b0 0x55c1045e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.764+0000 7feed1ffb640 10 client.17523 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:43:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.764+0000 7feeea59c640 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55c1045e25b0 0x55c1045e49a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d35c42434eb6a7 server_cookie=764b3cfab1f5d324 in_seq=0 out_seq=0 2024-04-16T05:43:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feed1ffb640 1 -- 172.21.15.32:0/3708558392 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c104537b00 con 0x55c1045e25b0 2024-04-16T05:43:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feed1ffb640 10 client.17523 handle_command_reply: tid=0 2024-04-16T05:43:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 client.17523 shutdown 2024-04-16T05:43:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 -- 172.21.15.32:0/3708558392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 msgr2=0x55c1045d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d0260 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7feee002c170 tx=0x7feee002c900 comp rx=0 tx=0).stop 2024-04-16T05:43:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 -- 172.21.15.32:0/3708558392 shutdown_connections 2024-04-16T05:43:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55c1045e25b0 0x55c1045e49a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045d2770 0x55c1045d0260 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feed1ffb640 0 client.17523 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:43:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 --2- 172.21.15.32:0/3708558392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045c9270 0x55c1045cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 -- 172.21.15.32:0/3708558392 >> 172.21.15.32:0/3708558392 conn(0x55c1045c39a0 msgr2=0x55c1045be730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 -- 172.21.15.32:0/3708558392 shutdown_connections 2024-04-16T05:43:07.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 1 -- 172.21.15.32:0/3708558392 wait complete. 2024-04-16T05:43:07.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.765+0000 7feef39fc740 20 client.17523 trim_cache size 0 max 16384 2024-04-16T05:43:07.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feef39fc740 1 Processor -- start 2024-04-16T05:43:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feef39fc740 1 -- start start 2024-04-16T05:43:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feef39fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9270 0x55c1045d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feef39fc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9b90 0x55c1045ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feef39fc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045d2770 0x55c1045e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feef39fc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1afbe0 con 0x55c1045c9b90 2024-04-16T05:43:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feef39fc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1ad4b0 con 0x55c1045d2770 2024-04-16T05:43:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feef39fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1ab320 con 0x55c1045c9270 2024-04-16T05:43:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9b90 0x55c1045ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9b90 0x55c1045ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34342/0 (socket says 172.21.15.32:34342) 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 -- 172.21.15.32:0/1334471602 learned_addr learned my addr 172.21.15.32:0/1334471602 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 -- 172.21.15.32:0/1334471602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9270 msgr2=0x55c1045d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feee9d9b640 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9270 0x55c1045d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9270 0x55c1045d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 -- 172.21.15.32:0/1334471602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045d2770 msgr2=0x55c1045e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045d2770 0x55c1045e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 -- 172.21.15.32:0/1334471602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1045b9b40 con 0x55c1045c9b90 2024-04-16T05:43:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feee9d9b640 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9270 0x55c1045d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.779+0000 7feeea59c640 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9b90 0x55c1045ce690 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x55c10452a1f0 tx=0x7feedc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa16258f7fa205a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feed3fff640 1 -- 172.21.15.32:0/1334471602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc021030 con 0x55c1045c9b90 2024-04-16T05:43:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feed3fff640 1 -- 172.21.15.32:0/1334471602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feedc01d900 con 0x55c1045c9b90 2024-04-16T05:43:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feed3fff640 1 -- 172.21.15.32:0/1334471602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc004a90 con 0x55c1045c9b90 2024-04-16T05:43:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 -- 172.21.15.32:0/1334471602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9b90 msgr2=0x55c1045ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9b90 0x55c1045ce690 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x55c10452a1f0 tx=0x7feedc011810 comp rx=0 tx=0).stop 2024-04-16T05:43:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 -- 172.21.15.32:0/1334471602 shutdown_connections 2024-04-16T05:43:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045d2770 0x55c1045e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9b90 0x55c1045ce690 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 --2- 172.21.15.32:0/1334471602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9270 0x55c1045d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 -- 172.21.15.32:0/1334471602 >> 172.21.15.32:0/1334471602 conn(0x55c1045c39a0 msgr2=0x55c1045cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 -- 172.21.15.32:0/1334471602 shutdown_connections 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.780+0000 7feef39fc740 1 -- 172.21.15.32:0/1334471602 wait complete. 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 1 -- start start 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 1 Processor -- start 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9270 0x55c1045e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045d2770 0x55c1045e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1ab970 con 0x55c1045c9270 2024-04-16T05:43:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1acde0 con 0x55c1045d2770 2024-04-16T05:43:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1ac040 con 0x55c1045c9b90 2024-04-16T05:43:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feef39fc740 10 client.0 _ll_register_callbacks cb 0x7feee0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9270 0x55c1045e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9270 0x55c1045e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34352/0 (socket says 172.21.15.32:34352) 2024-04-16T05:43:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 -- 172.21.15.32:0/2600296983 learned_addr learned my addr 172.21.15.32:0/2600296983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feeea59c640 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 -- 172.21.15.32:0/2600296983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 msgr2=0x55c1045e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 -- 172.21.15.32:0/2600296983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045d2770 msgr2=0x55c1045e54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045d2770 0x55c1045e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 -- 172.21.15.32:0/2600296983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1045e5af0 con 0x55c1045c9270 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feee9d9b640 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9270 0x55c1045e7be0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7feee00095a0 tx=0x7feee0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602d1ca0e2b212ca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.781+0000 7feed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:07.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feed1ffb640 1 -- 172.21.15.32:0/2600296983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee002c2b0 con 0x55c1045c9270 2024-04-16T05:43:07.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 1 -- 172.21.15.32:0/2600296983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feedc01d900 con 0x55c1045c9270 2024-04-16T05:43:07.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feed1ffb640 1 -- 172.21.15.32:0/2600296983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7feee00046c0 con 0x55c1045c9270 2024-04-16T05:43:07.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feed1ffb640 1 -- 172.21.15.32:0/2600296983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feee0071050 con 0x55c1045c9270 2024-04-16T05:43:07.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 10 client.17529 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:07.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 1 -- 172.21.15.32:0/2600296983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c104648550 con 0x55c1045c9270 2024-04-16T05:43:07.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feed1ffb640 1 -- 172.21.15.32:0/2600296983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feedc01d900 con 0x55c1045c9270 2024-04-16T05:43:07.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feed1ffb640 1 -- 172.21.15.32:0/2600296983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feee00035f0 con 0x55c1045c9270 2024-04-16T05:43:07.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 10 client.17529 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 10 client.17529 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 20 client.17529 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55c104648c40 0x55c10464b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 4 client.17529 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feef39fc740 1 -- 172.21.15.32:0/2600296983 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c104531970 con 0x55c104648c40 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.782+0000 7feeea59c640 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55c104648c40 0x55c10464b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feeea59c640 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55c104648c40 0x55c10464b060 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5ffea2e70d33b00 server_cookie=1559b1ee91fad1fe in_seq=0 out_seq=0 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feed1ffb640 10 client.17529 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:43:07.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feed1ffb640 1 -- 172.21.15.32:0/2600296983 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c104531970 con 0x55c104648c40 2024-04-16T05:43:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feed1ffb640 10 client.17529 handle_command_reply: tid=0 2024-04-16T05:43:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 client.17529 shutdown 2024-04-16T05:43:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 -- 172.21.15.32:0/2600296983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9270 msgr2=0x55c1045e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9270 0x55c1045e7be0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7feee00095a0 tx=0x7feee0005720 comp rx=0 tx=0).stop 2024-04-16T05:43:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 -- 172.21.15.32:0/2600296983 shutdown_connections 2024-04-16T05:43:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55c104648c40 0x55c10464b060 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1045d2770 0x55c1045e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1045c9b90 0x55c1045e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feed1ffb640 0 client.17529 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:43:07.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 --2- 172.21.15.32:0/2600296983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c1045c9270 0x55c1045e7be0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 -- 172.21.15.32:0/2600296983 >> 172.21.15.32:0/2600296983 conn(0x55c1045c39a0 msgr2=0x55c1045ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 -- 172.21.15.32:0/2600296983 shutdown_connections 2024-04-16T05:43:07.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 1 -- 172.21.15.32:0/2600296983 wait complete. 2024-04-16T05:43:07.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.783+0000 7feef39fc740 20 client.17529 trim_cache size 0 max 16384 2024-04-16T05:43:07.793 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:43:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 -- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feed40a3d20 msgr2=0x7feed40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feed40a3d20 0x7feed40a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feee409cb10 tx=0x7feee400efd0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 -- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec198730 msgr2=0x7feeec198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec198730 0x7feeec198bb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7feee400eb10 tx=0x7feee400c2d0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 -- 172.21.15.32:0/3817025763 shutdown_connections 2024-04-16T05:43:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feed40a3d20 0x7feed40a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec198730 0x7feeec198bb0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeec106200 0x7feeec19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 --2- 172.21.15.32:0/3817025763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeec101130 0x7feeec19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 -- 172.21.15.32:0/3817025763 >> 172.21.15.32:0/3817025763 conn(0x7feeec0780b0 msgr2=0x7feeec0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 -- 172.21.15.32:0/3817025763 shutdown_connections 2024-04-16T05:43:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.784+0000 7feef0e0d640 1 -- 172.21.15.32:0/3817025763 wait complete. 2024-04-16T05:43:07.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:43:07.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 -- 172.21.15.32:0/253506222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914101130 msgr2=0x7f7914101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 --2- 172.21.15.32:0/253506222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914101130 0x7f7914101510 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7910009900 tx=0x7f791002bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:07.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 -- 172.21.15.32:0/253506222 shutdown_connections 2024-04-16T05:43:07.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 --2- 172.21.15.32:0/253506222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7914106200 0x7f791410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 --2- 172.21.15.32:0/253506222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914101ae0 0x7f7914105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 --2- 172.21.15.32:0/253506222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914101130 0x7f7914101510 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 -- 172.21.15.32:0/253506222 >> 172.21.15.32:0/253506222 conn(0x7f79140780b0 msgr2=0x7f79140784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 -- 172.21.15.32:0/253506222 shutdown_connections 2024-04-16T05:43:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.908+0000 7f791bb38640 1 -- 172.21.15.32:0/253506222 wait complete. 2024-04-16T05:43:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791bb38640 1 Processor -- start 2024-04-16T05:43:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791bb38640 1 -- start start 2024-04-16T05:43:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791bb38640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914101130 0x7f7914102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791bb38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7914106200 0x7f7914103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791bb38640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914103640 0x7f79141a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791bb38640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79141112c0 con 0x7f7914103640 2024-04-16T05:43:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791bb38640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7914111140 con 0x7f7914101130 2024-04-16T05:43:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791bb38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7914111440 con 0x7f7914106200 2024-04-16T05:43:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791a0d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914103640 0x7f79141a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791a0d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914103640 0x7f79141a6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34358/0 (socket says 172.21.15.32:34358) 2024-04-16T05:43:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f79198d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914101130 0x7f7914102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791a0d5640 1 -- 172.21.15.32:0/2931600051 learned_addr learned my addr 172.21.15.32:0/2931600051 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f79190d3640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7914106200 0x7f7914103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791a0d5640 1 -- 172.21.15.32:0/2931600051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7914106200 msgr2=0x7f7914103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791a0d5640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7914106200 0x7f7914103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791a0d5640 1 -- 172.21.15.32:0/2931600051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914101130 msgr2=0x7f7914102bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791a0d5640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914101130 0x7f7914102bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.909+0000 7f791a0d5640 1 -- 172.21.15.32:0/2931600051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79141a7150 con 0x7f7914103640 2024-04-16T05:43:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.910+0000 7f791a0d5640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914103640 0x7f79141a6a50 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f790c00eb10 tx=0x7f790c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.910+0000 7f790affd640 1 -- 172.21.15.32:0/2931600051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c004b20 con 0x7f7914103640 2024-04-16T05:43:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.910+0000 7f790affd640 1 -- 172.21.15.32:0/2931600051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790c004cc0 con 0x7f7914103640 2024-04-16T05:43:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.910+0000 7f791bb38640 1 -- 172.21.15.32:0/2931600051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79141a7440 con 0x7f7914103640 2024-04-16T05:43:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.910+0000 7f790affd640 1 -- 172.21.15.32:0/2931600051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c010760 con 0x7f7914103640 2024-04-16T05:43:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.910+0000 7f791bb38640 1 -- 172.21.15.32:0/2931600051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79141a7990 con 0x7f7914103640 2024-04-16T05:43:07.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.912+0000 7f790affd640 1 -- 172.21.15.32:0/2931600051 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f790c005020 con 0x7f7914103640 2024-04-16T05:43:07.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f791e7f4740 1 Processor -- start 2024-04-16T05:43:07.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f791e7f4740 1 -- start start 2024-04-16T05:43:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d2a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d32e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9d35750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f791e7f4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5618f9c8f0e0 con 0x5618f9d2a780 2024-04-16T05:43:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f791e7f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5618f9c2c520 con 0x5618f9d29e60 2024-04-16T05:43:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f791e7f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5618f9c8b0a0 con 0x5618f9d33360 2024-04-16T05:43:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f790affd640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f79000a3d50 0x7f79000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f79198d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d32e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f79198d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d32e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34360/0 (socket says 172.21.15.32:34360) 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f79198d4640 1 -- 172.21.15.32:0/2767065597 learned_addr learned my addr 172.21.15.32:0/2767065597 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.913+0000 7f79190d3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d2a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f79190d3640 1 --2- 172.21.15.32:0/2767065597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9d35750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f791a0d5640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f79000a3d50 0x7f79000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f790affd640 1 -- 172.21.15.32:0/2931600051 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f790c0d1060 con 0x7f7914103640 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f79190d3640 1 -- 172.21.15.32:0/2767065597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 msgr2=0x5618f9d35750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f79190d3640 1 --2- 172.21.15.32:0/2767065597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9d35750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f79190d3640 1 -- 172.21.15.32:0/2767065597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 msgr2=0x5618f9d32e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f79190d3640 1 --2- 172.21.15.32:0/2767065597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d32e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f79190d3640 1 -- 172.21.15.32:0/2767065597 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f9c548d0 con 0x5618f9d29e60 2024-04-16T05:43:07.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f79190d3640 1 --2- 172.21.15.32:0/2767065597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d2a240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7904009730 tx=0x7f79040096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12b64b92b192f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f791a0d5640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f79000a3d50 0x7f79000a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f790c0ce9c0 tx=0x7f790c0bd040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f78fffff640 1 -- 172.21.15.32:0/2767065597 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904004050 con 0x5618f9d29e60 2024-04-16T05:43:07.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f78fffff640 1 -- 172.21.15.32:0/2767065597 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7904005500 con 0x5618f9d29e60 2024-04-16T05:43:07.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.914+0000 7f78fffff640 1 -- 172.21.15.32:0/2767065597 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904004d70 con 0x5618f9d29e60 2024-04-16T05:43:07.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 -- 172.21.15.32:0/2767065597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 msgr2=0x5618f9d2a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 --2- 172.21.15.32:0/2767065597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d2a240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7904009730 tx=0x7f79040096c0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 -- 172.21.15.32:0/2767065597 shutdown_connections 2024-04-16T05:43:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 --2- 172.21.15.32:0/2767065597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9d35750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 --2- 172.21.15.32:0/2767065597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d32e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 --2- 172.21.15.32:0/2767065597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d2a240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 -- 172.21.15.32:0/2767065597 >> 172.21.15.32:0/2767065597 conn(0x5618f9d24590 msgr2=0x5618f9d24970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 -- 172.21.15.32:0/2767065597 shutdown_connections 2024-04-16T05:43:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.915+0000 7f791e7f4740 1 -- 172.21.15.32:0/2767065597 wait complete. 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 1 -- start start 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 1 Processor -- start 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d29e60 0x5618f9d303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d30930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d33360 0x5618f9d30e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5618f9d36000 con 0x5618f9d2a780 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5618f9c484d0 con 0x5618f9d33360 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5618f9d36180 con 0x5618f9d29e60 2024-04-16T05:43:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f791e7f4740 10 client.0 _ll_register_callbacks cb 0x5618f9c2d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f79198d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d30930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f79198d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d30930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34370/0 (socket says 172.21.15.32:34370) 2024-04-16T05:43:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f79198d4640 1 -- 172.21.15.32:0/1335513928 learned_addr learned my addr 172.21.15.32:0/1335513928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f79190d3640 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d33360 0x5618f9d30e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.916+0000 7f79190d3640 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d29e60 0x5618f9d303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f79198d4640 1 -- 172.21.15.32:0/1335513928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d29e60 msgr2=0x5618f9d303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f79198d4640 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d29e60 0x5618f9d303f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f79198d4640 1 -- 172.21.15.32:0/1335513928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d33360 msgr2=0x5618f9d30e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f79198d4640 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d33360 0x5618f9d30e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f79198d4640 1 -- 172.21.15.32:0/1335513928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f99a13f0 con 0x5618f9d2a780 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f79198d4640 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d30930 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7910009230 tx=0x7f7910009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be8c635ceb862e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f78fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f78fd7fa640 1 -- 172.21.15.32:0/1335513928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910004300 con 0x5618f9d2a780 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f78fd7fa640 1 -- 172.21.15.32:0/1335513928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f79100046c0 con 0x5618f9d2a780 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f791e7f4740 1 -- 172.21.15.32:0/1335513928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7904005500 con 0x5618f9d2a780 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f78fd7fa640 1 -- 172.21.15.32:0/1335513928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79100044a0 con 0x5618f9d2a780 2024-04-16T05:43:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f791e7f4740 10 client.17541 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.917+0000 7f791e7f4740 1 -- 172.21.15.32:0/1335513928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618f9a6b7d0 con 0x5618f9d2a780 2024-04-16T05:43:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f78fd7fa640 1 -- 172.21.15.32:0/1335513928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7904005500 con 0x5618f9d2a780 2024-04-16T05:43:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f78fd7fa640 1 -- 172.21.15.32:0/1335513928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79100035f0 con 0x5618f9d2a780 2024-04-16T05:43:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f791e7f4740 10 client.17541 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f791e7f4740 10 client.17541 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:43:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f791e7f4740 20 client.17541 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f791e7f4740 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5618f9d433f0 0x5618f9d45810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f791e7f4740 4 client.17541 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f791e7f4740 1 -- 172.21.15.32:0/1335513928 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618f9c9cb00 con 0x5618f9d433f0 2024-04-16T05:43:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.918+0000 7f79190d3640 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5618f9d433f0 0x5618f9d45810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.919+0000 7f78fd7fa640 10 client.17541 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:43:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.919+0000 7f79190d3640 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5618f9d433f0 0x5618f9d45810 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=784acd040c8a12ca server_cookie=791e0c28e2ea4355 in_seq=0 out_seq=0 2024-04-16T05:43:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.919+0000 7f78fd7fa640 1 -- 172.21.15.32:0/1335513928 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618f9c9cb00 con 0x5618f9d433f0 2024-04-16T05:43:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.919+0000 7f78fd7fa640 10 client.17541 handle_command_reply: tid=0 2024-04-16T05:43:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.919+0000 7f791e7f4740 1 client.17541 shutdown 2024-04-16T05:43:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.919+0000 7f791e7f4740 1 -- 172.21.15.32:0/1335513928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 msgr2=0x5618f9d30930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.919+0000 7f791e7f4740 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d30930 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7910009230 tx=0x7f7910009560 comp rx=0 tx=0).stop 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 1 -- 172.21.15.32:0/1335513928 shutdown_connections 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5618f9d433f0 0x5618f9d45810 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d33360 0x5618f9d30e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d30930 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 1 --2- 172.21.15.32:0/1335513928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d29e60 0x5618f9d303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f78fd7fa640 0 client.17541 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 1 -- 172.21.15.32:0/1335513928 >> 172.21.15.32:0/1335513928 conn(0x5618f9d24590 msgr2=0x5618f9d262e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 1 -- 172.21.15.32:0/1335513928 shutdown_connections 2024-04-16T05:43:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 1 -- 172.21.15.32:0/1335513928 wait complete. 2024-04-16T05:43:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.920+0000 7f791e7f4740 20 client.17541 trim_cache size 0 max 16384 2024-04-16T05:43:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f791e7f4740 1 Processor -- start 2024-04-16T05:43:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f791e7f4740 1 -- start start 2024-04-16T05:43:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d3f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d1f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9d30c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f791e7f4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b4090 con 0x5618f9d2a780 2024-04-16T05:43:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f791e7f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b1960 con 0x5618f9d29e60 2024-04-16T05:43:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f791e7f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79141af7d0 con 0x5618f9d33360 2024-04-16T05:43:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79198d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d1f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79190d3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d3f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79198d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d1f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34386/0 (socket says 172.21.15.32:34386) 2024-04-16T05:43:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79190d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9d30c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79198d4640 1 -- 172.21.15.32:0/754161391 learned_addr learned my addr 172.21.15.32:0/754161391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79198d4640 1 -- 172.21.15.32:0/754161391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 msgr2=0x5618f9d30c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79198d4640 1 --2- 172.21.15.32:0/754161391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9d30c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79198d4640 1 -- 172.21.15.32:0/754161391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 msgr2=0x5618f9d3f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79198d4640 1 --2- 172.21.15.32:0/754161391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d3f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.934+0000 7f79198d4640 1 -- 172.21.15.32:0/754161391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f9d2f3a0 con 0x5618f9d2a780 2024-04-16T05:43:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.935+0000 7f79198d4640 1 --2- 172.21.15.32:0/754161391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d1f5c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7910003a10 tx=0x7f791002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9461f985072a4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.935+0000 7f78ff7fe640 1 -- 172.21.15.32:0/754161391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100035f0 con 0x5618f9d2a780 2024-04-16T05:43:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.935+0000 7f78ff7fe640 1 -- 172.21.15.32:0/754161391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910003790 con 0x5618f9d2a780 2024-04-16T05:43:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.935+0000 7f78ff7fe640 1 -- 172.21.15.32:0/754161391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100046c0 con 0x5618f9d2a780 2024-04-16T05:43:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.935+0000 7f791e7f4740 1 -- 172.21.15.32:0/754161391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 msgr2=0x5618f9d1f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.935+0000 7f791e7f4740 1 --2- 172.21.15.32:0/754161391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d1f5c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7910003a10 tx=0x7f791002bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 -- 172.21.15.32:0/754161391 shutdown_connections 2024-04-16T05:43:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 --2- 172.21.15.32:0/754161391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9d30c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 --2- 172.21.15.32:0/754161391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d1f5c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 --2- 172.21.15.32:0/754161391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d3f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 -- 172.21.15.32:0/754161391 >> 172.21.15.32:0/754161391 conn(0x5618f9d24590 msgr2=0x5618f9d26250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 -- 172.21.15.32:0/754161391 shutdown_connections 2024-04-16T05:43:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 -- 172.21.15.32:0/754161391 wait complete. 2024-04-16T05:43:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 -- start start 2024-04-16T05:43:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 Processor -- start 2024-04-16T05:43:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9da8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79141afe20 con 0x5618f9d2a780 2024-04-16T05:43:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b1290 con 0x5618f9d29e60 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b04f0 con 0x5618f9d33360 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.936+0000 7f791e7f4740 10 client.0 _ll_register_callbacks cb 0x7f7910009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79190d3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34392/0 (socket says 172.21.15.32:34392) 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 -- 172.21.15.32:0/620946498 learned_addr learned my addr 172.21.15.32:0/620946498 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79190d3640 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9da8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 -- 172.21.15.32:0/620946498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 msgr2=0x5618f9da8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9da8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 -- 172.21.15.32:0/620946498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 msgr2=0x5618f9d372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 -- 172.21.15.32:0/620946498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f9d2a5b0 con 0x5618f9d2a780 2024-04-16T05:43:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79190d3640 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:43:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f79198d4640 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d377f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7910003e50 tx=0x7f7910003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86bf1606bbbe7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f78fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f78fd7fa640 1 -- 172.21.15.32:0/620946498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910005230 con 0x5618f9d2a780 2024-04-16T05:43:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f791e7f4740 1 -- 172.21.15.32:0/620946498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7910003790 con 0x5618f9d2a780 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.937+0000 7f78fd7fa640 1 -- 172.21.15.32:0/620946498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f79100053d0 con 0x5618f9d2a780 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f78fd7fa640 1 -- 172.21.15.32:0/620946498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7910009520 con 0x5618f9d2a780 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f791e7f4740 10 client.17547 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f791e7f4740 1 -- 172.21.15.32:0/620946498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618f9da9dc0 con 0x5618f9d2a780 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f78fd7fa640 1 -- 172.21.15.32:0/620946498 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910003790 con 0x5618f9d2a780 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f78fd7fa640 1 -- 172.21.15.32:0/620946498 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7910056440 con 0x5618f9d2a780 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f791e7f4740 10 client.17547 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f791e7f4740 10 client.17547 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:43:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f791e7f4740 20 client.17547 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f791e7f4740 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5618f9daa340 0x5618f9dac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f791e7f4740 4 client.17547 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f791e7f4740 1 -- 172.21.15.32:0/620946498 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618f9c96970 con 0x5618f9daa340 2024-04-16T05:43:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.938+0000 7f79190d3640 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5618f9daa340 0x5618f9dac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f79190d3640 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5618f9daa340 0x5618f9dac760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bfa94aef08537ff server_cookie=3942608284936b41 in_seq=0 out_seq=0 2024-04-16T05:43:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f78fd7fa640 10 client.17547 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:43:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f78fd7fa640 1 -- 172.21.15.32:0/620946498 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618f9c96970 con 0x5618f9daa340 2024-04-16T05:43:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f78fd7fa640 10 client.17547 handle_command_reply: tid=0 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 client.17547 shutdown 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 -- 172.21.15.32:0/620946498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 msgr2=0x5618f9d377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d377f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7910003e50 tx=0x7f7910003ae0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 -- 172.21.15.32:0/620946498 shutdown_connections 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5618f9daa340 0x5618f9dac760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f9d33360 0x5618f9da8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5618f9d2a780 0x5618f9d377f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f78fd7fa640 0 client.17547 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:43:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 --2- 172.21.15.32:0/620946498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618f9d29e60 0x5618f9d372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 -- 172.21.15.32:0/620946498 >> 172.21.15.32:0/620946498 conn(0x5618f9d24590 msgr2=0x5618f9d22ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.939+0000 7f791e7f4740 1 -- 172.21.15.32:0/620946498 shutdown_connections 2024-04-16T05:43:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791e7f4740 1 -- 172.21.15.32:0/620946498 wait complete. 2024-04-16T05:43:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791e7f4740 20 client.17547 trim_cache size 0 max 16384 2024-04-16T05:43:07.948 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:43:07.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 -- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f79000a3d50 msgr2=0x7f79000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f79000a3d50 0x7f79000a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f790c0ce9c0 tx=0x7f790c0bd040 comp rx=0 tx=0).stop 2024-04-16T05:43:07.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 -- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914103640 msgr2=0x7f79141a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914103640 0x7f79141a6a50 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f790c00eb10 tx=0x7f790c00c2d0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 -- 172.21.15.32:0/2931600051 shutdown_connections 2024-04-16T05:43:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f79000a3d50 0x7f79000a6210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914103640 0x7f79141a6a50 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7914106200 0x7f7914103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 --2- 172.21.15.32:0/2931600051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914101130 0x7f7914102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 -- 172.21.15.32:0/2931600051 >> 172.21.15.32:0/2931600051 conn(0x7f79140780b0 msgr2=0x7f7914069560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:07.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 -- 172.21.15.32:0/2931600051 shutdown_connections 2024-04-16T05:43:07.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:07.940+0000 7f791bb38640 1 -- 172.21.15.32:0/2931600051 wait complete. 2024-04-16T05:43:07.951 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:07 smithi032 ceph-mon[32351]: pgmap v251: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:07.951 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:07 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2892308282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:07.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:43:08.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 -- 172.21.15.32:0/3362491254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00101ae0 msgr2=0x7feb00105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 --2- 172.21.15.32:0/3362491254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00101ae0 0x7feb00105b20 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7feae8009900 tx=0x7feae802bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:08.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 -- 172.21.15.32:0/3362491254 shutdown_connections 2024-04-16T05:43:08.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 --2- 172.21.15.32:0/3362491254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb00106200 0x7feb0010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 --2- 172.21.15.32:0/3362491254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00101ae0 0x7feb00105b20 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 --2- 172.21.15.32:0/3362491254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb00101130 0x7feb00101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 -- 172.21.15.32:0/3362491254 >> 172.21.15.32:0/3362491254 conn(0x7feb000780b0 msgr2=0x7feb000784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:08.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 -- 172.21.15.32:0/3362491254 shutdown_connections 2024-04-16T05:43:08.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.063+0000 7feb067a8640 1 -- 172.21.15.32:0/3362491254 wait complete. 2024-04-16T05:43:08.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb067a8640 1 Processor -- start 2024-04-16T05:43:08.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb067a8640 1 -- start start 2024-04-16T05:43:08.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb067a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb00101130 0x7feb0019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb067a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00106200 0x7feb0019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb067a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb00198680 0x7feb00198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb067a8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feb00111290 con 0x7feb00106200 2024-04-16T05:43:08.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb067a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feb00111110 con 0x7feb00101130 2024-04-16T05:43:08.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb067a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb00111410 con 0x7feb00198680 2024-04-16T05:43:08.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00106200 0x7feb0019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00106200 0x7feb0019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34418/0 (socket says 172.21.15.32:34418) 2024-04-16T05:43:08.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feaf7fff640 1 -- 172.21.15.32:0/2629920236 learned_addr learned my addr 172.21.15.32:0/2629920236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:08.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feafffff640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb00101130 0x7feb0019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feb04d45640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb00198680 0x7feb00198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feaf7fff640 1 -- 172.21.15.32:0/2629920236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb00198680 msgr2=0x7feb00198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feaf7fff640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb00198680 0x7feb00198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feaf7fff640 1 -- 172.21.15.32:0/2629920236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb00101130 msgr2=0x7feb0019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feaf7fff640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb00101130 0x7feb0019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.064+0000 7feaf7fff640 1 -- 172.21.15.32:0/2629920236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb00199280 con 0x7feb00106200 2024-04-16T05:43:08.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.065+0000 7feaf7fff640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00106200 0x7feb0019eb00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7feae8036040 tx=0x7feae802cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:08.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.065+0000 7feafdffb640 1 -- 172.21.15.32:0/2629920236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8041070 con 0x7feb00106200 2024-04-16T05:43:08.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.065+0000 7feafdffb640 1 -- 172.21.15.32:0/2629920236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae8004450 con 0x7feb00106200 2024-04-16T05:43:08.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.065+0000 7feb067a8640 1 -- 172.21.15.32:0/2629920236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb00199510 con 0x7feb00106200 2024-04-16T05:43:08.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.065+0000 7feafdffb640 1 -- 172.21.15.32:0/2629920236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8003dc0 con 0x7feb00106200 2024-04-16T05:43:08.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.065+0000 7feb067a8640 1 -- 172.21.15.32:0/2629920236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb001a56a0 con 0x7feb00106200 2024-04-16T05:43:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.067+0000 7feafdffb640 1 -- 172.21.15.32:0/2629920236 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feae8044070 con 0x7feb00106200 2024-04-16T05:43:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.067+0000 7feb0938f740 1 Processor -- start 2024-04-16T05:43:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb0938f740 1 -- start start 2024-04-16T05:43:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb0938f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca377240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb0938f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb0938f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 0x55b0ca382750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb0938f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b0ca2dc0e0 con 0x55b0ca377780 2024-04-16T05:43:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb0938f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b0ca279520 con 0x55b0ca376e60 2024-04-16T05:43:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb0938f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b0ca2d80a0 con 0x55b0ca380360 2024-04-16T05:43:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafdffb640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feaec0a3da0 0x7feaec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 0x55b0ca382750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb04d45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca377240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34430/0 (socket says 172.21.15.32:34430) 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafffff640 1 -- 172.21.15.32:0/1081115372 learned_addr learned my addr 172.21.15.32:0/1081115372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb04d45640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feaec0a3da0 0x7feaec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafffff640 1 -- 172.21.15.32:0/1081115372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 msgr2=0x55b0ca382750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafdffb640 1 -- 172.21.15.32:0/2629920236 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7feae80f2600 con 0x7feb00106200 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafffff640 1 --2- 172.21.15.32:0/1081115372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 0x55b0ca382750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafffff640 1 -- 172.21.15.32:0/1081115372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 msgr2=0x55b0ca377240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafffff640 1 --2- 172.21.15.32:0/1081115372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca377240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feafffff640 1 -- 172.21.15.32:0/1081115372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ca2a18d0 con 0x55b0ca377780 2024-04-16T05:43:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.068+0000 7feb04d45640 1 --2- 172.21.15.32:0/1081115372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca377240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:43:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.069+0000 7feafffff640 1 --2- 172.21.15.32:0/1081115372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37fe20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009d20 tx=0x7feaf0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a6b8896aa00801 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.069+0000 7feb04d45640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feaec0a3da0 0x7feaec0a6260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feaf80094d0 tx=0x7feaf8004ab0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.069+0000 7feaf67fc640 1 -- 172.21.15.32:0/1081115372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf001e070 con 0x55b0ca377780 2024-04-16T05:43:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.069+0000 7feaf67fc640 1 -- 172.21.15.32:0/1081115372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf0019b10 con 0x55b0ca377780 2024-04-16T05:43:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.069+0000 7feaf67fc640 1 -- 172.21.15.32:0/1081115372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf000eb30 con 0x55b0ca377780 2024-04-16T05:43:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.069+0000 7feb0938f740 1 -- 172.21.15.32:0/1081115372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 msgr2=0x55b0ca37fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.069+0000 7feb0938f740 1 --2- 172.21.15.32:0/1081115372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37fe20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009d20 tx=0x7feaf0011810 comp rx=0 tx=0).stop 2024-04-16T05:43:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 -- 172.21.15.32:0/1081115372 shutdown_connections 2024-04-16T05:43:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 --2- 172.21.15.32:0/1081115372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 0x55b0ca382750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 --2- 172.21.15.32:0/1081115372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37fe20 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 --2- 172.21.15.32:0/1081115372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca377240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 -- 172.21.15.32:0/1081115372 >> 172.21.15.32:0/1081115372 conn(0x55b0ca371590 msgr2=0x55b0ca371970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 -- 172.21.15.32:0/1081115372 shutdown_connections 2024-04-16T05:43:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 -- 172.21.15.32:0/1081115372 wait complete. 2024-04-16T05:43:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 -- start start 2024-04-16T05:43:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 Processor -- start 2024-04-16T05:43:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca376e60 0x55b0ca37d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca377780 0x55b0ca37d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.070+0000 7feb0938f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca37ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feb0938f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b0ca383000 con 0x55b0ca376e60 2024-04-16T05:43:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feb0938f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b0ca2954d0 con 0x55b0ca380360 2024-04-16T05:43:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feb0938f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b0ca383180 con 0x55b0ca377780 2024-04-16T05:43:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feb0938f740 10 client.0 _ll_register_callbacks cb 0x55b0ca27a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca376e60 0x55b0ca37d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca376e60 0x55b0ca37d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34438/0 (socket says 172.21.15.32:34438) 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 -- 172.21.15.32:0/932029352 learned_addr learned my addr 172.21.15.32:0/932029352 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feaf7fff640 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca377780 0x55b0ca37d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feb04d45640 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca37ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 -- 172.21.15.32:0/932029352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca377780 msgr2=0x55b0ca37d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca377780 0x55b0ca37d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 -- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 msgr2=0x55b0ca37ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca37ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 -- 172.21.15.32:0/932029352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c9fee3f0 con 0x55b0ca376e60 2024-04-16T05:43:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feb04d45640 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca37ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feafffff640 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca376e60 0x55b0ca37d370 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7feaf0011d10 tx=0x7feaf0019430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27f1a23232141fe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feadf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feadf7fe640 1 -- 172.21.15.32:0/932029352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf001e040 con 0x55b0ca376e60 2024-04-16T05:43:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feb0938f740 1 -- 172.21.15.32:0/932029352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf0019b10 con 0x55b0ca376e60 2024-04-16T05:43:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.071+0000 7feadf7fe640 1 -- 172.21.15.32:0/932029352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7feaf0019820 con 0x55b0ca376e60 2024-04-16T05:43:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feadf7fe640 1 -- 172.21.15.32:0/932029352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feaf0031050 con 0x55b0ca376e60 2024-04-16T05:43:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feb0938f740 10 client.17559 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feb0938f740 1 -- 172.21.15.32:0/932029352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0ca0b87d0 con 0x55b0ca376e60 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feadf7fe640 1 -- 172.21.15.32:0/932029352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf0019b10 con 0x55b0ca376e60 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feadf7fe640 1 -- 172.21.15.32:0/932029352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feaf000e890 con 0x55b0ca376e60 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feb0938f740 10 client.17559 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feb0938f740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feb0938f740 20 client.17559 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feb0938f740 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b0ca3902b0 0x55b0ca3926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feb0938f740 4 client.17559 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feb0938f740 1 -- 172.21.15.32:0/932029352 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0ca2e9b00 con 0x55b0ca3902b0 2024-04-16T05:43:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.072+0000 7feaf7fff640 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b0ca3902b0 0x55b0ca3926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.073+0000 7feadf7fe640 10 client.17559 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.073+0000 7feaf7fff640 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b0ca3902b0 0x55b0ca3926a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b5d598fedfcec1 server_cookie=5998b79d8c1c0e87 in_seq=0 out_seq=0 2024-04-16T05:43:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.073+0000 7feadf7fe640 1 -- 172.21.15.32:0/932029352 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0ca2e9b00 con 0x55b0ca3902b0 2024-04-16T05:43:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.073+0000 7feadf7fe640 10 client.17559 handle_command_reply: tid=0 2024-04-16T05:43:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.073+0000 7feb0938f740 1 client.17559 shutdown 2024-04-16T05:43:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 -- 172.21.15.32:0/932029352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca376e60 msgr2=0x55b0ca37d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca376e60 0x55b0ca37d370 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7feaf0011d10 tx=0x7feaf0019430 comp rx=0 tx=0).stop 2024-04-16T05:43:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 -- 172.21.15.32:0/932029352 shutdown_connections 2024-04-16T05:43:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca37ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b0ca3902b0 0x55b0ca3926a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca377780 0x55b0ca37d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 --2- 172.21.15.32:0/932029352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca376e60 0x55b0ca37d370 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feadf7fe640 0 client.17559 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 -- 172.21.15.32:0/932029352 >> 172.21.15.32:0/932029352 conn(0x55b0ca371590 msgr2=0x55b0ca3732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 -- 172.21.15.32:0/932029352 shutdown_connections 2024-04-16T05:43:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 1 -- 172.21.15.32:0/932029352 wait complete. 2024-04-16T05:43:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.074+0000 7feb0938f740 20 client.17559 trim_cache size 0 max 16384 2024-04-16T05:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb0938f740 1 Processor -- start 2024-04-16T05:43:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb0938f740 1 -- start start 2024-04-16T05:43:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb0938f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca38c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb0938f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb0938f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 0x55b0ca397e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb0938f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feb001b1df0 con 0x55b0ca377780 2024-04-16T05:43:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb0938f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feb001af6c0 con 0x55b0ca376e60 2024-04-16T05:43:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb0938f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb001ad530 con 0x55b0ca380360 2024-04-16T05:43:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34450/0 (socket says 172.21.15.32:34450) 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feaf7fff640 1 -- 172.21.15.32:0/4274177185 learned_addr learned my addr 172.21.15.32:0/4274177185 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feafffff640 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca38c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.087+0000 7feb04d45640 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 0x55b0ca397e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf7fff640 1 -- 172.21.15.32:0/4274177185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 msgr2=0x55b0ca397e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf7fff640 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 0x55b0ca397e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf7fff640 1 -- 172.21.15.32:0/4274177185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 msgr2=0x55b0ca38c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf7fff640 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca38c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf7fff640 1 -- 172.21.15.32:0/4274177185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ca378f50 con 0x55b0ca377780 2024-04-16T05:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf7fff640 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37f1a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7feae80ba670 tx=0x7feae802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805d339fcc5dd9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf5ffb640 1 -- 172.21.15.32:0/4274177185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae804b040 con 0x55b0ca377780 2024-04-16T05:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf5ffb640 1 -- 172.21.15.32:0/4274177185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae80be960 con 0x55b0ca377780 2024-04-16T05:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.088+0000 7feaf5ffb640 1 -- 172.21.15.32:0/4274177185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae80bc5f0 con 0x55b0ca377780 2024-04-16T05:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 -- 172.21.15.32:0/4274177185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 msgr2=0x55b0ca37f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37f1a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7feae80ba670 tx=0x7feae802bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 -- 172.21.15.32:0/4274177185 shutdown_connections 2024-04-16T05:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca380360 0x55b0ca397e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca37f1a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 --2- 172.21.15.32:0/4274177185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca376e60 0x55b0ca38c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 -- 172.21.15.32:0/4274177185 >> 172.21.15.32:0/4274177185 conn(0x55b0ca371590 msgr2=0x55b0ca37ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 -- 172.21.15.32:0/4274177185 shutdown_connections 2024-04-16T05:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 -- 172.21.15.32:0/4274177185 wait complete. 2024-04-16T05:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 -- start start 2024-04-16T05:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.089+0000 7feb0938f740 1 Processor -- start 2024-04-16T05:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feb0938f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca376e60 0x55b0ca3889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feb0938f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca383d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feb0938f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca384250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feb0938f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feb001adb80 con 0x55b0ca377780 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feb0938f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feb001aeff0 con 0x55b0ca380360 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feb0938f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb001ae250 con 0x55b0ca376e60 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feb0938f740 10 client.0 _ll_register_callbacks cb 0x7feaf001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca383d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca383d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34464/0 (socket says 172.21.15.32:34464) 2024-04-16T05:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feafffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca376e60 0x55b0ca3889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 -- 172.21.15.32:0/715544696 learned_addr learned my addr 172.21.15.32:0/715544696 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feb04d45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca384250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 -- 172.21.15.32:0/715544696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca376e60 msgr2=0x55b0ca3889a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca376e60 0x55b0ca3889a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 -- 172.21.15.32:0/715544696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 msgr2=0x55b0ca384250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca384250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 -- 172.21.15.32:0/715544696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ca392a10 con 0x55b0ca377780 2024-04-16T05:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feaf7fff640 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca383d10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7feae80bdf60 tx=0x7feae8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb668690428380b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feadf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.090+0000 7feadf7fe640 1 -- 172.21.15.32:0/715544696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae804b070 con 0x55b0ca377780 2024-04-16T05:43:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feadf7fe640 1 -- 172.21.15.32:0/715544696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7feae80ba8d0 con 0x55b0ca377780 2024-04-16T05:43:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 1 -- 172.21.15.32:0/715544696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae80be960 con 0x55b0ca377780 2024-04-16T05:43:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feadf7fe640 1 -- 172.21.15.32:0/715544696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feae80c6ed0 con 0x55b0ca377780 2024-04-16T05:43:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 10 client.17565 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 1 -- 172.21.15.32:0/715544696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0ca3f6580 con 0x55b0ca377780 2024-04-16T05:43:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feadf7fe640 1 -- 172.21.15.32:0/715544696 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae80be960 con 0x55b0ca377780 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feadf7fe640 1 -- 172.21.15.32:0/715544696 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feae80c6a40 con 0x55b0ca377780 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 10 client.17565 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 20 client.17565 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b0ca3f6bf0 0x55b0ca3f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 4 client.17565 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feb0938f740 1 -- 172.21.15.32:0/715544696 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0ca2e3970 con 0x55b0ca3f6bf0 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.091+0000 7feafffff640 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b0ca3f6bf0 0x55b0ca3f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feafffff640 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b0ca3f6bf0 0x55b0ca3f9010 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffa0afb11e7fcc8 server_cookie=a609f41e16cea22d in_seq=0 out_seq=0 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feadf7fe640 10 client.17565 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feadf7fe640 1 -- 172.21.15.32:0/715544696 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0ca2e3970 con 0x55b0ca3f6bf0 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feadf7fe640 10 client.17565 handle_command_reply: tid=0 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 client.17565 shutdown 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 -- 172.21.15.32:0/715544696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 msgr2=0x55b0ca383d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca383d10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7feae80bdf60 tx=0x7feae8005fd0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 -- 172.21.15.32:0/715544696 shutdown_connections 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0ca380360 0x55b0ca384250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0ca377780 0x55b0ca383d10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55b0ca3f6bf0 0x55b0ca3f9010 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 --2- 172.21.15.32:0/715544696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0ca376e60 0x55b0ca3889a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feadf7fe640 0 client.17565 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 -- 172.21.15.32:0/715544696 >> 172.21.15.32:0/715544696 conn(0x55b0ca371590 msgr2=0x55b0ca370730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 -- 172.21.15.32:0/715544696 shutdown_connections 2024-04-16T05:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 1 -- 172.21.15.32:0/715544696 wait complete. 2024-04-16T05:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.092+0000 7feb0938f740 20 client.17565 trim_cache size 0 max 16384 2024-04-16T05:43:08.103 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:43:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 -- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feaec0a3da0 msgr2=0x7feaec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feaec0a3da0 0x7feaec0a6260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feaf80094d0 tx=0x7feaf8004ab0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 -- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00106200 msgr2=0x7feb0019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00106200 0x7feb0019eb00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7feae8036040 tx=0x7feae802cb20 comp rx=0 tx=0).stop 2024-04-16T05:43:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 -- 172.21.15.32:0/2629920236 shutdown_connections 2024-04-16T05:43:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb00198680 0x7feb00198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7feaec0a3da0 0x7feaec0a6260 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb00106200 0x7feb0019eb00 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 --2- 172.21.15.32:0/2629920236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb00101130 0x7feb0019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 -- 172.21.15.32:0/2629920236 >> 172.21.15.32:0/2629920236 conn(0x7feb000780b0 msgr2=0x7feb000ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 -- 172.21.15.32:0/2629920236 shutdown_connections 2024-04-16T05:43:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:08.093+0000 7feb067a8640 1 -- 172.21.15.32:0/2629920236 wait complete. 2024-04-16T05:43:08.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:07 smithi134 ceph-mon[36153]: pgmap v251: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:08.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:07 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2892308282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:08.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:07 smithi082 ceph-mon[36195]: pgmap v251: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:08.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:07 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2892308282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:09.107 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:43:09.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:43:09.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:43:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:08 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:43:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:08 smithi134 ceph-mon[36153]: pgmap v252: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:08 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:43:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:08 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:43:09.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:08 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:43:09.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:08 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:43:09.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:08 smithi032 ceph-mon[32351]: pgmap v252: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:09.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:08 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:43:09.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:08 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:43:09.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:08 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:43:09.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:08 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:43:09.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:08 smithi082 ceph-mon[36195]: pgmap v252: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:09.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:08 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:43:09.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:08 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:43:09.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:08 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:43:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 -- 172.21.15.32:0/916418165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528106220 msgr2=0x7fa52810ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 --2- 172.21.15.32:0/916418165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528106220 0x7fa52810ddf0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa520009ca0 tx=0x7fa52002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 -- 172.21.15.32:0/916418165 shutdown_connections 2024-04-16T05:43:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 --2- 172.21.15.32:0/916418165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528106220 0x7fa52810ddf0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 --2- 172.21.15.32:0/916418165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa528101b00 0x7fa528105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 --2- 172.21.15.32:0/916418165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa528101150 0x7fa528101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 -- 172.21.15.32:0/916418165 >> 172.21.15.32:0/916418165 conn(0x7fa528077f60 msgr2=0x7fa528078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 -- 172.21.15.32:0/916418165 shutdown_connections 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.223+0000 7fa52d359640 1 -- 172.21.15.32:0/916418165 wait complete. 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa52d359640 1 Processor -- start 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa52d359640 1 -- start start 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa52d359640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa528101150 0x7fa5281996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa52d359640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa528106220 0x7fa528199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa52d359640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa52819a170 0x7fa5281a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa52d359640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281112c0 con 0x7fa52819a170 2024-04-16T05:43:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa52d359640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa528111140 con 0x7fa528106220 2024-04-16T05:43:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa52d359640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa528111440 con 0x7fa528101150 2024-04-16T05:43:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa5277fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa52819a170 0x7fa5281a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa5277fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa52819a170 0x7fa5281a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40586/0 (socket says 172.21.15.32:40586) 2024-04-16T05:43:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa5277fe640 1 -- 172.21.15.32:0/2688073172 learned_addr learned my addr 172.21.15.32:0/2688073172 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa528101150 0x7fa5281996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa528106220 0x7fa528199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa5277fe640 1 -- 172.21.15.32:0/2688073172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa528101150 msgr2=0x7fa5281996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.224+0000 7fa5277fe640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa528101150 0x7fa5281996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa5277fe640 1 -- 172.21.15.32:0/2688073172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa528106220 msgr2=0x7fa528199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa5277fe640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa528106220 0x7fa528199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa5277fe640 1 -- 172.21.15.32:0/2688073172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5281a4ef0 con 0x7fa52819a170 2024-04-16T05:43:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa528106220 0x7fa528199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa5277fe640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa52819a170 0x7fa5281a4720 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fa52002c700 tx=0x7fa520032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa524ff9640 1 -- 172.21.15.32:0/2688073172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520039040 con 0x7fa52819a170 2024-04-16T05:43:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa52d359640 1 -- 172.21.15.32:0/2688073172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5281a5180 con 0x7fa52819a170 2024-04-16T05:43:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa524ff9640 1 -- 172.21.15.32:0/2688073172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa520044070 con 0x7fa52819a170 2024-04-16T05:43:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa524ff9640 1 -- 172.21.15.32:0/2688073172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200042a0 con 0x7fa52819a170 2024-04-16T05:43:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.225+0000 7fa52d359640 1 -- 172.21.15.32:0/2688073172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5281a56c0 con 0x7fa52819a170 2024-04-16T05:43:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa530022740 1 Processor -- start 2024-04-16T05:43:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa524ff9640 1 -- 172.21.15.32:0/2688073172 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa520004440 con 0x7fa52819a170 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa530022740 1 -- start start 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa530022740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535094e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa530022740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x56553509da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa530022740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x5655350a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa530022740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x565534ff9e90 con 0x5655350953d0 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa530022740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565534f974c0 con 0x565535094ab0 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.228+0000 7fa530022740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565534ff5e50 con 0x56553509dfb0 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x56553509da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x56553509da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40598/0 (socket says 172.21.15.32:40598) 2024-04-16T05:43:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 -- 172.21.15.32:0/4083671374 learned_addr learned my addr 172.21.15.32:0/4083671374 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa51ffff640 1 --2- 172.21.15.32:0/4083671374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535094e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa524ff9640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa5140a3d20 0x7fa5140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 -- 172.21.15.32:0/4083671374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 msgr2=0x5655350a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 --2- 172.21.15.32:0/4083671374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x5655350a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 -- 172.21.15.32:0/4083671374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 msgr2=0x565535094e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 --2- 172.21.15.32:0/4083671374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535094e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 -- 172.21.15.32:0/4083671374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565534dd47e0 con 0x5655350953d0 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa5277fe640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa5140a3d20 0x7fa5140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa526ffd640 1 --2- 172.21.15.32:0/4083671374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x56553509da70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa51800ac20 tx=0x7fa51800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65359135aaf8259 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa524ff9640 1 -- 172.21.15.32:0/2688073172 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7fa5200f2c90 con 0x7fa52819a170 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa51d7fa640 1 -- 172.21.15.32:0/4083671374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51801d070 con 0x5655350953d0 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa51d7fa640 1 -- 172.21.15.32:0/4083671374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa518018b20 con 0x5655350953d0 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.229+0000 7fa51d7fa640 1 -- 172.21.15.32:0/4083671374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518010c00 con 0x5655350953d0 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 -- 172.21.15.32:0/4083671374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 msgr2=0x56553509da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa5277fe640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa5140a3d20 0x7fa5140a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5200f05f0 tx=0x7fa520090390 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 --2- 172.21.15.32:0/4083671374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x56553509da70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa51800ac20 tx=0x7fa51800f790 comp rx=0 tx=0).stop 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 -- 172.21.15.32:0/4083671374 shutdown_connections 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 --2- 172.21.15.32:0/4083671374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x5655350a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 --2- 172.21.15.32:0/4083671374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x56553509da70 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 --2- 172.21.15.32:0/4083671374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535094e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 -- 172.21.15.32:0/4083671374 >> 172.21.15.32:0/4083671374 conn(0x56553508f1e0 msgr2=0x56553508f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 -- 172.21.15.32:0/4083671374 shutdown_connections 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 -- 172.21.15.32:0/4083671374 wait complete. 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.230+0000 7fa530022740 1 -- start start 2024-04-16T05:43:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa530022740 1 Processor -- start 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa530022740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565535094ab0 0x56553509b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa530022740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5655350953d0 0x56553509b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa530022740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x56553509bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa530022740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5655350a0da0 con 0x565535094ab0 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa530022740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565534f82410 con 0x5655350953d0 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa530022740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5655350a0f20 con 0x56553509dfb0 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa530022740 10 client.0 _ll_register_callbacks cb 0x565534f7db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565535094ab0 0x56553509b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565535094ab0 0x56553509b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40606/0 (socket says 172.21.15.32:40606) 2024-04-16T05:43:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 -- 172.21.15.32:0/2052616857 learned_addr learned my addr 172.21.15.32:0/2052616857 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5655350953d0 0x56553509b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x56553509bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 -- 172.21.15.32:0/2052616857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 msgr2=0x56553509bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x56553509bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 -- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5655350953d0 msgr2=0x56553509b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5655350953d0 0x56553509b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 -- 172.21.15.32:0/2052616857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565534e34f70 con 0x565535094ab0 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5655350953d0 0x56553509b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565535094ab0 0x56553509b0a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa510005fd0 tx=0x7fa51000d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40944f188f59af6c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.231+0000 7fa50e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2052616857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510019070 con 0x565535094ab0 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 1 -- 172.21.15.32:0/2052616857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa518018b20 con 0x565535094ab0 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2052616857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7fa510004220 con 0x565535094ab0 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2052616857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa510026050 con 0x565535094ab0 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 10 client.17577 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 1 -- 172.21.15.32:0/2052616857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565534ccc170 con 0x565535094ab0 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2052616857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa518018b20 con 0x565535094ab0 2024-04-16T05:43:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2052616857 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5100038c0 con 0x565535094ab0 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 10 client.17577 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 20 client.17577 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5655350ae480 0x5655350b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 4 client.17577 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.232+0000 7fa530022740 1 -- 172.21.15.32:0/2052616857 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655350078b0 con 0x5655350ae480 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.233+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5655350ae480 0x5655350b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.233+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5655350ae480 0x5655350b0890 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5612f31a13e1f267 server_cookie=b03a8258e19ffaaf in_seq=0 out_seq=0 2024-04-16T05:43:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.233+0000 7fa50e7fc640 10 client.17577 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2052616857 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5655350078b0 con 0x5655350ae480 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa50e7fc640 10 client.17577 handle_command_reply: tid=0 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 client.17577 shutdown 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 -- 172.21.15.32:0/2052616857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565535094ab0 msgr2=0x56553509b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565535094ab0 0x56553509b0a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa510005fd0 tx=0x7fa51000d4f0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 -- 172.21.15.32:0/2052616857 shutdown_connections 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5655350ae480 0x5655350b0890 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x56553509bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5655350953d0 0x56553509b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 --2- 172.21.15.32:0/2052616857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565535094ab0 0x56553509b0a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa50e7fc640 0 client.17577 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 -- 172.21.15.32:0/2052616857 >> 172.21.15.32:0/2052616857 conn(0x56553508f1e0 msgr2=0x565535090f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 -- 172.21.15.32:0/2052616857 shutdown_connections 2024-04-16T05:43:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 1 -- 172.21.15.32:0/2052616857 wait complete. 2024-04-16T05:43:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.234+0000 7fa530022740 20 client.17577 trim_cache size 0 max 16384 2024-04-16T05:43:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa530022740 1 Processor -- start 2024-04-16T05:43:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa530022740 1 -- start start 2024-04-16T05:43:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa530022740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x5655350a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa530022740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x5655350a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa530022740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x56553509b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa530022740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281b1e90 con 0x5655350953d0 2024-04-16T05:43:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa530022740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281af760 con 0x565535094ab0 2024-04-16T05:43:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa530022740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281ad5d0 con 0x56553509dfb0 2024-04-16T05:43:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x5655350a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x5655350a6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40620/0 (socket says 172.21.15.32:40620) 2024-04-16T05:43:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 -- 172.21.15.32:0/2143117680 learned_addr learned my addr 172.21.15.32:0/2143117680 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x56553509b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x5655350a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 -- 172.21.15.32:0/2143117680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 msgr2=0x56553509b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x56553509b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 -- 172.21.15.32:0/2143117680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 msgr2=0x5655350a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x5655350a9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 -- 172.21.15.32:0/2143117680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553508d660 con 0x5655350953d0 2024-04-16T05:43:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x5655350a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.252+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x5655350a6580 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa51801a000 tx=0x7fa51800ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7548708bf8c7370 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa51cff9640 1 -- 172.21.15.32:0/2143117680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518028e60 con 0x5655350953d0 2024-04-16T05:43:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa51cff9640 1 -- 172.21.15.32:0/2143117680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51802c030 con 0x5655350953d0 2024-04-16T05:43:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa51cff9640 1 -- 172.21.15.32:0/2143117680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51801d040 con 0x5655350953d0 2024-04-16T05:43:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 -- 172.21.15.32:0/2143117680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 msgr2=0x5655350a6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x5655350a6580 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa51801a000 tx=0x7fa51800ab90 comp rx=0 tx=0).stop 2024-04-16T05:43:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 -- 172.21.15.32:0/2143117680 shutdown_connections 2024-04-16T05:43:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x56553509b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x5655350a6580 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 --2- 172.21.15.32:0/2143117680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x5655350a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 -- 172.21.15.32:0/2143117680 >> 172.21.15.32:0/2143117680 conn(0x56553508f1e0 msgr2=0x56553508e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 -- 172.21.15.32:0/2143117680 shutdown_connections 2024-04-16T05:43:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.253+0000 7fa530022740 1 -- 172.21.15.32:0/2143117680 wait complete. 2024-04-16T05:43:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 1 -- start start 2024-04-16T05:43:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 1 Processor -- start 2024-04-16T05:43:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535098a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x565535097310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x565535097850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281adc20 con 0x5655350953d0 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281af090 con 0x565535094ab0 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281ae2f0 con 0x56553509dfb0 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa530022740 10 client.0 _ll_register_callbacks cb 0x7fa51000d370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa526ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x565535097310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa526ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x565535097310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40622/0 (socket says 172.21.15.32:40622) 2024-04-16T05:43:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa526ffd640 1 -- 172.21.15.32:0/2066604463 learned_addr learned my addr 172.21.15.32:0/2066604463 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x565535097850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa526ffd640 1 -- 172.21.15.32:0/2066604463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 msgr2=0x565535097850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x565535097850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa526ffd640 1 -- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 msgr2=0x565535098a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535098a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa526ffd640 1 -- 172.21.15.32:0/2066604463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565535097e90 con 0x5655350953d0 2024-04-16T05:43:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.254+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa526ffd640 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x565535097310 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa51800a3b0 tx=0x7fa51800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757d5e271d6ced08 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa50e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2066604463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51801d070 con 0x5655350953d0 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2066604463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7fa518018540 con 0x5655350953d0 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 1 -- 172.21.15.32:0/2066604463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa51802c030 con 0x5655350953d0 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2066604463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa518042050 con 0x5655350953d0 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 10 client.17583 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 1 -- 172.21.15.32:0/2066604463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565535119830 con 0x5655350953d0 2024-04-16T05:43:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2066604463 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51802c030 con 0x5655350953d0 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2066604463 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa518028160 con 0x5655350953d0 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 10 client.17583 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 20 client.17583 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x565535119db0 0x56553511c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 4 client.17583 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.255+0000 7fa530022740 1 -- 172.21.15.32:0/2066604463 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565535001720 con 0x565535119db0 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.256+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x565535119db0 0x56553511c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.256+0000 7fa51ffff640 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x565535119db0 0x56553511c1d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60567c541aa09758 server_cookie=6d2c03b514a0ab4 in_seq=0 out_seq=0 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.256+0000 7fa50e7fc640 10 client.17583 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa50e7fc640 1 -- 172.21.15.32:0/2066604463 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565535001720 con 0x565535119db0 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa50e7fc640 10 client.17583 handle_command_reply: tid=0 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 client.17583 shutdown 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 -- 172.21.15.32:0/2066604463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 msgr2=0x565535097310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x565535097310 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa51800a3b0 tx=0x7fa51800f790 comp rx=0 tx=0).stop 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 -- 172.21.15.32:0/2066604463 shutdown_connections 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x565535119db0 0x56553511c1d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56553509dfb0 0x565535097850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa50e7fc640 0 client.17583 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5655350953d0 0x565535097310 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 --2- 172.21.15.32:0/2066604463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565535094ab0 0x565535098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.257+0000 7fa530022740 1 -- 172.21.15.32:0/2066604463 >> 172.21.15.32:0/2066604463 conn(0x56553508f1e0 msgr2=0x565535090610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa530022740 1 -- 172.21.15.32:0/2066604463 shutdown_connections 2024-04-16T05:43:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa530022740 1 -- 172.21.15.32:0/2066604463 wait complete. 2024-04-16T05:43:09.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa530022740 20 client.17583 trim_cache size 0 max 16384 2024-04-16T05:43:09.267 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:43:09.267 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:43:09.267 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "18462074-4bbd-4d47-be81-3386dea9be34", 2024-04-16T05:43:09.267 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:43:09.267 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:43:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 -- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa5140a3d20 msgr2=0x7fa5140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa5140a3d20 0x7fa5140a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5200f05f0 tx=0x7fa520090390 comp rx=0 tx=0).stop 2024-04-16T05:43:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 -- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa52819a170 msgr2=0x7fa5281a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa52819a170 0x7fa5281a4720 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fa52002c700 tx=0x7fa520032f20 comp rx=0 tx=0).stop 2024-04-16T05:43:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 -- 172.21.15.32:0/2688073172 shutdown_connections 2024-04-16T05:43:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa5140a3d20 0x7fa5140a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa52819a170 0x7fa5281a4720 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa528106220 0x7fa528199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 --2- 172.21.15.32:0/2688073172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa528101150 0x7fa5281996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 -- 172.21.15.32:0/2688073172 >> 172.21.15.32:0/2688073172 conn(0x7fa528077f60 msgr2=0x7fa5280fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 -- 172.21.15.32:0/2688073172 shutdown_connections 2024-04-16T05:43:09.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.258+0000 7fa52d359640 1 -- 172.21.15.32:0/2688073172 wait complete. 2024-04-16T05:43:09.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:43:09.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:43:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 -- 172.21.15.32:0/1824378220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101af0 msgr2=0x7f13b4105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 --2- 172.21.15.32:0/1824378220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101af0 0x7f13b4105b30 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f139c009900 tx=0x7f139c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 -- 172.21.15.32:0/1824378220 shutdown_connections 2024-04-16T05:43:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 --2- 172.21.15.32:0/1824378220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13b4106210 0x7f13b410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 --2- 172.21.15.32:0/1824378220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101af0 0x7f13b4105b30 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 --2- 172.21.15.32:0/1824378220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13b4101140 0x7f13b4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 -- 172.21.15.32:0/1824378220 >> 172.21.15.32:0/1824378220 conn(0x7f13b4078000 msgr2=0x7f13b4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 -- 172.21.15.32:0/1824378220 shutdown_connections 2024-04-16T05:43:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.418+0000 7f13b8d75640 1 -- 172.21.15.32:0/1824378220 wait complete. 2024-04-16T05:43:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b8d75640 1 Processor -- start 2024-04-16T05:43:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b8d75640 1 -- start start 2024-04-16T05:43:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b8d75640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101140 0x7f13b419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b8d75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13b4106210 0x7f13b419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b8d75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13b4198740 0x7f13b4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b8d75640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41123e0 con 0x7f13b4101140 2024-04-16T05:43:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b8d75640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4112260 con 0x7f13b4198740 2024-04-16T05:43:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b8d75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4112560 con 0x7f13b4106210 2024-04-16T05:43:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101140 0x7f13b419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101140 0x7f13b419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40640/0 (socket says 172.21.15.32:40640) 2024-04-16T05:43:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 -- 172.21.15.32:0/991891026 learned_addr learned my addr 172.21.15.32:0/991891026 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13abfff640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13b4106210 0x7f13b419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13b4198740 0x7f13b4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 -- 172.21.15.32:0/991891026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13b4106210 msgr2=0x7f13b419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13b4106210 0x7f13b419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 -- 172.21.15.32:0/991891026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13b4198740 msgr2=0x7f13b4198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13b4198740 0x7f13b4198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 -- 172.21.15.32:0/991891026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b4199340 con 0x7f13b4101140 2024-04-16T05:43:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13b4198740 0x7f13b4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.419+0000 7f13b259c640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101140 0x7f13b419e680 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f13a400e9f0 tx=0x7f13a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.420+0000 7f13ab7fe640 1 -- 172.21.15.32:0/991891026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a400cd80 con 0x7f13b4101140 2024-04-16T05:43:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.420+0000 7f13b8d75640 1 -- 172.21.15.32:0/991891026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b41a5420 con 0x7f13b4101140 2024-04-16T05:43:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.420+0000 7f13ab7fe640 1 -- 172.21.15.32:0/991891026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a4005500 con 0x7f13b4101140 2024-04-16T05:43:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.420+0000 7f13ab7fe640 1 -- 172.21.15.32:0/991891026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4010690 con 0x7f13b4101140 2024-04-16T05:43:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.420+0000 7f13b8d75640 1 -- 172.21.15.32:0/991891026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41a57d0 con 0x7f13b4101140 2024-04-16T05:43:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.422+0000 7f13ab7fe640 1 -- 172.21.15.32:0/991891026 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f13a40046c0 con 0x7f13b4101140 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.422+0000 7f13bba39740 1 Processor -- start 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.422+0000 7f13bba39740 1 -- start start 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.422+0000 7f13bba39740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec7434230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13bba39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7434770 0x55bec743ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13bba39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13bba39740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bec7399180 con 0x55bec7433e50 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13bba39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bec7336360 con 0x55bec7434770 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13bba39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bec7395140 con 0x55bec743d350 2024-04-16T05:43:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec7434230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec7434230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40654/0 (socket says 172.21.15.32:40654) 2024-04-16T05:43:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 -- 172.21.15.32:0/2235884400 learned_addr learned my addr 172.21.15.32:0/2235884400 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13ab7fe640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f13a00a3d20 0x7f13a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b259c640 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7434770 0x55bec743ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13abfff640 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 -- 172.21.15.32:0/2235884400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 msgr2=0x55bec743f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 -- 172.21.15.32:0/2235884400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7434770 msgr2=0x55bec743ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7434770 0x55bec743ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 -- 172.21.15.32:0/2235884400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec71757d0 con 0x55bec7433e50 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13abfff640 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b259c640 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7434770 0x55bec743ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13ab7fe640 1 -- 172.21.15.32:0/991891026 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f13a40d1110 con 0x7f13b4101140 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f13a00a3d20 0x7f13a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec7434230 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00e370 tx=0x7f13ac00e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2c659cbcd6d010 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13a8ff9640 1 -- 172.21.15.32:0/2235884400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac012d20 con 0x55bec7433e50 2024-04-16T05:43:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13a8ff9640 1 -- 172.21.15.32:0/2235884400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac0123f0 con 0x55bec7433e50 2024-04-16T05:43:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.423+0000 7f13a8ff9640 1 -- 172.21.15.32:0/2235884400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac005390 con 0x55bec7433e50 2024-04-16T05:43:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 -- 172.21.15.32:0/2235884400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 msgr2=0x55bec7434230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec7434230 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00e370 tx=0x7f13ac00e830 comp rx=0 tx=0).stop 2024-04-16T05:43:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f13a00a3d20 0x7f13a00a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13ac01f000 tx=0x7f13ac020000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 -- 172.21.15.32:0/2235884400 shutdown_connections 2024-04-16T05:43:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7434770 0x55bec743ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 --2- 172.21.15.32:0/2235884400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec7434230 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 -- 172.21.15.32:0/2235884400 >> 172.21.15.32:0/2235884400 conn(0x55bec742e580 msgr2=0x55bec742e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 -- 172.21.15.32:0/2235884400 shutdown_connections 2024-04-16T05:43:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.424+0000 7f13bba39740 1 -- 172.21.15.32:0/2235884400 wait complete. 2024-04-16T05:43:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 1 -- start start 2024-04-16T05:43:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 1 Processor -- start 2024-04-16T05:43:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec743a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec743a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bec7440140 con 0x55bec7434770 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bec73522c0 con 0x55bec7433e50 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bec74402c0 con 0x55bec743d350 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13bba39740 10 client.0 _ll_register_callbacks cb 0x55bec7339090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec743a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec743a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40660/0 (socket says 172.21.15.32:40660) 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13abfff640 1 -- 172.21.15.32:0/215877780 learned_addr learned my addr 172.21.15.32:0/215877780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13b259c640 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec743a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13abfff640 1 -- 172.21.15.32:0/215877780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 msgr2=0x55bec743ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13abfff640 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13abfff640 1 -- 172.21.15.32:0/215877780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 msgr2=0x55bec743a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13abfff640 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec743a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.425+0000 7f13abfff640 1 -- 172.21.15.32:0/215877780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec70ab3f0 con 0x55bec7434770 2024-04-16T05:43:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13abfff640 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec743a920 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f139c005ab0 tx=0x7f139c0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7850ad8a104b584 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f1399ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f1399ffb640 1 -- 172.21.15.32:0/215877780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c005020 con 0x55bec7434770 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f1399ffb640 1 -- 172.21.15.32:0/215877780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f139c0051c0 con 0x55bec7434770 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 1 -- 172.21.15.32:0/215877780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ac0123f0 con 0x55bec7434770 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f1399ffb640 1 -- 172.21.15.32:0/215877780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f139c04c050 con 0x55bec7434770 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 10 client.17595 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 1 -- 172.21.15.32:0/215877780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bec706b0f0 con 0x55bec7434770 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f1399ffb640 1 -- 172.21.15.32:0/215877780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac0123f0 con 0x55bec7434770 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f1399ffb640 1 -- 172.21.15.32:0/215877780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f139c0035f0 con 0x55bec7434770 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 10 client.17595 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 20 client.17595 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bec744d6a0 0x55bec744fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 4 client.17595 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.426+0000 7f13bba39740 1 -- 172.21.15.32:0/215877780 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bec73a6ba0 con 0x55bec744d6a0 2024-04-16T05:43:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.427+0000 7f13b259c640 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bec744d6a0 0x55bec744fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.427+0000 7f1399ffb640 10 client.17595 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.427+0000 7f13b259c640 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bec744d6a0 0x55bec744fad0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0bf22d4464ca37 server_cookie=617d471ec99cd68f in_seq=0 out_seq=0 2024-04-16T05:43:09.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f1399ffb640 1 -- 172.21.15.32:0/215877780 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bec73a6ba0 con 0x55bec744d6a0 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f1399ffb640 10 client.17595 handle_command_reply: tid=0 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 client.17595 shutdown 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 -- 172.21.15.32:0/215877780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 msgr2=0x55bec743a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec743a920 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f139c005ab0 tx=0x7f139c0095e0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 -- 172.21.15.32:0/215877780 shutdown_connections 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bec744d6a0 0x55bec744fad0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec743ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec743a920 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 --2- 172.21.15.32:0/215877780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec743a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f1399ffb640 0 client.17595 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 -- 172.21.15.32:0/215877780 >> 172.21.15.32:0/215877780 conn(0x55bec742e580 msgr2=0x55bec74302d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 -- 172.21.15.32:0/215877780 shutdown_connections 2024-04-16T05:43:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 1 -- 172.21.15.32:0/215877780 wait complete. 2024-04-16T05:43:09.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.428+0000 7f13bba39740 20 client.17595 trim_cache size 0 max 16384 2024-04-16T05:43:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.445+0000 7f13bba39740 1 Processor -- start 2024-04-16T05:43:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.445+0000 7f13bba39740 1 -- start start 2024-04-16T05:43:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13bba39740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec743cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13bba39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec7434770 0x55bec7449000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13bba39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec743d350 0x55bec74553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13bba39740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b1e50 con 0x55bec7433e50 2024-04-16T05:43:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13bba39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41af720 con 0x55bec743d350 2024-04-16T05:43:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13bba39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ad590 con 0x55bec7434770 2024-04-16T05:43:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec743cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec743cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40664/0 (socket says 172.21.15.32:40664) 2024-04-16T05:43:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 -- 172.21.15.32:0/1479142319 learned_addr learned my addr 172.21.15.32:0/1479142319 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13abfff640 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec7434770 0x55bec7449000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec743d350 0x55bec74553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 -- 172.21.15.32:0/1479142319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec7434770 msgr2=0x55bec7449000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec7434770 0x55bec7449000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 -- 172.21.15.32:0/1479142319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec743d350 msgr2=0x55bec74553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec743d350 0x55bec74553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 -- 172.21.15.32:0/1479142319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec743f740 con 0x55bec7433e50 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f13b259c640 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec743cb10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f13a40cea70 tx=0x7f13a40d3620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec025e016ecdb528 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f139bfff640 1 -- 172.21.15.32:0/1479142319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40aeae0 con 0x55bec7433e50 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f139bfff640 1 -- 172.21.15.32:0/1479142319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a40a5930 con 0x55bec7433e50 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.446+0000 7f139bfff640 1 -- 172.21.15.32:0/1479142319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4099330 con 0x55bec7433e50 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 -- 172.21.15.32:0/1479142319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 msgr2=0x55bec743cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec743cb10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f13a40cea70 tx=0x7f13a40d3620 comp rx=0 tx=0).stop 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 -- 172.21.15.32:0/1479142319 shutdown_connections 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec743d350 0x55bec74553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec7434770 0x55bec7449000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 --2- 172.21.15.32:0/1479142319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7433e50 0x55bec743cb10 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 -- 172.21.15.32:0/1479142319 >> 172.21.15.32:0/1479142319 conn(0x55bec742e580 msgr2=0x55bec742d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 -- 172.21.15.32:0/1479142319 shutdown_connections 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 -- 172.21.15.32:0/1479142319 wait complete. 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 -- start start 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.447+0000 7f13bba39740 1 Processor -- start 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13bba39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec74435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13bba39740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec7443b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13bba39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec74b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13bba39740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41adbe0 con 0x55bec7434770 2024-04-16T05:43:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13bba39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41af050 con 0x55bec7433e50 2024-04-16T05:43:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13bba39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ae2b0 con 0x55bec743d350 2024-04-16T05:43:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13bba39740 10 client.0 _ll_register_callbacks cb 0x7f13a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec7443b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec7443b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40666/0 (socket says 172.21.15.32:40666) 2024-04-16T05:43:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 -- 172.21.15.32:0/2595545537 learned_addr learned my addr 172.21.15.32:0/2595545537 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13b259c640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec74435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec74b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 -- 172.21.15.32:0/2595545537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 msgr2=0x55bec74b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec74b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 -- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 msgr2=0x55bec74435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec74435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 -- 172.21.15.32:0/2595545537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec74b3160 con 0x55bec7434770 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13b259c640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec74435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec74b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13abfff640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec7443b20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f139c000c00 tx=0x7f139c003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728a2f8ad2479cea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f1399ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f1399ffb640 1 -- 172.21.15.32:0/2595545537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c003e00 con 0x55bec7434770 2024-04-16T05:43:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f1399ffb640 1 -- 172.21.15.32:0/2595545537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9526+0+0 (secure 0 0 0) 0x7f139c0046c0 con 0x55bec7434770 2024-04-16T05:43:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f13bba39740 1 -- 172.21.15.32:0/2595545537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a40a5930 con 0x55bec7434770 2024-04-16T05:43:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.448+0000 7f1399ffb640 1 -- 172.21.15.32:0/2595545537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f139c0048c0 con 0x55bec7434770 2024-04-16T05:43:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13bba39740 10 client.17601 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13bba39740 1 -- 172.21.15.32:0/2595545537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bec74b4370 con 0x55bec7434770 2024-04-16T05:43:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f1399ffb640 1 -- 172.21.15.32:0/2595545537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a40a5930 con 0x55bec7434770 2024-04-16T05:43:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f1399ffb640 1 -- 172.21.15.32:0/2595545537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f139c004a60 con 0x55bec7434770 2024-04-16T05:43:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13bba39740 10 client.17601 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13bba39740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13bba39740 20 client.17601 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13bba39740 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bec74b4ae0 0x55bec74b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13bba39740 4 client.17601 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:43:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13bba39740 1 -- 172.21.15.32:0/2595545537 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bec73a0a10 con 0x55bec74b4ae0 2024-04-16T05:43:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.449+0000 7f13b259c640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bec74b4ae0 0x55bec74b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13b259c640 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bec74b4ae0 0x55bec74b6ed0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae83d885e9ebe8ef server_cookie=64b82142564012a2 in_seq=0 out_seq=0 2024-04-16T05:43:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f1399ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f1399ffb640 1 -- 172.21.15.32:0/2595545537 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bec73a0a10 con 0x55bec74b4ae0 2024-04-16T05:43:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f1399ffb640 10 client.17601 handle_command_reply: tid=0 2024-04-16T05:43:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 client.17601 shutdown 2024-04-16T05:43:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 -- 172.21.15.32:0/2595545537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 msgr2=0x55bec7443b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec7443b20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f139c000c00 tx=0x7f139c003880 comp rx=0 tx=0).stop 2024-04-16T05:43:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 -- 172.21.15.32:0/2595545537 shutdown_connections 2024-04-16T05:43:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55bec74b4ae0 0x55bec74b6ed0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bec743d350 0x55bec74b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f1399ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bec7434770 0x55bec7443b20 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 --2- 172.21.15.32:0/2595545537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bec7433e50 0x55bec74435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 -- 172.21.15.32:0/2595545537 >> 172.21.15.32:0/2595545537 conn(0x55bec742e580 msgr2=0x55bec743a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 -- 172.21.15.32:0/2595545537 shutdown_connections 2024-04-16T05:43:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 1 -- 172.21.15.32:0/2595545537 wait complete. 2024-04-16T05:43:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13bba39740 20 client.17601 trim_cache size 0 max 16384 2024-04-16T05:43:09.461 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:43:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:43:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:43:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "18462074-4bbd-4d47-be81-3386dea9be34": { 2024-04-16T05:43:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:43:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "18462074-4bbd-4d47-be81-3386dea9be34", 2024-04-16T05:43:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:43:09.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:43:09.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:43:09.462 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:43:09.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13b8d75640 1 -- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f13a00a3d20 msgr2=0x7f13a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13b8d75640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f13a00a3d20 0x7f13a00a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13ac01f000 tx=0x7f13ac020000 comp rx=0 tx=0).stop 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13b8d75640 1 -- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101140 msgr2=0x7f13b419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.450+0000 7f13b8d75640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101140 0x7f13b419e680 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f13a400e9f0 tx=0x7f13a400edf0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.451+0000 7f13b8d75640 1 -- 172.21.15.32:0/991891026 shutdown_connections 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.451+0000 7f13b8d75640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13b4198740 0x7f13b4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.451+0000 7f13b8d75640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13b4106210 0x7f13b419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.451+0000 7f13b8d75640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f13a00a3d20 0x7f13a00a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.451+0000 7f13b8d75640 1 --2- 172.21.15.32:0/991891026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b4101140 0x7f13b419e680 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.451+0000 7f13b8d75640 1 -- 172.21.15.32:0/991891026 >> 172.21.15.32:0/991891026 conn(0x7f13b4078000 msgr2=0x7f13b40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:09.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.451+0000 7f13b8d75640 1 -- 172.21.15.32:0/991891026 shutdown_connections 2024-04-16T05:43:09.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:09.451+0000 7f13b8d75640 1 -- 172.21.15.32:0/991891026 wait complete. 2024-04-16T05:43:09.465 INFO:tasks.cephfs.filesystem:scrub status for tag:18462074-4bbd-4d47-be81-3386dea9be34 - {'path': '/', 'tag': '18462074-4bbd-4d47-be81-3386dea9be34', 'options': 'recursive,force'} 2024-04-16T05:43:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:09 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:43:10.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:09 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:10.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:09 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:43:10.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:09 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:10.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:09 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:43:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:09 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:09 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:43:10.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:09 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:10.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:09 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:43:10.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:09 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:10.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:09 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:43:10.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:09 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:11.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:10 smithi134 ceph-mon[36153]: pgmap v253: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:11.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:10 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:43:11.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:10 smithi082 ceph-mon[36195]: pgmap v253: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:11.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:10 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:43:11.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:10 smithi032 ceph-mon[32351]: pgmap v253: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:43:11.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:10 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:43:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:13 smithi134 ceph-mon[36153]: pgmap v254: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:43:14.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:13 smithi134 ceph-mon[36153]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T05:43:14.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:13 smithi082 ceph-mon[36195]: pgmap v254: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:43:14.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:13 smithi082 ceph-mon[36195]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T05:43:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:13 smithi032 ceph-mon[32351]: pgmap v254: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:43:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:13 smithi032 ceph-mon[32351]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T05:43:16.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:15 smithi134 ceph-mon[36153]: pgmap v256: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 246 B/s wr, 9 op/s 2024-04-16T05:43:16.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:15 smithi082 ceph-mon[36195]: pgmap v256: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 246 B/s wr, 9 op/s 2024-04-16T05:43:16.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:15 smithi032 ceph-mon[32351]: pgmap v256: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 246 B/s wr, 9 op/s 2024-04-16T05:43:18.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:17 smithi134 ceph-mon[36153]: pgmap v257: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:43:18.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:17 smithi082 ceph-mon[36195]: pgmap v257: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:43:18.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:17 smithi032 ceph-mon[32351]: pgmap v257: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-16T05:43:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:19 smithi134 ceph-mon[36153]: pgmap v258: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:43:20.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:19 smithi082 ceph-mon[36195]: pgmap v258: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:43:20.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:19 smithi032 ceph-mon[32351]: pgmap v258: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:43:22.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:21 smithi134 ceph-mon[36153]: pgmap v259: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:43:22.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:21 smithi082 ceph-mon[36195]: pgmap v259: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:43:22.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:21 smithi032 ceph-mon[32351]: pgmap v259: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:43:24.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:23 smithi134 ceph-mon[36153]: pgmap v260: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:43:24.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:23 smithi082 ceph-mon[36195]: pgmap v260: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:43:24.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:23 smithi032 ceph-mon[32351]: pgmap v260: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:43:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:25 smithi134 ceph-mon[36153]: pgmap v261: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:43:26.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:25 smithi082 ceph-mon[36195]: pgmap v261: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:43:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:25 smithi032 ceph-mon[32351]: pgmap v261: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:43:28.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:27 smithi134 ceph-mon[36153]: pgmap v262: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:28.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:27 smithi082 ceph-mon[36195]: pgmap v262: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:28.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:27 smithi032 ceph-mon[32351]: pgmap v262: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:30.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:29 smithi134 ceph-mon[36153]: pgmap v263: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:30.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:29 smithi082 ceph-mon[36195]: pgmap v263: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:30.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:29 smithi032 ceph-mon[32351]: pgmap v263: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:32.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:31 smithi134 ceph-mon[36153]: pgmap v264: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:32.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:31 smithi082 ceph-mon[36195]: pgmap v264: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:32.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:31 smithi032 ceph-mon[32351]: pgmap v264: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:33 smithi134 ceph-mon[36153]: pgmap v265: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:34.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:33 smithi082 ceph-mon[36195]: pgmap v265: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:34.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:33 smithi032 ceph-mon[32351]: pgmap v265: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:35 smithi134 ceph-mon[36153]: pgmap v266: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:36.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:35 smithi082 ceph-mon[36195]: pgmap v266: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:36.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:35 smithi032 ceph-mon[32351]: pgmap v266: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:37 smithi134 ceph-mon[36153]: pgmap v267: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:38.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:37 smithi082 ceph-mon[36195]: pgmap v267: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:38.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:37 smithi032 ceph-mon[32351]: pgmap v267: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:39.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:43:39.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:43:39.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 -- 172.21.15.32:0/2541139993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c0742d0 msgr2=0x7f8c8c0746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/2541139993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c0742d0 0x7f8c8c0746b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009900 tx=0x7f8c7c02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 -- 172.21.15.32:0/2541139993 shutdown_connections 2024-04-16T05:43:39.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/2541139993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c8c10e8e0 0x7f8c8c110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/2541139993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c074bf0 0x7f8c8c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/2541139993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c0742d0 0x7f8c8c0746b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 -- 172.21.15.32:0/2541139993 >> 172.21.15.32:0/2541139993 conn(0x7f8c8c079450 msgr2=0x7f8c8c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:39.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 -- 172.21.15.32:0/2541139993 shutdown_connections 2024-04-16T05:43:39.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 -- 172.21.15.32:0/2541139993 wait complete. 2024-04-16T05:43:39.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.591+0000 7f8c91e2a640 1 Processor -- start 2024-04-16T05:43:39.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c91e2a640 1 -- start start 2024-04-16T05:43:39.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c91e2a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c0742d0 0x7f8c8c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c91e2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c074bf0 0x7f8c8c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c91e2a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c8c10e8e0 0x7f8c8c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c91e2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1145d0 con 0x7f8c8c074bf0 2024-04-16T05:43:39.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c91e2a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c114450 con 0x7f8c8c0742d0 2024-04-16T05:43:39.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c91e2a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c114750 con 0x7f8c8c10e8e0 2024-04-16T05:43:39.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c074bf0 0x7f8c8c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c074bf0 0x7f8c8c1a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60238/0 (socket says 172.21.15.32:60238) 2024-04-16T05:43:39.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8affd640 1 -- 172.21.15.32:0/1807732283 learned_addr learned my addr 172.21.15.32:0/1807732283 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:39.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c0742d0 0x7f8c8c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8bfff640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c8c10e8e0 0x7f8c8c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8affd640 1 -- 172.21.15.32:0/1807732283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c8c10e8e0 msgr2=0x7f8c8c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8affd640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c8c10e8e0 0x7f8c8c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8affd640 1 -- 172.21.15.32:0/1807732283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c0742d0 msgr2=0x7f8c8c1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8affd640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c0742d0 0x7f8c8c1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.592+0000 7f8c8affd640 1 -- 172.21.15.32:0/1807732283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c19b0d0 con 0x7f8c8c074bf0 2024-04-16T05:43:39.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.593+0000 7f8c8affd640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c074bf0 0x7f8c8c1a0ce0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c890 tx=0x7f8c8000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:39.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.593+0000 7f8c88ff9640 1 -- 172.21.15.32:0/1807732283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80004220 con 0x7f8c8c074bf0 2024-04-16T05:43:39.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.593+0000 7f8c91e2a640 1 -- 172.21.15.32:0/1807732283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c19b3c0 con 0x7f8c8c074bf0 2024-04-16T05:43:39.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.593+0000 7f8c88ff9640 1 -- 172.21.15.32:0/1807732283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c800043c0 con 0x7f8c8c074bf0 2024-04-16T05:43:39.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.593+0000 7f8c88ff9640 1 -- 172.21.15.32:0/1807732283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c800049e0 con 0x7f8c8c074bf0 2024-04-16T05:43:39.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.593+0000 7f8c91e2a640 1 -- 172.21.15.32:0/1807732283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c1a7660 con 0x7f8c8c074bf0 2024-04-16T05:43:39.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c88ff9640 1 -- 172.21.15.32:0/1807732283 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8c800079b0 con 0x7f8c8c074bf0 2024-04-16T05:43:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c94a14740 1 Processor -- start 2024-04-16T05:43:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c94a14740 1 -- start start 2024-04-16T05:43:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 0x56432a0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c94a14740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56432a02d180 con 0x56432a0c8770 2024-04-16T05:43:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c94a14740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564329fca360 con 0x56432a0c7e50 2024-04-16T05:43:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c94a14740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56432a029140 con 0x56432a0d1350 2024-04-16T05:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.596+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60254/0 (socket says 172.21.15.32:60254) 2024-04-16T05:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8b7fe640 1 -- 172.21.15.32:0/3292361082 learned_addr learned my addr 172.21.15.32:0/3292361082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8bfff640 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 0x56432a0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8affd640 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c88ff9640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8c740a3d20 0x7f8c740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8b7fe640 1 -- 172.21.15.32:0/3292361082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 msgr2=0x56432a0d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8b7fe640 1 -- 172.21.15.32:0/3292361082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 msgr2=0x56432a0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 0x56432a0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8b7fe640 1 -- 172.21.15.32:0/3292361082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564329e097d0 con 0x56432a0c8770 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8bfff640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8c740a3d20 0x7f8c740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c88ff9640 1 -- 172.21.15.32:0/1807732283 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f8c800d0b10 con 0x7f8c8c074bf0 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d0e10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009230 tx=0x7f8c7c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df2e64bb41038d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c71ffb640 1 -- 172.21.15.32:0/3292361082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c042070 con 0x56432a0c8770 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c71ffb640 1 -- 172.21.15.32:0/3292361082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c02cb20 con 0x56432a0c8770 2024-04-16T05:43:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c8bfff640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8c740a3d20 0x7f8c740a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c06b8d0 tx=0x7f8c84004a40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.597+0000 7f8c71ffb640 1 -- 172.21.15.32:0/3292361082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c005170 con 0x56432a0c8770 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 -- 172.21.15.32:0/3292361082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 msgr2=0x56432a0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d0e10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009230 tx=0x7f8c7c009670 comp rx=0 tx=0).stop 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 -- 172.21.15.32:0/3292361082 shutdown_connections 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d0e10 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 --2- 172.21.15.32:0/3292361082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 0x56432a0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 -- 172.21.15.32:0/3292361082 >> 172.21.15.32:0/3292361082 conn(0x56432a0c2580 msgr2=0x56432a0c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 -- 172.21.15.32:0/3292361082 shutdown_connections 2024-04-16T05:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 -- 172.21.15.32:0/3292361082 wait complete. 2024-04-16T05:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.598+0000 7f8c94a14740 1 -- start start 2024-04-16T05:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c94a14740 1 Processor -- start 2024-04-16T05:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c7e50 0x56432a0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c8770 0x56432a0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c94a14740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56432a0d40e0 con 0x56432a0c7e50 2024-04-16T05:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c94a14740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564329fe62c0 con 0x56432a0c8770 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c94a14740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56432a0d4260 con 0x56432a0d1350 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c94a14740 10 client.0 _ll_register_callbacks cb 0x564329fcd090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c7e50 0x56432a0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c7e50 0x56432a0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60264/0 (socket says 172.21.15.32:60264) 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8b7fe640 1 -- 172.21.15.32:0/491713153 learned_addr learned my addr 172.21.15.32:0/491713153 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8affd640 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c8770 0x56432a0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8bfff640 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8b7fe640 1 -- 172.21.15.32:0/491713153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 msgr2=0x56432a0cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8b7fe640 1 -- 172.21.15.32:0/491713153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c8770 msgr2=0x56432a0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c8770 0x56432a0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8b7fe640 1 -- 172.21.15.32:0/491713153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564329d3f3f0 con 0x56432a0c7e50 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.599+0000 7f8c8bfff640 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c7e50 0x56432a0ce370 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02c210 tx=0x7f8c7c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c084185b3eb95ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c6affd640 1 -- 172.21.15.32:0/491713153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c042040 con 0x56432a0c7e50 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c6affd640 1 -- 172.21.15.32:0/491713153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f8c7c004450 con 0x56432a0c7e50 2024-04-16T05:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 1 -- 172.21.15.32:0/491713153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c02cb20 con 0x56432a0c7e50 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c6affd640 1 -- 172.21.15.32:0/491713153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c058050 con 0x56432a0c7e50 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 10 client.17613 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 1 -- 172.21.15.32:0/491713153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564329cff0f0 con 0x56432a0c7e50 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c6affd640 1 -- 172.21.15.32:0/491713153 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c02cb20 con 0x56432a0c7e50 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c6affd640 1 -- 172.21.15.32:0/491713153 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c7c003b20 con 0x56432a0c7e50 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 10 client.17613 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 20 client.17613 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56432a0e1540 0x56432a0e3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 4 client.17613 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.600+0000 7f8c94a14740 1 -- 172.21.15.32:0/491713153 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56432a03aba0 con 0x56432a0e1540 2024-04-16T05:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.601+0000 7f8c8affd640 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56432a0e1540 0x56432a0e3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.601+0000 7f8c8affd640 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56432a0e1540 0x56432a0e3930 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a80f4cde68cecdb server_cookie=da80593f56c8bb79 in_seq=0 out_seq=0 2024-04-16T05:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.601+0000 7f8c6affd640 10 client.17613 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c6affd640 1 -- 172.21.15.32:0/491713153 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56432a03aba0 con 0x56432a0e1540 2024-04-16T05:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c6affd640 10 client.17613 handle_command_reply: tid=0 2024-04-16T05:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 client.17613 shutdown 2024-04-16T05:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 -- 172.21.15.32:0/491713153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c7e50 msgr2=0x56432a0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c7e50 0x56432a0ce370 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02c210 tx=0x7f8c7c02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 -- 172.21.15.32:0/491713153 shutdown_connections 2024-04-16T05:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56432a0e1540 0x56432a0e3930 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c8770 0x56432a0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c6affd640 0 client.17613 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 --2- 172.21.15.32:0/491713153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c7e50 0x56432a0ce370 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 -- 172.21.15.32:0/491713153 >> 172.21.15.32:0/491713153 conn(0x56432a0c2580 msgr2=0x56432a0c42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 -- 172.21.15.32:0/491713153 shutdown_connections 2024-04-16T05:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 1 -- 172.21.15.32:0/491713153 wait complete. 2024-04-16T05:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.602+0000 7f8c94a14740 20 client.17613 trim_cache size 0 max 16384 2024-04-16T05:43:39.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c94a14740 1 Processor -- start 2024-04-16T05:43:39.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c94a14740 1 -- start start 2024-04-16T05:43:39.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 0x56432a0dd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c94a14740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b3e20 con 0x56432a0c8770 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c94a14740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b16f0 con 0x56432a0c7e50 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c94a14740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1af540 con 0x56432a0d1350 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0cec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60268/0 (socket says 172.21.15.32:60268) 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8affd640 1 -- 172.21.15.32:0/3660675583 learned_addr learned my addr 172.21.15.32:0/3660675583 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/3660675583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 0x56432a0dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8affd640 1 -- 172.21.15.32:0/3660675583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 msgr2=0x56432a0e8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8affd640 1 --2- 172.21.15.32:0/3660675583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8affd640 1 -- 172.21.15.32:0/3660675583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 msgr2=0x56432a0dd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8affd640 1 --2- 172.21.15.32:0/3660675583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 0x56432a0dd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.620+0000 7f8c8affd640 1 -- 172.21.15.32:0/3660675583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432a0d3740 con 0x56432a0c8770 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c8affd640 1 --2- 172.21.15.32:0/3660675583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0cec10 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8c800058c0 tx=0x7f8c80087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e0df4a47466fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c717fa640 1 -- 172.21.15.32:0/3660675583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8001d040 con 0x56432a0c8770 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c717fa640 1 -- 172.21.15.32:0/3660675583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c800a0ad0 con 0x56432a0c8770 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c717fa640 1 -- 172.21.15.32:0/3660675583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c800d5040 con 0x56432a0c8770 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 -- 172.21.15.32:0/3660675583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 msgr2=0x56432a0cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 --2- 172.21.15.32:0/3660675583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0cec10 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8c800058c0 tx=0x7f8c80087040 comp rx=0 tx=0).stop 2024-04-16T05:43:39.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 -- 172.21.15.32:0/3660675583 shutdown_connections 2024-04-16T05:43:39.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 --2- 172.21.15.32:0/3660675583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0d1350 0x56432a0e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 --2- 172.21.15.32:0/3660675583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0cec10 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 --2- 172.21.15.32:0/3660675583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0c7e50 0x56432a0dd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 -- 172.21.15.32:0/3660675583 >> 172.21.15.32:0/3660675583 conn(0x56432a0c2580 msgr2=0x56432a0c1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:39.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 -- 172.21.15.32:0/3660675583 shutdown_connections 2024-04-16T05:43:39.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.621+0000 7f8c94a14740 1 -- 172.21.15.32:0/3660675583 wait complete. 2024-04-16T05:43:39.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 1 -- start start 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 1 Processor -- start 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0c7e50 0x56432a0d9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0d1350 0x56432a0d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1afbb0 con 0x56432a0c8770 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b1020 con 0x56432a0d1350 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b0280 con 0x56432a0c7e50 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c94a14740 10 client.0 _ll_register_callbacks cb 0x7f8c7c005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60282/0 (socket says 172.21.15.32:60282) 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.622+0000 7f8c8affd640 1 -- 172.21.15.32:0/2198776563 learned_addr learned my addr 172.21.15.32:0/2198776563 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c8affd640 1 -- 172.21.15.32:0/2198776563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0c7e50 msgr2=0x56432a0d9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c8affd640 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0c7e50 0x56432a0d9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c8affd640 1 -- 172.21.15.32:0/2198776563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0d1350 msgr2=0x56432a0d72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c8affd640 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0d1350 0x56432a0d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c8affd640 1 -- 172.21.15.32:0/2198776563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56432a0d7af0 con 0x56432a0c8770 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c8affd640 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d6d20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8c80005fd0 tx=0x7f8c800054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d91372efb53a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c6affd640 1 -- 172.21.15.32:0/2198776563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c800d5050 con 0x56432a0c8770 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c6affd640 1 -- 172.21.15.32:0/2198776563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f8c800ce150 con 0x56432a0c8770 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 1 -- 172.21.15.32:0/2198776563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c800a0ad0 con 0x56432a0c8770 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c6affd640 1 -- 172.21.15.32:0/2198776563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c8001d070 con 0x56432a0c8770 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 10 client.17619 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 1 -- 172.21.15.32:0/2198776563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56432a147940 con 0x56432a0c8770 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c6affd640 1 -- 172.21.15.32:0/2198776563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c800a0ad0 con 0x56432a0c8770 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c6affd640 1 -- 172.21.15.32:0/2198776563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c800a4f00 con 0x56432a0c8770 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 10 client.17619 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 20 client.17619 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56432a148110 0x56432a14a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 4 client.17619 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.623+0000 7f8c94a14740 1 -- 172.21.15.32:0/2198776563 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56432a034a10 con 0x56432a148110 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56432a148110 0x56432a14a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c8b7fe640 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56432a148110 0x56432a14a530 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347a8e47e23cb1f1 server_cookie=17d6fe92e86b2999 in_seq=0 out_seq=0 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c6affd640 10 client.17619 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c6affd640 1 -- 172.21.15.32:0/2198776563 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56432a034a10 con 0x56432a148110 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c6affd640 10 client.17619 handle_command_reply: tid=0 2024-04-16T05:43:39.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 client.17619 shutdown 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 -- 172.21.15.32:0/2198776563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 msgr2=0x56432a0d6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d6d20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8c80005fd0 tx=0x7f8c800054a0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 -- 172.21.15.32:0/2198776563 shutdown_connections 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56432a0d1350 0x56432a0d72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56432a0c8770 0x56432a0d6d20 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56432a148110 0x56432a14a530 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 --2- 172.21.15.32:0/2198776563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432a0c7e50 0x56432a0d9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c6affd640 0 client.17619 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.624+0000 7f8c94a14740 1 -- 172.21.15.32:0/2198776563 >> 172.21.15.32:0/2198776563 conn(0x56432a0c2580 msgr2=0x56432a0c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:39.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c94a14740 1 -- 172.21.15.32:0/2198776563 shutdown_connections 2024-04-16T05:43:39.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c94a14740 1 -- 172.21.15.32:0/2198776563 wait complete. 2024-04-16T05:43:39.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c94a14740 20 client.17619 trim_cache size 0 max 16384 2024-04-16T05:43:39.632 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:43:39.632 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:43:39.632 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:43:39.632 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:43:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 -- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8c740a3d20 msgr2=0x7f8c740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8c740a3d20 0x7f8c740a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c06b8d0 tx=0x7f8c84004a40 comp rx=0 tx=0).stop 2024-04-16T05:43:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 -- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c074bf0 msgr2=0x7f8c8c1a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c074bf0 0x7f8c8c1a0ce0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c890 tx=0x7f8c8000cd50 comp rx=0 tx=0).stop 2024-04-16T05:43:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 -- 172.21.15.32:0/1807732283 shutdown_connections 2024-04-16T05:43:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c8c10e8e0 0x7f8c8c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8c740a3d20 0x7f8c740a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c8c074bf0 0x7f8c8c1a0ce0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 --2- 172.21.15.32:0/1807732283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c0742d0 0x7f8c8c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 -- 172.21.15.32:0/1807732283 >> 172.21.15.32:0/1807732283 conn(0x7f8c8c079450 msgr2=0x7f8c8c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 -- 172.21.15.32:0/1807732283 shutdown_connections 2024-04-16T05:43:39.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.625+0000 7f8c91e2a640 1 -- 172.21.15.32:0/1807732283 wait complete. 2024-04-16T05:43:39.635 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:43:39.636 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:43:39.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 -- 172.21.15.32:0/837247630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc101ba0 msgr2=0x7fbdbc10ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/837247630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc101ba0 0x7fbdbc10ddc0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009ca0 tx=0x7fbdb402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 -- 172.21.15.32:0/837247630 shutdown_connections 2024-04-16T05:43:39.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/837247630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc101ba0 0x7fbdbc10ddc0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/837247630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdbc1011e0 0x7fbdbc101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/837247630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdbc107040 0x7fbdbc107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 -- 172.21.15.32:0/837247630 >> 172.21.15.32:0/837247630 conn(0x7fbdbc077ee0 msgr2=0x7fbdbc0782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:39.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 -- 172.21.15.32:0/837247630 shutdown_connections 2024-04-16T05:43:39.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.743+0000 7fbdc27f2640 1 -- 172.21.15.32:0/837247630 wait complete. 2024-04-16T05:43:39.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc27f2640 1 Processor -- start 2024-04-16T05:43:39.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc27f2640 1 -- start start 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc27f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdbc1011e0 0x7fbdbc19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc27f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdbc101ba0 0x7fbdbc19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc27f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc107040 0x7fbdbc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc27f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc111280 con 0x7fbdbc107040 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc27f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc111100 con 0x7fbdbc1011e0 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc27f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc111400 con 0x7fbdbc101ba0 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc0d8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc107040 0x7fbdbc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc0d8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc107040 0x7fbdbc198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60288/0 (socket says 172.21.15.32:60288) 2024-04-16T05:43:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc0d8f640 1 -- 172.21.15.32:0/859656561 learned_addr learned my addr 172.21.15.32:0/859656561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdbb7fe640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdbc101ba0 0x7fbdbc19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdbbfff640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdbc1011e0 0x7fbdbc19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc0d8f640 1 -- 172.21.15.32:0/859656561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdbc101ba0 msgr2=0x7fbdbc19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc0d8f640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdbc101ba0 0x7fbdbc19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc0d8f640 1 -- 172.21.15.32:0/859656561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdbc1011e0 msgr2=0x7fbdbc19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc0d8f640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdbc1011e0 0x7fbdbc19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdc0d8f640 1 -- 172.21.15.32:0/859656561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdbc198d90 con 0x7fbdbc107040 2024-04-16T05:43:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.744+0000 7fbdbb7fe640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdbc101ba0 0x7fbdbc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.745+0000 7fbdc0d8f640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc107040 0x7fbdbc198610 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbdb402c700 tx=0x7fbdb4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.745+0000 7fbdb97fa640 1 -- 172.21.15.32:0/859656561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4042020 con 0x7fbdbc107040 2024-04-16T05:43:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.745+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdbc199020 con 0x7fbdbc107040 2024-04-16T05:43:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.745+0000 7fbdb97fa640 1 -- 172.21.15.32:0/859656561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb4034640 con 0x7fbdbc107040 2024-04-16T05:43:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.745+0000 7fbdb97fa640 1 -- 172.21.15.32:0/859656561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4039040 con 0x7fbdbc107040 2024-04-16T05:43:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.745+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdbc1a5320 con 0x7fbdbc107040 2024-04-16T05:43:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.746+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdbc069270 con 0x7fbdbc107040 2024-04-16T05:43:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.747+0000 7fbdb97fa640 1 -- 172.21.15.32:0/859656561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbdb4045020 con 0x7fbdbc107040 2024-04-16T05:43:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.748+0000 7fbdb97fa640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbda40a3d20 0x7fbda40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.748+0000 7fbdbbfff640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbda40a3d20 0x7fbda40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.748+0000 7fbdb97fa640 1 -- 172.21.15.32:0/859656561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7fbdb40f26f0 con 0x7fbdbc107040 2024-04-16T05:43:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.748+0000 7fbdbbfff640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbda40a3d20 0x7fbda40a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009aa0 tx=0x7fbdac005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.753+0000 7fbdb97fa640 1 -- 172.21.15.32:0/859656561 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbdb404c020 con 0x7fbdbc107040 2024-04-16T05:43:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.980+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdbc1a5590 con 0x7fbdbc107040 2024-04-16T05:43:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.981+0000 7fbdb97fa640 1 -- 172.21.15.32:0/859656561 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7fbdb40beb10 con 0x7fbdbc107040 2024-04-16T05:43:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:43:39.980 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:43:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:43:39.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbda40a3d20 msgr2=0x7fbda40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbda40a3d20 0x7fbda40a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009aa0 tx=0x7fbdac005f90 comp rx=0 tx=0).stop 2024-04-16T05:43:39.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc107040 msgr2=0x7fbdbc198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc107040 0x7fbdbc198610 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbdb402c700 tx=0x7fbdb4032f20 comp rx=0 tx=0).stop 2024-04-16T05:43:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 shutdown_connections 2024-04-16T05:43:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fbda40a3d20 0x7fbda40a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdbc107040 0x7fbdbc198610 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdbc101ba0 0x7fbdbc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 --2- 172.21.15.32:0/859656561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdbc1011e0 0x7fbdbc19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.983+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 >> 172.21.15.32:0/859656561 conn(0x7fbdbc077ee0 msgr2=0x7fbdbc06a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.984+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 shutdown_connections 2024-04-16T05:43:39.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:39.984+0000 7fbdc27f2640 1 -- 172.21.15.32:0/859656561 wait complete. 2024-04-16T05:43:39.993 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:43:40.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 -- 172.21.15.32:0/3004120937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 msgr2=0x7fe1e010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 --2- 172.21.15.32:0/3004120937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 0x7fe1e010ddd0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009ca0 tx=0x7fe1d802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 -- 172.21.15.32:0/3004120937 shutdown_connections 2024-04-16T05:43:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 --2- 172.21.15.32:0/3004120937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 0x7fe1e010ddd0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 --2- 172.21.15.32:0/3004120937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0101ae0 0x7fe1e0105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 --2- 172.21.15.32:0/3004120937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e0101130 0x7fe1e0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 -- 172.21.15.32:0/3004120937 >> 172.21.15.32:0/3004120937 conn(0x7fe1e00780b0 msgr2=0x7fe1e00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 -- 172.21.15.32:0/3004120937 shutdown_connections 2024-04-16T05:43:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 -- 172.21.15.32:0/3004120937 wait complete. 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 Processor -- start 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.103+0000 7fe1e7982640 1 -- start start 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e7982640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e0101130 0x7fe1e00febe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e7982640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 0x7fe1e00ff120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e7982640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e00ff660 0x7fe1e01a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e7982640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01112a0 con 0x7fe1e0106200 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e7982640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0111120 con 0x7fe1e0101130 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e7982640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0111420 con 0x7fe1e00ff660 2024-04-16T05:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 0x7fe1e00ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e571e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e0101130 0x7fe1e00febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 0x7fe1e00ff120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60302/0 (socket says 172.21.15.32:60302) 2024-04-16T05:43:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/436092884 learned_addr learned my addr 172.21.15.32:0/436092884 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e00ff660 0x7fe1e01a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/436092884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e00ff660 msgr2=0x7fe1e01a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e00ff660 0x7fe1e01a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/436092884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e0101130 msgr2=0x7fe1e00febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e0101130 0x7fe1e00febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/436092884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e01a7000 con 0x7fe1e0106200 2024-04-16T05:43:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 0x7fe1e00ff120 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000c970 tx=0x7fe1d000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1d67fc640 1 -- 172.21.15.32:0/436092884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0014070 con 0x7fe1e0106200 2024-04-16T05:43:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e7982640 1 -- 172.21.15.32:0/436092884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e01a72f0 con 0x7fe1e0106200 2024-04-16T05:43:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1d67fc640 1 -- 172.21.15.32:0/436092884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d0004440 con 0x7fe1e0106200 2024-04-16T05:43:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1d67fc640 1 -- 172.21.15.32:0/436092884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0003e10 con 0x7fe1e0106200 2024-04-16T05:43:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.104+0000 7fe1e7982640 1 -- 172.21.15.32:0/436092884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e01a7890 con 0x7fe1e0106200 2024-04-16T05:43:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.106+0000 7fe1d67fc640 1 -- 172.21.15.32:0/436092884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1d0017070 con 0x7fe1e0106200 2024-04-16T05:43:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1ea56e740 1 Processor -- start 2024-04-16T05:43:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1ea56e740 1 -- start start 2024-04-16T05:43:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d9139d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 0x557d913a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d913a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557d913020e0 con 0x557d9139ce60 2024-04-16T05:43:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557d9129f520 con 0x557d913a6360 2024-04-16T05:43:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d912fe0a0 con 0x557d9139d780 2024-04-16T05:43:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1d67fc640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe1cc0a3d50 0x7fe1cc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1e5f1f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d9139d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1e5f1f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d9139d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60308/0 (socket says 172.21.15.32:60308) 2024-04-16T05:43:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.107+0000 7fe1e5f1f640 1 -- 172.21.15.32:0/2696221776 learned_addr learned my addr 172.21.15.32:0/2696221776 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe1cc0a3d50 0x7fe1cc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1d67fc640 1 -- 172.21.15.32:0/436092884 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7fe1d00d0680 con 0x7fe1e0106200 2024-04-16T05:43:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e571e640 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 0x557d913a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d913a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e5f1f640 1 -- 172.21.15.32:0/2696221776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 msgr2=0x557d913a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 0x557d913a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e5f1f640 1 -- 172.21.15.32:0/2696221776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 msgr2=0x557d913a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d913a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e5f1f640 1 -- 172.21.15.32:0/2696221776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d912c78d0 con 0x557d9139ce60 2024-04-16T05:43:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d9139d240 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8005fa0 tx=0x7fe1d8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dfce4619454dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1cb7fe640 1 -- 172.21.15.32:0/2696221776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8039040 con 0x557d9139ce60 2024-04-16T05:43:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1cb7fe640 1 -- 172.21.15.32:0/2696221776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d8003b70 con 0x557d9139ce60 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.108+0000 7fe1cb7fe640 1 -- 172.21.15.32:0/2696221776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8034440 con 0x557d9139ce60 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 -- 172.21.15.32:0/2696221776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 msgr2=0x557d9139d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe1cc0a3d50 0x7fe1cc0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe1d804c000 tx=0x7fe1d8048000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d9139d240 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8005fa0 tx=0x7fe1d8032d10 comp rx=0 tx=0).stop 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 -- 172.21.15.32:0/2696221776 shutdown_connections 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d913a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 0x557d913a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/2696221776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d9139d240 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 -- 172.21.15.32:0/2696221776 >> 172.21.15.32:0/2696221776 conn(0x557d91397590 msgr2=0x557d91397970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 -- 172.21.15.32:0/2696221776 shutdown_connections 2024-04-16T05:43:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 -- 172.21.15.32:0/2696221776 wait complete. 2024-04-16T05:43:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 -- start start 2024-04-16T05:43:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.109+0000 7fe1ea56e740 1 Processor -- start 2024-04-16T05:43:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d913a3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 0x557d913a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d913a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557d913a9000 con 0x557d9139ce60 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557d912bb4d0 con 0x557d913a6360 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d913a9180 con 0x557d9139d780 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1ea56e740 10 client.0 _ll_register_callbacks cb 0x557d912a02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d913a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d913a3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60320/0 (socket says 172.21.15.32:60320) 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 -- 172.21.15.32:0/3117611164 learned_addr learned my addr 172.21.15.32:0/3117611164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 0x557d913a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d913a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 -- 172.21.15.32:0/3117611164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 msgr2=0x557d913a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 0x557d913a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 -- 172.21.15.32:0/3117611164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 msgr2=0x557d913a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d913a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 -- 172.21.15.32:0/3117611164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d910143f0 con 0x557d9139ce60 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1e571e640 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d913a3360 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0127d0 tx=0x7fe1dc012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3489f4b3ca5a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.110+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/3117611164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc01f060 con 0x557d9139ce60 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/3117611164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7fe1dc010dc0 con 0x557d9139ce60 2024-04-16T05:43:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3117611164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d8003b70 con 0x557d9139ce60 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/3117611164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe1dc026050 con 0x557d9139ce60 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 10 client.17637 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3117611164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d910de7d0 con 0x557d9139ce60 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/3117611164 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d8003b70 con 0x557d9139ce60 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/3117611164 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe1dc00d570 con 0x557d9139ce60 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 10 client.17637 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 10 client.17637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 20 client.17637 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x557d913b62c0 0x557d913b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 4 client.17637 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:40.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3117611164 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d9130fb00 con 0x557d913b62c0 2024-04-16T05:43:40.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.111+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x557d913b62c0 0x557d913b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.112+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x557d913b62c0 0x557d913b86f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba2d32b87207defe server_cookie=be8f5015eb6b3904 in_seq=0 out_seq=0 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.112+0000 7fe1c8ff9640 10 client.17637 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/3117611164 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d9130fb00 con 0x557d913b62c0 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1c8ff9640 10 client.17637 handle_command_reply: tid=0 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 client.17637 shutdown 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3117611164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 msgr2=0x557d913a3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d913a3360 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0127d0 tx=0x7fe1dc012c90 comp rx=0 tx=0).stop 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3117611164 shutdown_connections 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x557d913b62c0 0x557d913b86f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d913a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139d780 0x557d913a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1c8ff9640 0 client.17637 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:43:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3117611164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139ce60 0x557d913a3360 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3117611164 >> 172.21.15.32:0/3117611164 conn(0x557d91397590 msgr2=0x557d913992e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3117611164 shutdown_connections 2024-04-16T05:43:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3117611164 wait complete. 2024-04-16T05:43:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.113+0000 7fe1ea56e740 20 client.17637 trim_cache size 0 max 16384 2024-04-16T05:43:40.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.126+0000 7fe1ea56e740 1 Processor -- start 2024-04-16T05:43:40.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.126+0000 7fe1ea56e740 1 -- start start 2024-04-16T05:43:40.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9139ce60 0x557d913a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d913a6360 0x557d913965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01b3eb0 con 0x557d9139d780 2024-04-16T05:43:40.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01b1780 con 0x557d9139ce60 2024-04-16T05:43:40.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01af5f0 con 0x557d913a6360 2024-04-16T05:43:40.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60336/0 (socket says 172.21.15.32:60336) 2024-04-16T05:43:40.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/3311080335 learned_addr learned my addr 172.21.15.32:0/3311080335 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d913a6360 0x557d913965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/3311080335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d913a6360 msgr2=0x557d913965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e571e640 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9139ce60 0x557d913a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d913a6360 0x557d913965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/3311080335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9139ce60 msgr2=0x557d913a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9139ce60 0x557d913a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/3311080335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d9139d5b0 con 0x557d9139d780 2024-04-16T05:43:40.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d913a6360 0x557d913965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913b2030 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00cdaa0 tx=0x7fe1d009f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b303b294a4151e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1caffd640 1 -- 172.21.15.32:0/3311080335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a4f00 con 0x557d9139d780 2024-04-16T05:43:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1caffd640 1 -- 172.21.15.32:0/3311080335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d009a330 con 0x557d9139d780 2024-04-16T05:43:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.127+0000 7fe1caffd640 1 -- 172.21.15.32:0/3311080335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a4a70 con 0x557d9139d780 2024-04-16T05:43:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3311080335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 msgr2=0x557d913b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913b2030 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00cdaa0 tx=0x7fe1d009f040 comp rx=0 tx=0).stop 2024-04-16T05:43:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3311080335 shutdown_connections 2024-04-16T05:43:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d913a6360 0x557d913965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913b2030 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/3311080335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9139ce60 0x557d913a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3311080335 >> 172.21.15.32:0/3311080335 conn(0x557d91397590 msgr2=0x557d91395ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3311080335 shutdown_connections 2024-04-16T05:43:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 -- 172.21.15.32:0/3311080335 wait complete. 2024-04-16T05:43:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 -- start start 2024-04-16T05:43:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.128+0000 7fe1ea56e740 1 Processor -- start 2024-04-16T05:43:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139ce60 0x557d913aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1ea56e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d9141b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01afc40 con 0x557d9139d780 2024-04-16T05:43:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01b10b0 con 0x557d913a6360 2024-04-16T05:43:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1ea56e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01b0310 con 0x557d9139ce60 2024-04-16T05:43:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1ea56e740 10 client.0 _ll_register_callbacks cb 0x7fe1dc00a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60338/0 (socket says 172.21.15.32:60338) 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/684815929 learned_addr learned my addr 172.21.15.32:0/684815929 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e5f1f640 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d9141b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e571e640 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139ce60 0x557d913aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/684815929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139ce60 msgr2=0x557d913aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139ce60 0x557d913aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/684815929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 msgr2=0x557d9141b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d9141b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 -- 172.21.15.32:0/684815929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d9141bd70 con 0x557d9139d780 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1e4f1d640 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913aaae0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe1d009c220 tx=0x7fe1d0098570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e80846d1507ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/684815929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a4f00 con 0x557d9139d780 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/684815929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7fe1d00cdcc0 con 0x557d9139d780 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.129+0000 7fe1ea56e740 1 -- 172.21.15.32:0/684815929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d009a330 con 0x557d9139d780 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/684815929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe1d00a4a70 con 0x557d9139d780 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1ea56e740 10 client.17643 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1ea56e740 1 -- 172.21.15.32:0/684815929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d9141cee0 con 0x557d9139d780 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/684815929 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d009a330 con 0x557d9139d780 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/684815929 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe1d009b6e0 con 0x557d9139d780 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1ea56e740 10 client.17643 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1ea56e740 10 client.17643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1ea56e740 20 client.17643 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x557d9141d520 0x557d9141f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1ea56e740 4 client.17643 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1ea56e740 1 -- 172.21.15.32:0/684815929 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d91309970 con 0x557d9141d520 2024-04-16T05:43:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.130+0000 7fe1e571e640 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x557d9141d520 0x557d9141f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1c8ff9640 10 client.17643 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1e571e640 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x557d9141d520 0x557d9141f940 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4dae93c54a161ddf server_cookie=23df4c55a29c794f in_seq=0 out_seq=0 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1c8ff9640 1 -- 172.21.15.32:0/684815929 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d91309970 con 0x557d9141d520 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1c8ff9640 10 client.17643 handle_command_reply: tid=0 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 client.17643 shutdown 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 -- 172.21.15.32:0/684815929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 msgr2=0x557d913aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913aaae0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe1d009c220 tx=0x7fe1d0098570 comp rx=0 tx=0).stop 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 -- 172.21.15.32:0/684815929 shutdown_connections 2024-04-16T05:43:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x557d9141d520 0x557d9141f940 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d913a6360 0x557d9141b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1c8ff9640 0 client.17643 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557d9139d780 0x557d913aaae0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 --2- 172.21.15.32:0/684815929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9139ce60 0x557d913aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 -- 172.21.15.32:0/684815929 >> 172.21.15.32:0/684815929 conn(0x557d91397590 msgr2=0x557d91396730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 -- 172.21.15.32:0/684815929 shutdown_connections 2024-04-16T05:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 1 -- 172.21.15.32:0/684815929 wait complete. 2024-04-16T05:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.131+0000 7fe1ea56e740 20 client.17643 trim_cache size 0 max 16384 2024-04-16T05:43:40.144 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:43:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 -- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe1cc0a3d50 msgr2=0x7fe1cc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe1cc0a3d50 0x7fe1cc0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe1d804c000 tx=0x7fe1d8048000 comp rx=0 tx=0).stop 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 -- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 msgr2=0x7fe1e00ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 0x7fe1e00ff120 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000c970 tx=0x7fe1d000cd70 comp rx=0 tx=0).stop 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 -- 172.21.15.32:0/436092884 shutdown_connections 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e00ff660 0x7fe1e01a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fe1cc0a3d50 0x7fe1cc0a6210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0106200 0x7fe1e00ff120 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 --2- 172.21.15.32:0/436092884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e0101130 0x7fe1e00febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 -- 172.21.15.32:0/436092884 >> 172.21.15.32:0/436092884 conn(0x7fe1e00780b0 msgr2=0x7fe1e0069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 -- 172.21.15.32:0/436092884 shutdown_connections 2024-04-16T05:43:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.132+0000 7fe1e7982640 1 -- 172.21.15.32:0/436092884 wait complete. 2024-04-16T05:43:40.146 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:39 smithi032 ceph-mon[32351]: pgmap v268: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:40.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:43:40.147 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:39 smithi134 ceph-mon[36153]: pgmap v268: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:40.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:39 smithi082 ceph-mon[36195]: pgmap v268: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:43:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.256+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3589831910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0101050 msgr2=0x7fa3d0101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.256+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3589831910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0101050 0x7fa3d0101430 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009930 tx=0x7fa3cc02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.256+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3589831910 shutdown_connections 2024-04-16T05:43:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.256+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3589831910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d0106450 0x7fa3d0106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.256+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3589831910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0101970 0x7fa3d0105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.256+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3589831910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0101050 0x7fa3d0101430 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.256+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3589831910 >> 172.21.15.32:0/3589831910 conn(0x7fa3d0077280 msgr2=0x7fa3d0077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.256+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3589831910 shutdown_connections 2024-04-16T05:43:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3589831910 wait complete. 2024-04-16T05:43:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 Processor -- start 2024-04-16T05:43:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 -- start start 2024-04-16T05:43:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d0101050 0x7fa3d019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0101970 0x7fa3d019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0106450 0x7fa3d01a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01189e0 con 0x7fa3d0106450 2024-04-16T05:43:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0118860 con 0x7fa3d0101970 2024-04-16T05:43:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d7fe9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0118b60 con 0x7fa3d0101050 2024-04-16T05:43:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d6586640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0106450 0x7fa3d01a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d6586640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0106450 0x7fa3d01a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60366/0 (socket says 172.21.15.32:60366) 2024-04-16T05:43:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.257+0000 7fa3d6586640 1 -- 172.21.15.32:0/3682388042 learned_addr learned my addr 172.21.15.32:0/3682388042 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d0101050 0x7fa3d019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d6586640 1 -- 172.21.15.32:0/3682388042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d0101050 msgr2=0x7fa3d019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d6586640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d0101050 0x7fa3d019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d6586640 1 -- 172.21.15.32:0/3682388042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0101970 msgr2=0x7fa3d019f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0101970 0x7fa3d019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d6586640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0101970 0x7fa3d019f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d6586640 1 -- 172.21.15.32:0/3682388042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d01a3ec0 con 0x7fa3d0106450 2024-04-16T05:43:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0101970 0x7fa3d019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d6586640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0106450 0x7fa3d01a3790 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800eaa0 tx=0x7fa3c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3c6ffd640 1 -- 172.21.15.32:0/3682388042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800cdb0 con 0x7fa3d0106450 2024-04-16T05:43:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3c6ffd640 1 -- 172.21.15.32:0/3682388042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c8005020 con 0x7fa3d0106450 2024-04-16T05:43:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3682388042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d01a41b0 con 0x7fa3d0106450 2024-04-16T05:43:40.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3c6ffd640 1 -- 172.21.15.32:0/3682388042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80116b0 con 0x7fa3d0106450 2024-04-16T05:43:40.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.258+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3682388042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d01aba40 con 0x7fa3d0106450 2024-04-16T05:43:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.260+0000 7fa3c6ffd640 1 -- 172.21.15.32:0/3682388042 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa3c8005560 con 0x7fa3d0106450 2024-04-16T05:43:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.260+0000 7fa3dabd3740 1 Processor -- start 2024-04-16T05:43:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3dabd3740 1 -- start start 2024-04-16T05:43:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f26240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f2ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f31750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557168e8b0e0 con 0x557168f2f360 2024-04-16T05:43:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557168e28520 con 0x557168f25e60 2024-04-16T05:43:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557168e870a0 con 0x557168f26780 2024-04-16T05:43:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f31750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f31750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60368/0 (socket says 172.21.15.32:60368) 2024-04-16T05:43:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 -- 172.21.15.32:0/924138029 learned_addr learned my addr 172.21.15.32:0/924138029 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f26240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f2ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3c6ffd640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa3bc0a3d20 0x7fa3bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 -- 172.21.15.32:0/924138029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 msgr2=0x557168f2ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f2ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d6586640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa3bc0a3d20 0x7fa3bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 -- 172.21.15.32:0/924138029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 msgr2=0x557168f26240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f26240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.261+0000 7fa3d5584640 1 -- 172.21.15.32:0/924138029 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557168e508d0 con 0x557168f2f360 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.262+0000 7fa3c6ffd640 1 -- 172.21.15.32:0/3682388042 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7fa3c80d0e40 con 0x7fa3d0106450 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.262+0000 7fa3d5584640 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f31750 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0012820 tx=0x7fa3c000d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533685a3f426e95c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.262+0000 7fa3bbfff640 1 -- 172.21.15.32:0/924138029 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0012e60 con 0x557168f2f360 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.262+0000 7fa3d6586640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa3bc0a3d20 0x7fa3bc0a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80ce800 tx=0x7fa3c8087040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.262+0000 7fa3bbfff640 1 -- 172.21.15.32:0/924138029 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c001d430 con 0x557168f2f360 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.262+0000 7fa3bbfff640 1 -- 172.21.15.32:0/924138029 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0010630 con 0x557168f2f360 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.262+0000 7fa3dabd3740 1 -- 172.21.15.32:0/924138029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 msgr2=0x557168f31750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.262+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f31750 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0012820 tx=0x7fa3c000d3c0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 -- 172.21.15.32:0/924138029 shutdown_connections 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f31750 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f2ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/924138029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f26240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 -- 172.21.15.32:0/924138029 >> 172.21.15.32:0/924138029 conn(0x557168f20590 msgr2=0x557168f20970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 -- 172.21.15.32:0/924138029 shutdown_connections 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 -- 172.21.15.32:0/924138029 wait complete. 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 -- start start 2024-04-16T05:43:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 Processor -- start 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f2c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f2ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f2cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557168f31fa0 con 0x557168f2f360 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557168e444d0 con 0x557168f25e60 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557168f32120 con 0x557168f26780 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.263+0000 7fa3dabd3740 10 client.0 _ll_register_callbacks cb 0x557168e292c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f2cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f2cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60376/0 (socket says 172.21.15.32:60376) 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 -- 172.21.15.32:0/1763402190 learned_addr learned my addr 172.21.15.32:0/1763402190 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f2ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f2c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 -- 172.21.15.32:0/1763402190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 msgr2=0x557168f2ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f2ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 -- 172.21.15.32:0/1763402190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 msgr2=0x557168f2c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f2c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 -- 172.21.15.32:0/1763402190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557168b9d3f0 con 0x557168f2f360 2024-04-16T05:43:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3d5584640 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f2cfa0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0005fd0 tx=0x7fa3c0005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8427b33e8567d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3b97fa640 1 -- 172.21.15.32:0/1763402190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0012e60 con 0x557168f2f360 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3b97fa640 1 -- 172.21.15.32:0/1763402190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7fa3c001d7f0 con 0x557168f2f360 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3dabd3740 1 -- 172.21.15.32:0/1763402190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c001d430 con 0x557168f2f360 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3b97fa640 1 -- 172.21.15.32:0/1763402190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3c0012020 con 0x557168f2f360 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3dabd3740 10 client.17655 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.264+0000 7fa3dabd3740 1 -- 172.21.15.32:0/1763402190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557168c677d0 con 0x557168f2f360 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3b97fa640 1 -- 172.21.15.32:0/1763402190 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c001d430 con 0x557168f2f360 2024-04-16T05:43:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3b97fa640 1 -- 172.21.15.32:0/1763402190 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa3c0011470 con 0x557168f2f360 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3dabd3740 10 client.17655 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3dabd3740 10 client.17655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3dabd3740 20 client.17655 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x557168f3f2d0 0x557168f416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3dabd3740 4 client.17655 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3dabd3740 1 -- 172.21.15.32:0/1763402190 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557168e98b00 con 0x557168f3f2d0 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.265+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x557168f3f2d0 0x557168f416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.266+0000 7fa3b97fa640 10 client.17655 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:43:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.266+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x557168f3f2d0 0x557168f416c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3348fb77abc78ef2 server_cookie=9b609c0dd9cab0f8 in_seq=0 out_seq=0 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.266+0000 7fa3b97fa640 1 -- 172.21.15.32:0/1763402190 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557168e98b00 con 0x557168f3f2d0 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.266+0000 7fa3b97fa640 10 client.17655 handle_command_reply: tid=0 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.266+0000 7fa3dabd3740 1 client.17655 shutdown 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.266+0000 7fa3dabd3740 1 -- 172.21.15.32:0/1763402190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 msgr2=0x557168f2cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f2cfa0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0005fd0 tx=0x7fa3c0005b70 comp rx=0 tx=0).stop 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 -- 172.21.15.32:0/1763402190 shutdown_connections 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x557168f3f2d0 0x557168f416c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f2cfa0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f2ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/1763402190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f2c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3b97fa640 0 client.17655 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:43:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 -- 172.21.15.32:0/1763402190 >> 172.21.15.32:0/1763402190 conn(0x557168f20590 msgr2=0x557168f222e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 -- 172.21.15.32:0/1763402190 shutdown_connections 2024-04-16T05:43:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 1 -- 172.21.15.32:0/1763402190 wait complete. 2024-04-16T05:43:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.267+0000 7fa3dabd3740 20 client.17655 trim_cache size 0 max 16384 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3dabd3740 1 Processor -- start 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3dabd3740 1 -- start start 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f46d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f1eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01b83e0 con 0x557168f2f360 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01b5cb0 con 0x557168f25e60 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01b3b20 con 0x557168f26780 2024-04-16T05:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3d5584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f1eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3d5584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f1eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60390/0 (socket says 172.21.15.32:60390) 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.280+0000 7fa3d5584640 1 -- 172.21.15.32:0/3436612430 learned_addr learned my addr 172.21.15.32:0/3436612430 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f46d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3d5584640 1 -- 172.21.15.32:0/3436612430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 msgr2=0x557168f472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3d5584640 1 -- 172.21.15.32:0/3436612430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 msgr2=0x557168f46d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f46d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3d5584640 1 -- 172.21.15.32:0/3436612430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557168f321f0 con 0x557168f2f360 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f1eea0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0000c00 tx=0x7fa3c0011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0e290b5b1c3209 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3bb7fe640 1 -- 172.21.15.32:0/3436612430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c001ee60 con 0x557168f2f360 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3bb7fe640 1 -- 172.21.15.32:0/3436612430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c001d430 con 0x557168f2f360 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3bb7fe640 1 -- 172.21.15.32:0/3436612430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0012270 con 0x557168f2f360 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3436612430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 msgr2=0x557168f1eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.281+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f1eea0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0000c00 tx=0x7fa3c0011f70 comp rx=0 tx=0).stop 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3436612430 shutdown_connections 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f2f360 0x557168f1eea0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f26780 0x557168f472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3436612430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f46d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3436612430 >> 172.21.15.32:0/3436612430 conn(0x557168f20590 msgr2=0x557168f2adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3436612430 shutdown_connections 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3436612430 wait complete. 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 -- start start 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 Processor -- start 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f37920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f26780 0x557168f29af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f2f360 0x557168f2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01b4170 con 0x557168f26780 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01b55e0 con 0x557168f25e60 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01b4840 con 0x557168f2f360 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3dabd3740 10 client.0 _ll_register_callbacks cb 0x7fa3c000dde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3d5d85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f26780 0x557168f29af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3d5d85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f26780 0x557168f29af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60400/0 (socket says 172.21.15.32:60400) 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.282+0000 7fa3d5d85640 1 -- 172.21.15.32:0/3021906328 learned_addr learned my addr 172.21.15.32:0/3021906328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f2f360 0x557168f2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f37920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3d5d85640 1 -- 172.21.15.32:0/3021906328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f2f360 msgr2=0x557168f2a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f2f360 0x557168f2a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3d5d85640 1 -- 172.21.15.32:0/3021906328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 msgr2=0x557168f37920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f37920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3d5d85640 1 -- 172.21.15.32:0/3021906328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557168f27310 con 0x557168f26780 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3d5d85640 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f26780 0x557168f29af0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc02e010 tx=0x7fa3cc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d83163f1c4959f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3021906328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc046b50 con 0x557168f26780 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3021906328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7fa3cc046cf0 con 0x557168f26780 2024-04-16T05:43:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3021906328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c001d430 con 0x557168f26780 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3021906328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc054050 con 0x557168f26780 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3dabd3740 10 client.17661 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3021906328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557168fa48e0 con 0x557168f26780 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.283+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3021906328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c001d430 con 0x557168f26780 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3021906328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa3cc005020 con 0x557168f26780 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3dabd3740 10 client.17661 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3dabd3740 10 client.17661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3dabd3740 20 client.17661 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:40.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x557168fa5060 0x557168fa7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3dabd3740 4 client.17661 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:40.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3021906328 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557168e92970 con 0x557168fa5060 2024-04-16T05:43:40.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x557168fa5060 0x557168fa7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3d5584640 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x557168fa5060 0x557168fa7480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ca9c92d6b0837c5 server_cookie=33a9f7d268f312a4 in_seq=0 out_seq=0 2024-04-16T05:43:40.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3b97fa640 10 client.17661 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:43:40.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3021906328 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557168e92970 con 0x557168fa5060 2024-04-16T05:43:40.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3b97fa640 10 client.17661 handle_command_reply: tid=0 2024-04-16T05:43:40.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.284+0000 7fa3dabd3740 1 client.17661 shutdown 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3021906328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f26780 msgr2=0x557168f29af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f26780 0x557168f29af0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc02e010 tx=0x7fa3cc0058d0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3021906328 shutdown_connections 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x557168fa5060 0x557168fa7480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557168f2f360 0x557168f2a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557168f26780 0x557168f29af0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 --2- 172.21.15.32:0/3021906328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557168f25e60 0x557168f37920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3b97fa640 0 client.17661 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3021906328 >> 172.21.15.32:0/3021906328 conn(0x557168f20590 msgr2=0x557168f2b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3021906328 shutdown_connections 2024-04-16T05:43:40.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 1 -- 172.21.15.32:0/3021906328 wait complete. 2024-04-16T05:43:40.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3dabd3740 20 client.17661 trim_cache size 0 max 16384 2024-04-16T05:43:40.296 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:43:40.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa3bc0a3d20 msgr2=0x7fa3bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa3bc0a3d20 0x7fa3bc0a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80ce800 tx=0x7fa3c8087040 comp rx=0 tx=0).stop 2024-04-16T05:43:40.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0106450 msgr2=0x7fa3d01a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0106450 0x7fa3d01a3790 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800eaa0 tx=0x7fa3c800ef60 comp rx=0 tx=0).stop 2024-04-16T05:43:40.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3682388042 shutdown_connections 2024-04-16T05:43:40.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa3bc0a3d20 0x7fa3bc0a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d0106450 0x7fa3d01a3790 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0101970 0x7fa3d019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 --2- 172.21.15.32:0/3682388042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d0101050 0x7fa3d019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3682388042 >> 172.21.15.32:0/3682388042 conn(0x7fa3d0077280 msgr2=0x7fa3d0103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3682388042 shutdown_connections 2024-04-16T05:43:40.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.285+0000 7fa3d7fe9640 1 -- 172.21.15.32:0/3682388042 wait complete. 2024-04-16T05:43:40.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:43:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- 172.21.15.32:0/150177106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d80101ae0 msgr2=0x7f5d80105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 --2- 172.21.15.32:0/150177106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d80101ae0 0x7f5d80105b20 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009900 tx=0x7f5d7402bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- 172.21.15.32:0/150177106 shutdown_connections 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 --2- 172.21.15.32:0/150177106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d80106200 0x7f5d8010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 --2- 172.21.15.32:0/150177106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d80101ae0 0x7f5d80105b20 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 --2- 172.21.15.32:0/150177106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d80101130 0x7f5d80101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- 172.21.15.32:0/150177106 >> 172.21.15.32:0/150177106 conn(0x7f5d800780b0 msgr2=0x7f5d800784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- 172.21.15.32:0/150177106 shutdown_connections 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- 172.21.15.32:0/150177106 wait complete. 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 Processor -- start 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- start start 2024-04-16T05:43:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d80101130 0x7f5d801996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d80106200 0x7f5d80199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8019a1f0 0x7f5d801a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80111290 con 0x7f5d8019a1f0 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80111110 con 0x7f5d80106200 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.408+0000 7f5d87f50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80111410 con 0x7f5d80101130 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8019a1f0 0x7f5d801a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8019a1f0 0x7f5d801a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60422/0 (socket says 172.21.15.32:60422) 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 -- 172.21.15.32:0/758936337 learned_addr learned my addr 172.21.15.32:0/758936337 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d854eb640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d80106200 0x7f5d80199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d85cec640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d80101130 0x7f5d801996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 -- 172.21.15.32:0/758936337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d80101130 msgr2=0x7f5d801996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d80101130 0x7f5d801996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 -- 172.21.15.32:0/758936337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d80106200 msgr2=0x7f5d80199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d80106200 0x7f5d80199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 -- 172.21.15.32:0/758936337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d801a4f70 con 0x7f5d8019a1f0 2024-04-16T05:43:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d864ed640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8019a1f0 0x7f5d801a4750 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000eaa0 tx=0x7f5d7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d7effd640 1 -- 172.21.15.32:0/758936337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000cdb0 con 0x7f5d8019a1f0 2024-04-16T05:43:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d7effd640 1 -- 172.21.15.32:0/758936337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d70005020 con 0x7f5d8019a1f0 2024-04-16T05:43:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d87f50640 1 -- 172.21.15.32:0/758936337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d801a5260 con 0x7f5d8019a1f0 2024-04-16T05:43:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d7effd640 1 -- 172.21.15.32:0/758936337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700116b0 con 0x7f5d8019a1f0 2024-04-16T05:43:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.409+0000 7f5d87f50640 1 -- 172.21.15.32:0/758936337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801a5730 con 0x7f5d8019a1f0 2024-04-16T05:43:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.412+0000 7f5d7effd640 1 -- 172.21.15.32:0/758936337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d70005560 con 0x7f5d8019a1f0 2024-04-16T05:43:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.412+0000 7f5d8ac11740 1 Processor -- start 2024-04-16T05:43:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.412+0000 7f5d8ac11740 1 -- start start 2024-04-16T05:43:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56be60 0x5619aa56c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56c780 0x5619aa574e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 0x5619aa577750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5619aa4d10e0 con 0x5619aa56be60 2024-04-16T05:43:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5619aa46e520 con 0x5619aa575360 2024-04-16T05:43:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5619aa4cd0a0 con 0x5619aa56c780 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56be60 0x5619aa56c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56be60 0x5619aa56c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60428/0 (socket says 172.21.15.32:60428) 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 -- 172.21.15.32:0/2490512195 learned_addr learned my addr 172.21.15.32:0/2490512195 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d85cec640 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56c780 0x5619aa574e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d7effd640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5d680a3d20 0x7f5d680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 0x5619aa577750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d864ed640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5d680a3d20 0x7f5d680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 -- 172.21.15.32:0/2490512195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56c780 msgr2=0x5619aa574e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56c780 0x5619aa574e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 -- 172.21.15.32:0/2490512195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 msgr2=0x5619aa577750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d7effd640 1 -- 172.21.15.32:0/758936337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f5d700d0e80 con 0x7f5d8019a1f0 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 0x5619aa577750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 -- 172.21.15.32:0/2490512195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619aa4968d0 con 0x5619aa56be60 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.413+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56be60 0x5619aa56c240 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005ab0 tx=0x7f5d740058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1da032fd8a8ed70 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d864ed640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5d680a3d20 0x7f5d680a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d7009c9c0 tx=0x7f5d70091ec0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d67fff640 1 -- 172.21.15.32:0/2490512195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7402ccf0 con 0x5619aa56be60 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d67fff640 1 -- 172.21.15.32:0/2490512195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d74005040 con 0x5619aa56be60 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d67fff640 1 -- 172.21.15.32:0/2490512195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d740049e0 con 0x5619aa56be60 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2490512195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56be60 msgr2=0x5619aa56c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56be60 0x5619aa56c240 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005ab0 tx=0x7f5d740058b0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2490512195 shutdown_connections 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 0x5619aa577750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56c780 0x5619aa574e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.414+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2490512195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56be60 0x5619aa56c240 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2490512195 >> 172.21.15.32:0/2490512195 conn(0x5619aa566590 msgr2=0x5619aa566970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2490512195 shutdown_connections 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2490512195 wait complete. 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 -- start start 2024-04-16T05:43:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 Processor -- start 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56be60 0x5619aa572460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa5729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa575360 0x5619aa572ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5619aa577fa0 con 0x5619aa56c780 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5619aa48a4d0 con 0x5619aa56be60 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.415+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5619aa578120 con 0x5619aa575360 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d8ac11740 10 client.0 _ll_register_callbacks cb 0x5619aa46f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa5729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa5729a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60442/0 (socket says 172.21.15.32:60442) 2024-04-16T05:43:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 -- 172.21.15.32:0/2099339648 learned_addr learned my addr 172.21.15.32:0/2099339648 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa575360 0x5619aa572ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 -- 172.21.15.32:0/2099339648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa575360 msgr2=0x5619aa572ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa575360 0x5619aa572ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 -- 172.21.15.32:0/2099339648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56be60 msgr2=0x5619aa572460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56be60 0x5619aa572460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 -- 172.21.15.32:0/2099339648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619aa1e33f0 con 0x5619aa56c780 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa575360 0x5619aa572ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d85cec640 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa5729a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009e00 tx=0x7f5d780117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c5b0340251eb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d657fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d657fa640 1 -- 172.21.15.32:0/2099339648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78012860 con 0x5619aa56c780 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d657fa640 1 -- 172.21.15.32:0/2099339648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f5d78012a00 con 0x5619aa56c780 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2099339648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d74005040 con 0x5619aa56c780 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d657fa640 1 -- 172.21.15.32:0/2099339648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d78025050 con 0x5619aa56c780 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d8ac11740 10 client.17673 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2099339648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619aa2ad7d0 con 0x5619aa56c780 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.416+0000 7f5d657fa640 1 -- 172.21.15.32:0/2099339648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d74005040 con 0x5619aa56c780 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d657fa640 1 -- 172.21.15.32:0/2099339648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d7800f040 con 0x5619aa56c780 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d8ac11740 10 client.17673 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:40.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d8ac11740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d8ac11740 20 client.17673 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5619aa585390 0x5619aa587830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d8ac11740 4 client.17673 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2099339648 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619aa4deb00 con 0x5619aa585390 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5619aa585390 0x5619aa587830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d657fa640 10 client.17673 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.417+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5619aa585390 0x5619aa587830 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5803b7075b17f1 server_cookie=c5bcd9814be73d9b in_seq=0 out_seq=0 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d657fa640 1 -- 172.21.15.32:0/2099339648 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619aa4deb00 con 0x5619aa585390 2024-04-16T05:43:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d657fa640 10 client.17673 handle_command_reply: tid=0 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 client.17673 shutdown 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2099339648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 msgr2=0x5619aa5729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa5729a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009e00 tx=0x7f5d780117b0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2099339648 shutdown_connections 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5619aa585390 0x5619aa587830 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa575360 0x5619aa572ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa5729a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2099339648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56be60 0x5619aa572460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d657fa640 0 client.17673 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:40.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.418+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2099339648 >> 172.21.15.32:0/2099339648 conn(0x5619aa566590 msgr2=0x5619aa5682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.419+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2099339648 shutdown_connections 2024-04-16T05:43:40.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.419+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2099339648 wait complete. 2024-04-16T05:43:40.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.419+0000 7f5d8ac11740 20 client.17673 trim_cache size 0 max 16384 2024-04-16T05:43:40.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d8ac11740 1 Processor -- start 2024-04-16T05:43:40.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d8ac11740 1 -- start start 2024-04-16T05:43:40.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 0x5619aa574b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa58d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 0x5619aa58d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801b1bc0 con 0x5619aa56c780 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801af490 con 0x5619aa575360 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801ad300 con 0x5619aa56be60 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d85cec640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa58d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d85cec640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa58d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60448/0 (socket says 172.21.15.32:60448) 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d85cec640 1 -- 172.21.15.32:0/702492133 learned_addr learned my addr 172.21.15.32:0/702492133 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d854eb640 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 0x5619aa58d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d854eb640 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 0x5619aa574b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d85cec640 1 -- 172.21.15.32:0/702492133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 msgr2=0x5619aa574b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d85cec640 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 0x5619aa574b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d85cec640 1 -- 172.21.15.32:0/702492133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 msgr2=0x5619aa58d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d85cec640 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 0x5619aa58d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.432+0000 7f5d85cec640 1 -- 172.21.15.32:0/702492133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619aa56c5b0 con 0x5619aa56c780 2024-04-16T05:43:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d85cec640 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa58d140 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009380 tx=0x7f5d780099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8580ed9b10dd544a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d677fe640 1 -- 172.21.15.32:0/702492133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78005220 con 0x5619aa56c780 2024-04-16T05:43:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d677fe640 1 -- 172.21.15.32:0/702492133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7802d070 con 0x5619aa56c780 2024-04-16T05:43:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d677fe640 1 -- 172.21.15.32:0/702492133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7801ab70 con 0x5619aa56c780 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 -- 172.21.15.32:0/702492133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 msgr2=0x5619aa58d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa58d140 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009380 tx=0x7f5d780099f0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 -- 172.21.15.32:0/702492133 shutdown_connections 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa575360 0x5619aa58d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa56c780 0x5619aa58d140 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/702492133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 0x5619aa574b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 -- 172.21.15.32:0/702492133 >> 172.21.15.32:0/702492133 conn(0x5619aa566590 msgr2=0x5619aa565730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 -- 172.21.15.32:0/702492133 shutdown_connections 2024-04-16T05:43:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.433+0000 7f5d8ac11740 1 -- 172.21.15.32:0/702492133 wait complete. 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 1 -- start start 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 1 Processor -- start 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 0x5619aa56f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56c780 0x5619aa56f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa575360 0x5619aa56fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801ad950 con 0x5619aa575360 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801aedc0 con 0x5619aa56c780 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801ae020 con 0x5619aa56be60 2024-04-16T05:43:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d8ac11740 10 client.0 _ll_register_callbacks cb 0x7f5d78009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d854eb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa575360 0x5619aa56fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d854eb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa575360 0x5619aa56fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60452/0 (socket says 172.21.15.32:60452) 2024-04-16T05:43:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d854eb640 1 -- 172.21.15.32:0/2064395794 learned_addr learned my addr 172.21.15.32:0/2064395794 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d854eb640 1 -- 172.21.15.32:0/2064395794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 msgr2=0x5619aa56f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 0x5619aa56f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d854eb640 1 -- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56c780 msgr2=0x5619aa56f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d85cec640 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56c780 0x5619aa56f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56c780 0x5619aa56f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d854eb640 1 -- 172.21.15.32:0/2064395794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619aa5eac50 con 0x5619aa575360 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.434+0000 7f5d85cec640 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56c780 0x5619aa56f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d854eb640 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa575360 0x5619aa56fe20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5d74004650 tx=0x7f5d74005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fe844aa5460bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d657fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d657fa640 1 -- 172.21.15.32:0/2064395794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74005450 con 0x5619aa575360 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d657fa640 1 -- 172.21.15.32:0/2064395794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f5d740055f0 con 0x5619aa575360 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2064395794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7802d070 con 0x5619aa575360 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d657fa640 1 -- 172.21.15.32:0/2064395794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d7402ca70 con 0x5619aa575360 2024-04-16T05:43:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 10 client.17679 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2064395794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619aa5ebf90 con 0x5619aa575360 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d657fa640 1 -- 172.21.15.32:0/2064395794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7802d070 con 0x5619aa575360 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d657fa640 1 -- 172.21.15.32:0/2064395794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d740406c0 con 0x5619aa575360 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 10 client.17679 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 20 client.17679 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5619aa5ec2d0 0x5619aa5ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 4 client.17679 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:43:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.435+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2064395794 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619aa4d8970 con 0x5619aa5ec2d0 2024-04-16T05:43:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.436+0000 7f5d85cec640 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5619aa5ec2d0 0x5619aa5ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.436+0000 7f5d657fa640 10 client.17679 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.436+0000 7f5d85cec640 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5619aa5ec2d0 0x5619aa5ee6c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1dae2bc6b59ca8 server_cookie=136dccbecd8341cb in_seq=0 out_seq=0 2024-04-16T05:43:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.436+0000 7f5d657fa640 1 -- 172.21.15.32:0/2064395794 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619aa4d8970 con 0x5619aa5ec2d0 2024-04-16T05:43:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.436+0000 7f5d657fa640 10 client.17679 handle_command_reply: tid=0 2024-04-16T05:43:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.436+0000 7f5d8ac11740 1 client.17679 shutdown 2024-04-16T05:43:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.436+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2064395794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa575360 msgr2=0x5619aa56fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.436+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa575360 0x5619aa56fe20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5d74004650 tx=0x7f5d74005bd0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2064395794 shutdown_connections 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619aa575360 0x5619aa56fe20 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619aa56c780 0x5619aa56f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5619aa5ec2d0 0x5619aa5ee6c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 1 --2- 172.21.15.32:0/2064395794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619aa56be60 0x5619aa56f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d657fa640 0 client.17679 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2064395794 >> 172.21.15.32:0/2064395794 conn(0x5619aa566590 msgr2=0x5619aa564ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2064395794 shutdown_connections 2024-04-16T05:43:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 1 -- 172.21.15.32:0/2064395794 wait complete. 2024-04-16T05:43:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d8ac11740 20 client.17679 trim_cache size 0 max 16384 2024-04-16T05:43:40.446 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:43:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 -- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5d680a3d20 msgr2=0x7f5d680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5d680a3d20 0x7f5d680a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d7009c9c0 tx=0x7f5d70091ec0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 -- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8019a1f0 msgr2=0x7f5d801a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8019a1f0 0x7f5d801a4750 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000eaa0 tx=0x7f5d7000ef60 comp rx=0 tx=0).stop 2024-04-16T05:43:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 -- 172.21.15.32:0/758936337 shutdown_connections 2024-04-16T05:43:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f5d680a3d20 0x7f5d680a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8019a1f0 0x7f5d801a4750 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d80106200 0x7f5d80199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 --2- 172.21.15.32:0/758936337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d80101130 0x7f5d801996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 -- 172.21.15.32:0/758936337 >> 172.21.15.32:0/758936337 conn(0x7f5d800780b0 msgr2=0x7f5d800fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 -- 172.21.15.32:0/758936337 shutdown_connections 2024-04-16T05:43:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:40.437+0000 7f5d87f50640 1 -- 172.21.15.32:0/758936337 wait complete. 2024-04-16T05:43:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:40 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/859656561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:41.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:40 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/859656561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:41.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:40 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/859656561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:43:41.450 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:43:41.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:43:41.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:43:41.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 -- 172.21.15.32:0/243791389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103280 msgr2=0x7f0ce4103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 --2- 172.21.15.32:0/243791389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103280 0x7f0ce4103660 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009900 tx=0x7f0cd402bd60 comp rx=0 tx=0).stop 2024-04-16T05:43:41.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 -- 172.21.15.32:0/243791389 shutdown_connections 2024-04-16T05:43:41.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 --2- 172.21.15.32:0/243791389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce410c5c0 0x7f0ce410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 --2- 172.21.15.32:0/243791389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ce4103ba0 0x7f0ce410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 --2- 172.21.15.32:0/243791389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103280 0x7f0ce4103660 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 -- 172.21.15.32:0/243791389 >> 172.21.15.32:0/243791389 conn(0x7f0ce4078000 msgr2=0x7f0ce4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 -- 172.21.15.32:0/243791389 shutdown_connections 2024-04-16T05:43:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.575+0000 7f0cea64a640 1 -- 172.21.15.32:0/243791389 wait complete. 2024-04-16T05:43:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cea64a640 1 Processor -- start 2024-04-16T05:43:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cea64a640 1 -- start start 2024-04-16T05:43:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cea64a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ce4103280 0x7f0ce419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cea64a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103ba0 0x7f0ce419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cea64a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce410c5c0 0x7f0ce4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cea64a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41123f0 con 0x7f0ce4103ba0 2024-04-16T05:43:41.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cea64a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4112270 con 0x7f0ce4103280 2024-04-16T05:43:41.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cea64a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4112570 con 0x7f0ce410c5c0 2024-04-16T05:43:41.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cdbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103ba0 0x7f0ce419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cdbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103ba0 0x7f0ce419eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60470/0 (socket says 172.21.15.32:60470) 2024-04-16T05:43:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cdbfff640 1 -- 172.21.15.32:0/59056585 learned_addr learned my addr 172.21.15.32:0/59056585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ce4103280 0x7f0ce419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce410c5c0 0x7f0ce4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cdbfff640 1 -- 172.21.15.32:0/59056585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce410c5c0 msgr2=0x7f0ce4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce410c5c0 0x7f0ce4198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cdbfff640 1 -- 172.21.15.32:0/59056585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ce4103280 msgr2=0x7f0ce419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ce4103280 0x7f0ce419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0cdbfff640 1 -- 172.21.15.32:0/59056585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce4198ef0 con 0x7f0ce4103ba0 2024-04-16T05:43:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.576+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ce4103280 0x7f0ce419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.577+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103ba0 0x7f0ce419eab0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00c920 tx=0x7f0ccc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.577+0000 7f0ce1ffb640 1 -- 172.21.15.32:0/59056585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc007c70 con 0x7f0ce4103ba0 2024-04-16T05:43:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.577+0000 7f0ce1ffb640 1 -- 172.21.15.32:0/59056585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ccc007e10 con 0x7f0ce4103ba0 2024-04-16T05:43:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.577+0000 7f0cea64a640 1 -- 172.21.15.32:0/59056585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce41991e0 con 0x7f0ce4103ba0 2024-04-16T05:43:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.577+0000 7f0ce1ffb640 1 -- 172.21.15.32:0/59056585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc018890 con 0x7f0ce4103ba0 2024-04-16T05:43:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.577+0000 7f0cea64a640 1 -- 172.21.15.32:0/59056585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce41a54f0 con 0x7f0ce4103ba0 2024-04-16T05:43:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.579+0000 7f0ce1ffb640 1 -- 172.21.15.32:0/59056585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ccc018a70 con 0x7f0ce4103ba0 2024-04-16T05:43:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.579+0000 7f0ced232740 1 Processor -- start 2024-04-16T05:43:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ced232740 1 -- start start 2024-04-16T05:43:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce1ffb640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0cd00a3da0 0x7f0cd00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ced232740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a34e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ced232740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a3da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ced232740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ced232740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561fc1999e90 con 0x561fc1a34ab0 2024-04-16T05:43:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ced232740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561fc19374c0 con 0x561fc1a3dfb0 2024-04-16T05:43:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ced232740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fc1995e50 con 0x561fc1a353d0 2024-04-16T05:43:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a34e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0cd00a3da0 0x7f0cd00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a34e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60486/0 (socket says 172.21.15.32:60486) 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce3fff640 1 -- 172.21.15.32:0/2219726015 learned_addr learned my addr 172.21.15.32:0/2219726015 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce1ffb640 1 -- 172.21.15.32:0/59056585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f0ccc0d0d40 con 0x7f0ce4103ba0 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a3da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce3fff640 1 -- 172.21.15.32:0/2219726015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 msgr2=0x561fc1a3da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a3da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce3fff640 1 -- 172.21.15.32:0/2219726015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 msgr2=0x561fc1a403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce3fff640 1 -- 172.21.15.32:0/2219726015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc17747e0 con 0x561fc1a34ab0 2024-04-16T05:43:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.580+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0cd00a3da0 0x7f0cd00a6260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0092f0 tx=0x7f0cdc009f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a34e90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009900 tx=0x7f0cd402c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937ac535f46a066e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0cda7fc640 1 -- 172.21.15.32:0/2219726015 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd402dcc0 con 0x561fc1a34ab0 2024-04-16T05:43:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0cda7fc640 1 -- 172.21.15.32:0/2219726015 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4005040 con 0x561fc1a34ab0 2024-04-16T05:43:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0cda7fc640 1 -- 172.21.15.32:0/2219726015 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40048c0 con 0x561fc1a34ab0 2024-04-16T05:43:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ced232740 1 -- 172.21.15.32:0/2219726015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 msgr2=0x561fc1a34e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ced232740 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a34e90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009900 tx=0x7f0cd402c830 comp rx=0 tx=0).stop 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ced232740 1 -- 172.21.15.32:0/2219726015 shutdown_connections 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ced232740 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ced232740 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a3da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ced232740 1 --2- 172.21.15.32:0/2219726015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a34e90 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ced232740 1 -- 172.21.15.32:0/2219726015 >> 172.21.15.32:0/2219726015 conn(0x561fc1a2f1e0 msgr2=0x561fc1a2f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.581+0000 7f0ced232740 1 -- 172.21.15.32:0/2219726015 shutdown_connections 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 -- 172.21.15.32:0/2219726015 wait complete. 2024-04-16T05:43:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 -- start start 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 Processor -- start 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a3b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561fc1a40d40 con 0x561fc1a34ab0 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561fc1922410 con 0x561fc1a3dfb0 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fc1a40ec0 con 0x561fc1a353d0 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ced232740 10 client.0 _ll_register_callbacks cb 0x561fc191db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60502/0 (socket says 172.21.15.32:60502) 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.582+0000 7f0ce3fff640 1 -- 172.21.15.32:0/986843696 learned_addr learned my addr 172.21.15.32:0/986843696 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a3b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ce3fff640 1 -- 172.21.15.32:0/986843696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 msgr2=0x561fc1a3b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a3b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ce3fff640 1 -- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 msgr2=0x561fc1a3ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a3ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ce3fff640 1 -- 172.21.15.32:0/986843696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc17d4f70 con 0x561fc1a34ab0 2024-04-16T05:43:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a3ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3af90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0cd402c2a0 tx=0x7f0cd40054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c963803025c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0cc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0cc37fe640 1 -- 172.21.15.32:0/986843696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd404c020 con 0x561fc1a34ab0 2024-04-16T05:43:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0cc37fe640 1 -- 172.21.15.32:0/986843696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f0cd4004b90 con 0x561fc1a34ab0 2024-04-16T05:43:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ced232740 1 -- 172.21.15.32:0/986843696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd4005040 con 0x561fc1a34ab0 2024-04-16T05:43:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0cc37fe640 1 -- 172.21.15.32:0/986843696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0cd4057050 con 0x561fc1a34ab0 2024-04-16T05:43:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ced232740 10 client.17691 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.583+0000 7f0ced232740 1 -- 172.21.15.32:0/986843696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561fc166c170 con 0x561fc1a34ab0 2024-04-16T05:43:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0cc37fe640 1 -- 172.21.15.32:0/986843696 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4005040 con 0x561fc1a34ab0 2024-04-16T05:43:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0cc37fe640 1 -- 172.21.15.32:0/986843696 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0cd4034440 con 0x561fc1a34ab0 2024-04-16T05:43:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0ced232740 10 client.17691 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0ced232740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0ced232740 20 client.17691 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0ced232740 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x561fc1a4e110 0x561fc1a50500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0ced232740 4 client.17691 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0ced232740 1 -- 172.21.15.32:0/986843696 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fc19a78b0 con 0x561fc1a4e110 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x561fc1a4e110 0x561fc1a50500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x561fc1a4e110 0x561fc1a50500 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b8d1ac98ba3926 server_cookie=7935f316a4fece3a in_seq=0 out_seq=0 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.584+0000 7f0cc37fe640 10 client.17691 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0cc37fe640 1 -- 172.21.15.32:0/986843696 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fc19a78b0 con 0x561fc1a4e110 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0cc37fe640 10 client.17691 handle_command_reply: tid=0 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 client.17691 shutdown 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 -- 172.21.15.32:0/986843696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 msgr2=0x561fc1a3af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3af90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0cd402c2a0 tx=0x7f0cd40054a0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 -- 172.21.15.32:0/986843696 shutdown_connections 2024-04-16T05:43:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a3ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x561fc1a4e110 0x561fc1a50500 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a3b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 --2- 172.21.15.32:0/986843696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3af90 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0cc37fe640 0 client.17691 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.585+0000 7f0ced232740 1 -- 172.21.15.32:0/986843696 >> 172.21.15.32:0/986843696 conn(0x561fc1a2f1e0 msgr2=0x561fc1a30f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.586+0000 7f0ced232740 1 -- 172.21.15.32:0/986843696 shutdown_connections 2024-04-16T05:43:41.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.586+0000 7f0ced232740 1 -- 172.21.15.32:0/986843696 wait complete. 2024-04-16T05:43:41.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.586+0000 7f0ced232740 20 client.17691 trim_cache size 0 max 16384 2024-04-16T05:43:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.603+0000 7f0ced232740 1 Processor -- start 2024-04-16T05:43:41.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.603+0000 7f0ced232740 1 -- start start 2024-04-16T05:43:41.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ced232740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ced232740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ced232740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a55ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ced232740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41b1ca0 con 0x561fc1a34ab0 2024-04-16T05:43:41.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ced232740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41af570 con 0x561fc1a3dfb0 2024-04-16T05:43:41.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ced232740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41ad3e0 con 0x561fc1a353d0 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60506/0 (socket says 172.21.15.32:60506) 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 -- 172.21.15.32:0/3399424030 learned_addr learned my addr 172.21.15.32:0/3399424030 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a55ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 -- 172.21.15.32:0/3399424030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 msgr2=0x561fc1a464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 -- 172.21.15.32:0/3399424030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 msgr2=0x561fc1a55ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a55ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 -- 172.21.15.32:0/3399424030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc1a30b50 con 0x561fc1a34ab0 2024-04-16T05:43:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3d6c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4004650 tx=0x7f0cd4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511642fcd4b82de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0cd9ffb640 1 -- 172.21.15.32:0/3399424030 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd404c040 con 0x561fc1a34ab0 2024-04-16T05:43:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0cd9ffb640 1 -- 172.21.15.32:0/3399424030 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4004e70 con 0x561fc1a34ab0 2024-04-16T05:43:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.604+0000 7f0cd9ffb640 1 -- 172.21.15.32:0/3399424030 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4040470 con 0x561fc1a34ab0 2024-04-16T05:43:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 -- 172.21.15.32:0/3399424030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 msgr2=0x561fc1a3d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3d6c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4004650 tx=0x7f0cd4005fd0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 -- 172.21.15.32:0/3399424030 shutdown_connections 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1a55ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a353d0 0x561fc1a464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 --2- 172.21.15.32:0/3399424030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a34ab0 0x561fc1a3d6c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 -- 172.21.15.32:0/3399424030 >> 172.21.15.32:0/3399424030 conn(0x561fc1a2f1e0 msgr2=0x561fc1a2d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 -- 172.21.15.32:0/3399424030 shutdown_connections 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 -- 172.21.15.32:0/3399424030 wait complete. 2024-04-16T05:43:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 -- start start 2024-04-16T05:43:41.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.605+0000 7f0ced232740 1 Processor -- start 2024-04-16T05:43:41.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ced232740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a34ab0 0x561fc1ab1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ced232740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a353d0 0x561fc1ab20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ced232740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1aac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ced232740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41ada30 con 0x561fc1a353d0 2024-04-16T05:43:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ced232740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41aeea0 con 0x561fc1a3dfb0 2024-04-16T05:43:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ced232740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41ae100 con 0x561fc1a34ab0 2024-04-16T05:43:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ced232740 10 client.0 _ll_register_callbacks cb 0x7f0ccc098b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a353d0 0x561fc1ab20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a353d0 0x561fc1ab20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60516/0 (socket says 172.21.15.32:60516) 2024-04-16T05:43:41.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 -- 172.21.15.32:0/378925744 learned_addr learned my addr 172.21.15.32:0/378925744 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a34ab0 0x561fc1ab1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1aac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 -- 172.21.15.32:0/378925744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a34ab0 msgr2=0x561fc1ab1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a34ab0 0x561fc1ab1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 -- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 msgr2=0x561fc1aac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1aac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 -- 172.21.15.32:0/378925744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc1aac9f0 con 0x561fc1a353d0 2024-04-16T05:43:41.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a34ab0 0x561fc1ab1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ce8be7640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1aac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cdbfff640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a353d0 0x561fc1ab20d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0ce100 tx=0x7f0ccc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6000ed1239c034a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cc37fe640 1 -- 172.21.15.32:0/378925744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc09abf0 con 0x561fc1a353d0 2024-04-16T05:43:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0cc37fe640 1 -- 172.21.15.32:0/378925744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7f0ccc018d60 con 0x561fc1a353d0 2024-04-16T05:43:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.606+0000 7f0ced232740 1 -- 172.21.15.32:0/378925744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd4004e70 con 0x561fc1a353d0 2024-04-16T05:43:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0cc37fe640 1 -- 172.21.15.32:0/378925744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ccc00fb30 con 0x561fc1a353d0 2024-04-16T05:43:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ced232740 10 client.17697 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ced232740 1 -- 172.21.15.32:0/378925744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561fc1ab92b0 con 0x561fc1a353d0 2024-04-16T05:43:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0cc37fe640 1 -- 172.21.15.32:0/378925744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4004e70 con 0x561fc1a353d0 2024-04-16T05:43:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0cc37fe640 1 -- 172.21.15.32:0/378925744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0ccc0c54f0 con 0x561fc1a353d0 2024-04-16T05:43:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ced232740 10 client.17697 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ced232740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ced232740 20 client.17697 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ced232740 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x561fc1ab99e0 0x561fc1abbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ced232740 4 client.17697 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:43:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ced232740 1 -- 172.21.15.32:0/378925744 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561fc19a1720 con 0x561fc1ab99e0 2024-04-16T05:43:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.607+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x561fc1ab99e0 0x561fc1abbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.608+0000 7f0cc37fe640 10 client.17697 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.608+0000 7f0ce3fff640 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x561fc1ab99e0 0x561fc1abbe00 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2702bb47accc3e6 server_cookie=2ca1eb0542122f86 in_seq=0 out_seq=0 2024-04-16T05:43:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0cc37fe640 1 -- 172.21.15.32:0/378925744 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561fc19a1720 con 0x561fc1ab99e0 2024-04-16T05:43:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0cc37fe640 10 client.17697 handle_command_reply: tid=0 2024-04-16T05:43:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 client.17697 shutdown 2024-04-16T05:43:41.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 -- 172.21.15.32:0/378925744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a353d0 msgr2=0x561fc1ab20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a353d0 0x561fc1ab20d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0ce100 tx=0x7f0ccc00cf10 comp rx=0 tx=0).stop 2024-04-16T05:43:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 -- 172.21.15.32:0/378925744 shutdown_connections 2024-04-16T05:43:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fc1a3dfb0 0x561fc1aac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561fc1a353d0 0x561fc1ab20d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x561fc1ab99e0 0x561fc1abbe00 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 --2- 172.21.15.32:0/378925744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fc1a34ab0 0x561fc1ab1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 -- 172.21.15.32:0/378925744 >> 172.21.15.32:0/378925744 conn(0x561fc1a2f1e0 msgr2=0x561fc1a2e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0cc37fe640 0 client.17697 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 -- 172.21.15.32:0/378925744 shutdown_connections 2024-04-16T05:43:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.609+0000 7f0ced232740 1 -- 172.21.15.32:0/378925744 wait complete. 2024-04-16T05:43:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0ced232740 20 client.17697 trim_cache size 0 max 16384 2024-04-16T05:43:41.618 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:43:41.618 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:43:41.619 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "ca567085-84b2-4e3a-a897-5e29f8f24349", 2024-04-16T05:43:41.619 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:43:41.619 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:43:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 -- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0cd00a3da0 msgr2=0x7f0cd00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0cd00a3da0 0x7f0cd00a6260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0092f0 tx=0x7f0cdc009f90 comp rx=0 tx=0).stop 2024-04-16T05:43:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 -- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103ba0 msgr2=0x7f0ce419eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103ba0 0x7f0ce419eab0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00c920 tx=0x7f0ccc00cde0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 -- 172.21.15.32:0/59056585 shutdown_connections 2024-04-16T05:43:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce410c5c0 0x7f0ce4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f0cd00a3da0 0x7f0cd00a6260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce4103ba0 0x7f0ce419eab0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 --2- 172.21.15.32:0/59056585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ce4103280 0x7f0ce419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 -- 172.21.15.32:0/59056585 >> 172.21.15.32:0/59056585 conn(0x7f0ce4078000 msgr2=0x7f0ce40fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 -- 172.21.15.32:0/59056585 shutdown_connections 2024-04-16T05:43:41.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.610+0000 7f0cea64a640 1 -- 172.21.15.32:0/59056585 wait complete. 2024-04-16T05:43:41.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:43:41.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:43:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/1973791480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0102240 msgr2=0x7ff8f0102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/1973791480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0102240 0x7ff8f0102620 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009900 tx=0x7ff8e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:43:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/1973791480 shutdown_connections 2024-04-16T05:43:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/1973791480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0107310 0x7ff8f010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/1973791480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8f0102bf0 0x7ff8f0106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/1973791480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0102240 0x7ff8f0102620 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/1973791480 >> 172.21.15.32:0/1973791480 conn(0x7ff8f0079450 msgr2=0x7ff8f0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/1973791480 shutdown_connections 2024-04-16T05:43:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.732+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/1973791480 wait complete. 2024-04-16T05:43:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f6e0d640 1 Processor -- start 2024-04-16T05:43:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f6e0d640 1 -- start start 2024-04-16T05:43:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f6e0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0102240 0x7ff8f019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f6e0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0107310 0x7ff8f019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f6e0d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8f0198700 0x7ff8f0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f6e0d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01123c0 con 0x7ff8f0107310 2024-04-16T05:43:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f6e0d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f0112240 con 0x7ff8f0102240 2024-04-16T05:43:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f6e0d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f0112540 con 0x7ff8f0198700 2024-04-16T05:43:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0107310 0x7ff8f019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0107310 0x7ff8f019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60536/0 (socket says 172.21.15.32:60536) 2024-04-16T05:43:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 -- 172.21.15.32:0/4132607911 learned_addr learned my addr 172.21.15.32:0/4132607911 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8f0198700 0x7ff8f0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0102240 0x7ff8f019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 -- 172.21.15.32:0/4132607911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8f0198700 msgr2=0x7ff8f0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8f0198700 0x7ff8f0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 -- 172.21.15.32:0/4132607911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0102240 msgr2=0x7ff8f019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0102240 0x7ff8f019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 -- 172.21.15.32:0/4132607911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f0199300 con 0x7ff8f0107310 2024-04-16T05:43:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0102240 0x7ff8f019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.733+0000 7ff8effff640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0107310 0x7ff8f019eb80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00d740 tx=0x7ff8dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.734+0000 7ff8edffb640 1 -- 172.21.15.32:0/4132607911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc017070 con 0x7ff8f0107310 2024-04-16T05:43:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.734+0000 7ff8edffb640 1 -- 172.21.15.32:0/4132607911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc00de50 con 0x7ff8f0107310 2024-04-16T05:43:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.734+0000 7ff8edffb640 1 -- 172.21.15.32:0/4132607911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc004e00 con 0x7ff8f0107310 2024-04-16T05:43:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.734+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/4132607911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f01a53e0 con 0x7ff8f0107310 2024-04-16T05:43:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.734+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/4132607911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f01a57e0 con 0x7ff8f0107310 2024-04-16T05:43:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.736+0000 7ff8edffb640 1 -- 172.21.15.32:0/4132607911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff8dc004fa0 con 0x7ff8f0107310 2024-04-16T05:43:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.736+0000 7ff8f99f5740 1 Processor -- start 2024-04-16T05:43:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f99f5740 1 -- start start 2024-04-16T05:43:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8edffb640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8e00a3da0 0x7ff8e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff0a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff12e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff13350 0x55ae1ff15740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ae1fe6f180 con 0x55ae1ff09e50 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ae1fe0c360 con 0x55ae1ff13350 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ae1fe6b140 con 0x55ae1ff0a770 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8e00a3da0 0x7ff8e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f4ba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff0a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f4ba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff0a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60552/0 (socket says 172.21.15.32:60552) 2024-04-16T05:43:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f4ba9640 1 -- 172.21.15.32:0/1598678007 learned_addr learned my addr 172.21.15.32:0/1598678007 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8edffb640 1 -- 172.21.15.32:0/4132607911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7ff8dc0d0530 con 0x7ff8f0107310 2024-04-16T05:43:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8effff640 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff12e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff13350 0x55ae1ff15740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f4ba9640 1 -- 172.21.15.32:0/1598678007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 msgr2=0x55ae1ff12e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff12e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f4ba9640 1 -- 172.21.15.32:0/1598678007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff13350 msgr2=0x55ae1ff15740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff13350 0x55ae1ff15740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.737+0000 7ff8f4ba9640 1 -- 172.21.15.32:0/1598678007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae1fc4b7d0 con 0x55ae1ff09e50 2024-04-16T05:43:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8e00a3da0 0x7ff8e00a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005fd0 tx=0x7ff8e8009db0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff0a230 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4000c00 tx=0x7ff8e402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143fe3598bcedacb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8daffd640 1 -- 172.21.15.32:0/1598678007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e402ca80 con 0x55ae1ff09e50 2024-04-16T05:43:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8daffd640 1 -- 172.21.15.32:0/1598678007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e402cc20 con 0x55ae1ff09e50 2024-04-16T05:43:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8daffd640 1 -- 172.21.15.32:0/1598678007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4004920 con 0x55ae1ff09e50 2024-04-16T05:43:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f99f5740 1 -- 172.21.15.32:0/1598678007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 msgr2=0x55ae1ff0a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff0a230 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4000c00 tx=0x7ff8e402c5d0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f99f5740 1 -- 172.21.15.32:0/1598678007 shutdown_connections 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff13350 0x55ae1ff15740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff12e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/1598678007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff0a230 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.738+0000 7ff8f99f5740 1 -- 172.21.15.32:0/1598678007 >> 172.21.15.32:0/1598678007 conn(0x55ae1ff04580 msgr2=0x55ae1ff04960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 -- 172.21.15.32:0/1598678007 shutdown_connections 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 -- 172.21.15.32:0/1598678007 wait complete. 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 -- start start 2024-04-16T05:43:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 Processor -- start 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff10330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff10870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff10db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ae1ff160e0 con 0x55ae1ff13350 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ae1fe282c0 con 0x55ae1ff09e50 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ae1ff16260 con 0x55ae1ff0a770 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f99f5740 10 client.0 _ll_register_callbacks cb 0x55ae1fe0f090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f53aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff10db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f53aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff10db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60564/0 (socket says 172.21.15.32:60564) 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f53aa640 1 -- 172.21.15.32:0/2215594645 learned_addr learned my addr 172.21.15.32:0/2215594645 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff10330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.739+0000 7ff8effff640 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff10870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f53aa640 1 -- 172.21.15.32:0/2215594645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 msgr2=0x55ae1ff10870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff10870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f53aa640 1 -- 172.21.15.32:0/2215594645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 msgr2=0x55ae1ff10330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff10330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f53aa640 1 -- 172.21.15.32:0/2215594645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae1fb813f0 con 0x55ae1ff13350 2024-04-16T05:43:41.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff10db0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8010ea0 tx=0x7ff8e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43612f563583392 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8cffff640 1 -- 172.21.15.32:0/2215594645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8011e30 con 0x55ae1ff13350 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8cffff640 1 -- 172.21.15.32:0/2215594645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7ff8e8004b20 con 0x55ae1ff13350 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f99f5740 1 -- 172.21.15.32:0/2215594645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e402cc20 con 0x55ae1ff13350 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8cffff640 1 -- 172.21.15.32:0/2215594645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8e8030050 con 0x55ae1ff13350 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f99f5740 10 client.17709 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8f99f5740 1 -- 172.21.15.32:0/2215594645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae1fb410f0 con 0x55ae1ff13350 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.740+0000 7ff8cffff640 1 -- 172.21.15.32:0/2215594645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e402cc20 con 0x55ae1ff13350 2024-04-16T05:43:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8cffff640 1 -- 172.21.15.32:0/2215594645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff8e801f050 con 0x55ae1ff13350 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8f99f5740 10 client.17709 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8f99f5740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8f99f5740 20 client.17709 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ae1ff23610 0x55ae1ff25a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8f99f5740 4 client.17709 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8f99f5740 1 -- 172.21.15.32:0/2215594645 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae1fe7cba0 con 0x55ae1ff23610 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ae1ff23610 0x55ae1ff25a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8cffff640 10 client.17709 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.741+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ae1ff23610 0x55ae1ff25a40 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55a21f3eb706355 server_cookie=422bbc1b81e7bcec in_seq=0 out_seq=0 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8cffff640 1 -- 172.21.15.32:0/2215594645 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae1fe7cba0 con 0x55ae1ff23610 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8cffff640 10 client.17709 handle_command_reply: tid=0 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 client.17709 shutdown 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 -- 172.21.15.32:0/2215594645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 msgr2=0x55ae1ff10db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff10db0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8010ea0 tx=0x7ff8e8004ab0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 -- 172.21.15.32:0/2215594645 shutdown_connections 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff10db0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ae1ff23610 0x55ae1ff25a40 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff10870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/2215594645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff10330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8cffff640 0 client.17709 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 -- 172.21.15.32:0/2215594645 >> 172.21.15.32:0/2215594645 conn(0x55ae1ff04580 msgr2=0x55ae1ff062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 -- 172.21.15.32:0/2215594645 shutdown_connections 2024-04-16T05:43:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.742+0000 7ff8f99f5740 1 -- 172.21.15.32:0/2215594645 wait complete. 2024-04-16T05:43:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.743+0000 7ff8f99f5740 20 client.17709 trim_cache size 0 max 16384 2024-04-16T05:43:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f99f5740 1 Processor -- start 2024-04-16T05:43:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f99f5740 1 -- start start 2024-04-16T05:43:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff12a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff0a770 0x55ae1ff0f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff13350 0x55ae1ff1efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01b1c70 con 0x55ae1ff09e50 2024-04-16T05:43:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01af540 con 0x55ae1ff0a770 2024-04-16T05:43:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01ad3b0 con 0x55ae1ff13350 2024-04-16T05:43:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f4ba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff12a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f4ba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff12a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60574/0 (socket says 172.21.15.32:60574) 2024-04-16T05:43:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f4ba9640 1 -- 172.21.15.32:0/643338887 learned_addr learned my addr 172.21.15.32:0/643338887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8effff640 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff0a770 0x55ae1ff0f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f4ba9640 1 -- 172.21.15.32:0/643338887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff13350 msgr2=0x55ae1ff1efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff13350 0x55ae1ff1efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.760+0000 7ff8f4ba9640 1 -- 172.21.15.32:0/643338887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff0a770 msgr2=0x55ae1ff0f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff0a770 0x55ae1ff0f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f4ba9640 1 -- 172.21.15.32:0/643338887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae1ff09170 con 0x55ae1ff09e50 2024-04-16T05:43:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8effff640 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff0a770 0x55ae1ff0f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:43:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff12a60 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005c40 tx=0x7ff8e40054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5c4c020b9611b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8da7fc640 1 -- 172.21.15.32:0/643338887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e402c9c0 con 0x55ae1ff09e50 2024-04-16T05:43:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8da7fc640 1 -- 172.21.15.32:0/643338887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e402cb60 con 0x55ae1ff09e50 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8da7fc640 1 -- 172.21.15.32:0/643338887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4034780 con 0x55ae1ff09e50 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 -- 172.21.15.32:0/643338887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 msgr2=0x55ae1ff12a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff12a60 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005c40 tx=0x7ff8e40054d0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 -- 172.21.15.32:0/643338887 shutdown_connections 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff13350 0x55ae1ff1efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff0a770 0x55ae1ff0f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/643338887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff09e50 0x55ae1ff12a60 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 -- 172.21.15.32:0/643338887 >> 172.21.15.32:0/643338887 conn(0x55ae1ff04580 msgr2=0x55ae1ff03760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 -- 172.21.15.32:0/643338887 shutdown_connections 2024-04-16T05:43:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.761+0000 7ff8f99f5740 1 -- 172.21.15.32:0/643338887 wait complete. 2024-04-16T05:43:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 1 -- start start 2024-04-16T05:43:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 1 Processor -- start 2024-04-16T05:43:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff1b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff18b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01ada00 con 0x55ae1ff13350 2024-04-16T05:43:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01aee70 con 0x55ae1ff09e50 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01ae0d0 con 0x55ae1ff0a770 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f99f5740 10 client.0 _ll_register_callbacks cb 0x7ff8e8012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f53aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f53aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff190c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60586/0 (socket says 172.21.15.32:60586) 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f53aa640 1 -- 172.21.15.32:0/363697585 learned_addr learned my addr 172.21.15.32:0/363697585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff1b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.762+0000 7ff8effff640 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff18b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f53aa640 1 -- 172.21.15.32:0/363697585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 msgr2=0x55ae1ff18b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff18b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f53aa640 1 -- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 msgr2=0x55ae1ff1b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:43:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff1b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f53aa640 1 -- 172.21.15.32:0/363697585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae1ff197b0 con 0x55ae1ff13350 2024-04-16T05:43:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff1b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:43:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f53aa640 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff190c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80203e0 tx=0x7ff8e8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f270ad95bd7c729 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:43:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:43:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8cffff640 1 -- 172.21.15.32:0/363697585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e802c020 con 0x55ae1ff13350 2024-04-16T05:43:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8cffff640 1 -- 172.21.15.32:0/363697585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9438+0+0 (secure 0 0 0) 0x7ff8e80127a0 con 0x55ae1ff13350 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 1 -- 172.21.15.32:0/363697585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e402cb60 con 0x55ae1ff13350 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8cffff640 1 -- 172.21.15.32:0/363697585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8e803d050 con 0x55ae1ff13350 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 10 client.17715 fetch_fsmap learned FSMap version 25 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 1 -- 172.21.15.32:0/363697585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae1ff89a10 con 0x55ae1ff13350 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8cffff640 1 -- 172.21.15.32:0/363697585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e402cb60 con 0x55ae1ff13350 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8cffff640 1 -- 172.21.15.32:0/363697585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff8e800f040 con 0x55ae1ff13350 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 10 client.17715 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:43:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 20 client.17715 populate_metadata read hostname 'smithi032' 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ae1ff8a100 0x55ae1ff8c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 4 client.17715 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.763+0000 7ff8f99f5740 1 -- 172.21.15.32:0/363697585 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae1fe76a10 con 0x55ae1ff8a100 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.764+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ae1ff8a100 0x55ae1ff8c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.764+0000 7ff8f4ba9640 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ae1ff8a100 0x55ae1ff8c520 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918e20e0cdeb04e3 server_cookie=748c87f69a34308c in_seq=0 out_seq=0 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.764+0000 7ff8cffff640 10 client.17715 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.764+0000 7ff8cffff640 1 -- 172.21.15.32:0/363697585 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae1fe76a10 con 0x55ae1ff8a100 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.764+0000 7ff8cffff640 10 client.17715 handle_command_reply: tid=0 2024-04-16T05:43:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.764+0000 7ff8f99f5740 1 client.17715 shutdown 2024-04-16T05:43:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.764+0000 7ff8f99f5740 1 -- 172.21.15.32:0/363697585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 msgr2=0x55ae1ff190c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.764+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff190c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80203e0 tx=0x7ff8e8011c40 comp rx=0 tx=0).stop 2024-04-16T05:43:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 1 -- 172.21.15.32:0/363697585 shutdown_connections 2024-04-16T05:43:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae1ff13350 0x55ae1ff190c0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55ae1ff8a100 0x55ae1ff8c520 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae1ff0a770 0x55ae1ff18b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 1 --2- 172.21.15.32:0/363697585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae1ff09e50 0x55ae1ff1b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8cffff640 0 client.17715 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:43:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 1 -- 172.21.15.32:0/363697585 >> 172.21.15.32:0/363697585 conn(0x55ae1ff04580 msgr2=0x55ae1ff02ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 1 -- 172.21.15.32:0/363697585 shutdown_connections 2024-04-16T05:43:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 1 -- 172.21.15.32:0/363697585 wait complete. 2024-04-16T05:43:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f99f5740 20 client.17715 trim_cache size 0 max 16384 2024-04-16T05:43:41.774 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:43:41.774 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:43:41.775 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:43:41.775 INFO:teuthology.orchestra.run.smithi032.stdout: "ca567085-84b2-4e3a-a897-5e29f8f24349": { 2024-04-16T05:43:41.775 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:43:41.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "ca567085-84b2-4e3a-a897-5e29f8f24349", 2024-04-16T05:43:41.775 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:43:41.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:43:41.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:43:41.775 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:43:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8e00a3da0 msgr2=0x7ff8e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8e00a3da0 0x7ff8e00a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005fd0 tx=0x7ff8e8009db0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0107310 msgr2=0x7ff8f019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:43:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0107310 0x7ff8f019eb80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00d740 tx=0x7ff8dc00dc00 comp rx=0 tx=0).stop 2024-04-16T05:43:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/4132607911 shutdown_connections 2024-04-16T05:43:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8f0198700 0x7ff8f0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff8e00a3da0 0x7ff8e00a6260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8f0107310 0x7ff8f019eb80 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 --2- 172.21.15.32:0/4132607911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0102240 0x7ff8f019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:43:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/4132607911 >> 172.21.15.32:0/4132607911 conn(0x7ff8f0079450 msgr2=0x7ff8f00fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:43:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/4132607911 shutdown_connections 2024-04-16T05:43:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:43:41.765+0000 7ff8f6e0d640 1 -- 172.21.15.32:0/4132607911 wait complete. 2024-04-16T05:43:41.778 INFO:tasks.cephfs.filesystem:scrub status for tag:ca567085-84b2-4e3a-a897-5e29f8f24349 - {'path': '/', 'tag': 'ca567085-84b2-4e3a-a897-5e29f8f24349', 'options': 'recursive,force'} 2024-04-16T05:43:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:41 smithi134 ceph-mon[36153]: pgmap v269: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:41 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:43:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:41 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:41 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:43:42.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:41 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:42.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:41 smithi082 ceph-mon[36195]: pgmap v269: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:42.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:41 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:43:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:41 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:41 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:43:42.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:41 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:41 smithi032 ceph-mon[32351]: pgmap v269: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:41 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:43:42.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:41 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:42.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:41 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:43:42.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:41 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:43:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:43 smithi134 ceph-mon[36153]: pgmap v270: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:44.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:43 smithi082 ceph-mon[36195]: pgmap v270: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:43 smithi032 ceph-mon[32351]: pgmap v270: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:45 smithi134 ceph-mon[36153]: pgmap v271: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:45 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:43:46.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:45 smithi082 ceph-mon[36195]: pgmap v271: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:46.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:45 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:43:46.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:45 smithi032 ceph-mon[32351]: pgmap v271: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:46.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:45 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:43:48.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:47 smithi134 ceph-mon[36153]: pgmap v272: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:48.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:47 smithi082 ceph-mon[36195]: pgmap v272: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:48.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:47 smithi032 ceph-mon[32351]: pgmap v272: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:49 smithi134 ceph-mon[36153]: pgmap v273: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:50.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:49 smithi082 ceph-mon[36195]: pgmap v273: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:50.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:49 smithi032 ceph-mon[32351]: pgmap v273: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:51 smithi134 ceph-mon[36153]: pgmap v274: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:52.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:51 smithi082 ceph-mon[36195]: pgmap v274: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:52.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:51 smithi032 ceph-mon[32351]: pgmap v274: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:43:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:53 smithi134 ceph-mon[36153]: pgmap v275: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:54.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:53 smithi082 ceph-mon[36195]: pgmap v275: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:54.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:53 smithi032 ceph-mon[32351]: pgmap v275: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:43:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:55 smithi134 ceph-mon[36153]: pgmap v276: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:56.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:55 smithi082 ceph-mon[36195]: pgmap v276: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:56.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:55 smithi032 ceph-mon[32351]: pgmap v276: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:57 smithi134 ceph-mon[36153]: pgmap v277: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:58.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:57 smithi082 ceph-mon[36195]: pgmap v277: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:43:58.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:57 smithi032 ceph-mon[32351]: pgmap v277: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:43:59 smithi134 ceph-mon[36153]: pgmap v278: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:00.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:43:59 smithi082 ceph-mon[36195]: pgmap v278: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:00.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:43:59 smithi032 ceph-mon[32351]: pgmap v278: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:02.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:01 smithi134 ceph-mon[36153]: pgmap v279: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:02.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:01 smithi082 ceph-mon[36195]: pgmap v279: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:02.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:01 smithi032 ceph-mon[32351]: pgmap v279: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:04.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:03 smithi134 ceph-mon[36153]: pgmap v280: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:44:04.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:03 smithi082 ceph-mon[36195]: pgmap v280: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:44:04.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:03 smithi032 ceph-mon[32351]: pgmap v280: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:44:06.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:05 smithi134 ceph-mon[36153]: pgmap v281: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:44:06.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:05 smithi082 ceph-mon[36195]: pgmap v281: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:44:06.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:05 smithi032 ceph-mon[32351]: pgmap v281: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:44:07.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:06 smithi134 ceph-mon[36153]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T05:44:07.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:06 smithi134 ceph-mon[36153]: pgmap v283: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:07.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:06 smithi082 ceph-mon[36195]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T05:44:07.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:06 smithi082 ceph-mon[36195]: pgmap v283: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:07.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:06 smithi032 ceph-mon[32351]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T05:44:07.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:06 smithi032 ceph-mon[32351]: pgmap v283: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:09.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:09 smithi032 ceph-mon[32351]: pgmap v284: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:09.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:09 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:44:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:09 smithi134 ceph-mon[36153]: pgmap v284: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:09 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:44:09.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:09 smithi082 ceph-mon[36195]: pgmap v284: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:09.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:09 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:44:10.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:10 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:44:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:10 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:44:10.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:10 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:44:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:10 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:44:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:10 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:44:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:10 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:44:10.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:10 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:44:10.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:10 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:44:10.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:10 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:44:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:11 smithi134 ceph-mon[36153]: pgmap v285: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:11.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:11 smithi082 ceph-mon[36195]: pgmap v285: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:11.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:11 smithi032 ceph-mon[32351]: pgmap v285: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-16T05:44:11.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:44:11.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:44:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 -- 172.21.15.32:0/817736263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 msgr2=0x7f677c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 --2- 172.21.15.32:0/817736263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 0x7f677c1016b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6764009930 tx=0x7f676402bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 -- 172.21.15.32:0/817736263 shutdown_connections 2024-04-16T05:44:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 --2- 172.21.15.32:0/817736263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c101bf0 0x7f677c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 --2- 172.21.15.32:0/817736263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 0x7f677c1016b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 --2- 172.21.15.32:0/817736263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c1070e0 0x7f677c1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 -- 172.21.15.32:0/817736263 >> 172.21.15.32:0/817736263 conn(0x7f677c076d00 msgr2=0x7f677c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 -- 172.21.15.32:0/817736263 shutdown_connections 2024-04-16T05:44:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.899+0000 7f6781372640 1 -- 172.21.15.32:0/817736263 wait complete. 2024-04-16T05:44:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f6781372640 1 Processor -- start 2024-04-16T05:44:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f6781372640 1 -- start start 2024-04-16T05:44:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f6781372640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 0x7f677c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f6781372640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c101bf0 0x7f677c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f6781372640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c1070e0 0x7f677c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f6781372640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f677c112460 con 0x7f677c101250 2024-04-16T05:44:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f6781372640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1122e0 con 0x7f677c101bf0 2024-04-16T05:44:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f6781372640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1125e0 con 0x7f677c1070e0 2024-04-16T05:44:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f677affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 0x7f677c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f677affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 0x7f677c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58428/0 (socket says 172.21.15.32:58428) 2024-04-16T05:44:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.900+0000 7f677affd640 1 -- 172.21.15.32:0/2020879822 learned_addr learned my addr 172.21.15.32:0/2020879822 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f67727fc640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c101bf0 0x7f677c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f677b7fe640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c1070e0 0x7f677c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f677affd640 1 -- 172.21.15.32:0/2020879822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c1070e0 msgr2=0x7f677c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f677affd640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c1070e0 0x7f677c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f677affd640 1 -- 172.21.15.32:0/2020879822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c101bf0 msgr2=0x7f677c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f677affd640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c101bf0 0x7f677c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f677affd640 1 -- 172.21.15.32:0/2020879822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c198fa0 con 0x7f677c101250 2024-04-16T05:44:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f67727fc640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c101bf0 0x7f677c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f677affd640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 0x7f677c19e620 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f676c00d770 tx=0x7f676c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f6778ff9640 1 -- 172.21.15.32:0/2020879822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c017070 con 0x7f677c101250 2024-04-16T05:44:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f6778ff9640 1 -- 172.21.15.32:0/2020879822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c005500 con 0x7f677c101250 2024-04-16T05:44:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f6781372640 1 -- 172.21.15.32:0/2020879822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c199290 con 0x7f677c101250 2024-04-16T05:44:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f6778ff9640 1 -- 172.21.15.32:0/2020879822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c004e00 con 0x7f677c101250 2024-04-16T05:44:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.901+0000 7f6781372640 1 -- 172.21.15.32:0/2020879822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1a55a0 con 0x7f677c101250 2024-04-16T05:44:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.903+0000 7f6778ff9640 1 -- 172.21.15.32:0/2020879822 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f676c004fa0 con 0x7f677c101250 2024-04-16T05:44:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.903+0000 7f6784032740 1 Processor -- start 2024-04-16T05:44:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f6784032740 1 -- start start 2024-04-16T05:44:11.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f6784032740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d561f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f6784032740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f6784032740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d56f760 0x560d7d571b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f6784032740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560d7d4c7180 con 0x560d7d56f760 2024-04-16T05:44:11.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f6784032740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560d7d464360 con 0x560d7d5624a0 2024-04-16T05:44:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f6784032740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d7d4c3140 con 0x560d7d561b80 2024-04-16T05:44:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f67727fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d56f760 0x560d7d571b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f67727fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d56f760 0x560d7d571b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58434/0 (socket says 172.21.15.32:58434) 2024-04-16T05:44:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f67727fc640 1 -- 172.21.15.32:0/362086335 learned_addr learned my addr 172.21.15.32:0/362086335 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f677b7fe640 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d561f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f6778ff9640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67680a3d20 0x7f67680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f677affd640 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f67727fc640 1 -- 172.21.15.32:0/362086335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 msgr2=0x560d7d561f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f67727fc640 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d561f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f677b7fe640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67680a3d20 0x7f67680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f67727fc640 1 -- 172.21.15.32:0/362086335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 msgr2=0x560d7d56f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f67727fc640 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.904+0000 7f67727fc640 1 -- 172.21.15.32:0/362086335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7d2a37d0 con 0x560d7d56f760 2024-04-16T05:44:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.905+0000 7f677affd640 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.905+0000 7f6778ff9640 1 -- 172.21.15.32:0/2020879822 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f676c0d0780 con 0x7f677c101250 2024-04-16T05:44:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.905+0000 7f67727fc640 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d56f760 0x560d7d571b50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6764006010 tx=0x7f676402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1c2670f3d052b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.905+0000 7f677b7fe640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67680a3d20 0x7f67680a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6774014750 tx=0x7f6774004ab0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.905+0000 7f67717fa640 1 -- 172.21.15.32:0/362086335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676402cb00 con 0x560d7d56f760 2024-04-16T05:44:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.905+0000 7f67717fa640 1 -- 172.21.15.32:0/362086335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676402cca0 con 0x560d7d56f760 2024-04-16T05:44:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.905+0000 7f67717fa640 1 -- 172.21.15.32:0/362086335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004920 con 0x560d7d56f760 2024-04-16T05:44:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 -- 172.21.15.32:0/362086335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d56f760 msgr2=0x560d7d571b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d56f760 0x560d7d571b50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6764006010 tx=0x7f676402c650 comp rx=0 tx=0).stop 2024-04-16T05:44:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 -- 172.21.15.32:0/362086335 shutdown_connections 2024-04-16T05:44:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d56f760 0x560d7d571b50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 --2- 172.21.15.32:0/362086335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d561f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 -- 172.21.15.32:0/362086335 >> 172.21.15.32:0/362086335 conn(0x560d7d55c580 msgr2=0x560d7d55c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 -- 172.21.15.32:0/362086335 shutdown_connections 2024-04-16T05:44:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 -- 172.21.15.32:0/362086335 wait complete. 2024-04-16T05:44:11.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 -- start start 2024-04-16T05:44:11.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.906+0000 7f6784032740 1 Processor -- start 2024-04-16T05:44:11.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f6784032740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d561b80 0x560d7d56c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f6784032740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d5624a0 0x560d7d56cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f6784032740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d56f760 0x560d7d56d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f6784032740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560d7d5724c0 con 0x560d7d561b80 2024-04-16T05:44:11.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f6784032740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560d7d4802c0 con 0x560d7d56f760 2024-04-16T05:44:11.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f6784032740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d7d572640 con 0x560d7d5624a0 2024-04-16T05:44:11.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f6784032740 10 client.0 _ll_register_callbacks cb 0x560d7d467090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:11.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d561b80 0x560d7d56c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d561b80 0x560d7d56c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58440/0 (socket says 172.21.15.32:58440) 2024-04-16T05:44:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 -- 172.21.15.32:0/567022694 learned_addr learned my addr 172.21.15.32:0/567022694 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677b7fe640 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d56f760 0x560d7d56d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f67727fc640 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d5624a0 0x560d7d56cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 -- 172.21.15.32:0/567022694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d5624a0 msgr2=0x560d7d56cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d5624a0 0x560d7d56cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 -- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d56f760 msgr2=0x560d7d56d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d56f760 0x560d7d56d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 -- 172.21.15.32:0/567022694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7d1d93f0 con 0x560d7d561b80 2024-04-16T05:44:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677b7fe640 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d56f760 0x560d7d56d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f677affd640 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d561b80 0x560d7d56c6f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f676c098730 tx=0x7f676c09bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddabc713fa8a3ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f67627fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f67627fc640 1 -- 172.21.15.32:0/567022694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c020050 con 0x560d7d561b80 2024-04-16T05:44:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f67627fc640 1 -- 172.21.15.32:0/567022694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f676c009cc0 con 0x560d7d561b80 2024-04-16T05:44:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f6784032740 1 -- 172.21.15.32:0/567022694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676402cca0 con 0x560d7d561b80 2024-04-16T05:44:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.907+0000 7f67627fc640 1 -- 172.21.15.32:0/567022694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f676c09c9d0 con 0x560d7d561b80 2024-04-16T05:44:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f6784032740 10 client.17727 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f6784032740 1 -- 172.21.15.32:0/567022694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d7d1990f0 con 0x560d7d561b80 2024-04-16T05:44:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f67627fc640 1 -- 172.21.15.32:0/567022694 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676402cca0 con 0x560d7d561b80 2024-04-16T05:44:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f67627fc640 1 -- 172.21.15.32:0/567022694 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f676c0cdd00 con 0x560d7d561b80 2024-04-16T05:44:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f6784032740 10 client.17727 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f6784032740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f6784032740 20 client.17727 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f6784032740 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560d7d57fab0 0x560d7d581ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f6784032740 4 client.17727 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f6784032740 1 -- 172.21.15.32:0/567022694 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d7d4d4ba0 con 0x560d7d57fab0 2024-04-16T05:44:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.908+0000 7f67727fc640 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560d7d57fab0 0x560d7d581ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.909+0000 7f67727fc640 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560d7d57fab0 0x560d7d581ea0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfd463bd228b85c server_cookie=618a325669b63b27 in_seq=0 out_seq=0 2024-04-16T05:44:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.909+0000 7f67627fc640 10 client.17727 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.909+0000 7f67627fc640 1 -- 172.21.15.32:0/567022694 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d7d4d4ba0 con 0x560d7d57fab0 2024-04-16T05:44:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.909+0000 7f67627fc640 10 client.17727 handle_command_reply: tid=0 2024-04-16T05:44:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.909+0000 7f6784032740 1 client.17727 shutdown 2024-04-16T05:44:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 -- 172.21.15.32:0/567022694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d561b80 msgr2=0x560d7d56c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d561b80 0x560d7d56c6f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f676c098730 tx=0x7f676c09bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 -- 172.21.15.32:0/567022694 shutdown_connections 2024-04-16T05:44:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d56f760 0x560d7d56d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560d7d57fab0 0x560d7d581ea0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d5624a0 0x560d7d56cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 --2- 172.21.15.32:0/567022694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d561b80 0x560d7d56c6f0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f67627fc640 0 client.17727 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 -- 172.21.15.32:0/567022694 >> 172.21.15.32:0/567022694 conn(0x560d7d55c580 msgr2=0x560d7d55e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:11.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 -- 172.21.15.32:0/567022694 shutdown_connections 2024-04-16T05:44:11.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 1 -- 172.21.15.32:0/567022694 wait complete. 2024-04-16T05:44:11.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.910+0000 7f6784032740 20 client.17727 trim_cache size 0 max 16384 2024-04-16T05:44:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.927+0000 7f6784032740 1 Processor -- start 2024-04-16T05:44:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.927+0000 7f6784032740 1 -- start start 2024-04-16T05:44:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f6784032740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d55c580 0x560d7d57b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f6784032740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d56cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f6784032740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f6784032740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1afb60 con 0x560d7d55c580 2024-04-16T05:44:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f6784032740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1ad430 con 0x560d7d5624a0 2024-04-16T05:44:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f6784032740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1ab2a0 con 0x560d7d561b80 2024-04-16T05:44:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d55c580 0x560d7d57b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d55c580 0x560d7d57b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58450/0 (socket says 172.21.15.32:58450) 2024-04-16T05:44:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 -- 172.21.15.32:0/3639601882 learned_addr learned my addr 172.21.15.32:0/3639601882 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f67727fc640 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d56cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 -- 172.21.15.32:0/3639601882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 msgr2=0x560d7d56cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677b7fe640 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d56cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 -- 172.21.15.32:0/3639601882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 msgr2=0x560d7d56d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:11.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 -- 172.21.15.32:0/3639601882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7d5622d0 con 0x560d7d55c580 2024-04-16T05:44:11.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f67727fc640 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d56cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:11.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677b7fe640 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:11.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f677affd640 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d55c580 0x560d7d57b420 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f676c012040 tx=0x7f676c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fd0528d4c7d10f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:11.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.928+0000 7f6770ff9640 1 -- 172.21.15.32:0/3639601882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c020070 con 0x560d7d55c580 2024-04-16T05:44:11.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6770ff9640 1 -- 172.21.15.32:0/3639601882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c0046c0 con 0x560d7d55c580 2024-04-16T05:44:11.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6770ff9640 1 -- 172.21.15.32:0/3639601882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c009cc0 con 0x560d7d55c580 2024-04-16T05:44:11.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 -- 172.21.15.32:0/3639601882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d55c580 msgr2=0x560d7d57b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d55c580 0x560d7d57b420 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f676c012040 tx=0x7f676c0048a0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 -- 172.21.15.32:0/3639601882 shutdown_connections 2024-04-16T05:44:11.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d5624a0 0x560d7d56d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d561b80 0x560d7d56cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 --2- 172.21.15.32:0/3639601882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d55c580 0x560d7d57b420 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 -- 172.21.15.32:0/3639601882 >> 172.21.15.32:0/3639601882 conn(0x560d7d56b330 msgr2=0x560d7d56c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:11.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 -- 172.21.15.32:0/3639601882 shutdown_connections 2024-04-16T05:44:11.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 -- 172.21.15.32:0/3639601882 wait complete. 2024-04-16T05:44:11.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.929+0000 7f6784032740 1 -- start start 2024-04-16T05:44:11.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f6784032740 1 Processor -- start 2024-04-16T05:44:11.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f6784032740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d55c580 0x560d7d575a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f6784032740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d561b80 0x560d7d575fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f6784032740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d5624a0 0x560d7d5e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f6784032740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1ab8f0 con 0x560d7d5624a0 2024-04-16T05:44:11.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f6784032740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1acd60 con 0x560d7d561b80 2024-04-16T05:44:11.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f6784032740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1abfc0 con 0x560d7d55c580 2024-04-16T05:44:11.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f6784032740 10 client.0 _ll_register_callbacks cb 0x7f676c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:11.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d5624a0 0x560d7d5e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d5624a0 0x560d7d5e4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58452/0 (socket says 172.21.15.32:58452) 2024-04-16T05:44:11.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 -- 172.21.15.32:0/4212854708 learned_addr learned my addr 172.21.15.32:0/4212854708 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:11.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677affd640 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d55c580 0x560d7d575a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f67727fc640 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d561b80 0x560d7d575fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 -- 172.21.15.32:0/4212854708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d55c580 msgr2=0x560d7d575a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d55c580 0x560d7d575a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 -- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d561b80 msgr2=0x560d7d575fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d561b80 0x560d7d575fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 -- 172.21.15.32:0/4212854708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7d5e5440 con 0x560d7d5624a0 2024-04-16T05:44:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f67727fc640 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d561b80 0x560d7d575fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:11.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f677b7fe640 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d5624a0 0x560d7d5e4f00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6774011bf0 tx=0x7f6774004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf2fe8c2c824818 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:11.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.930+0000 7f67627fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:11.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f67627fc640 1 -- 172.21.15.32:0/4212854708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774025070 con 0x560d7d5624a0 2024-04-16T05:44:11.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f67627fc640 1 -- 172.21.15.32:0/4212854708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f6774004b20 con 0x560d7d5624a0 2024-04-16T05:44:11.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 1 -- 172.21.15.32:0/4212854708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d7d4cea10 con 0x560d7d5624a0 2024-04-16T05:44:11.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f67627fc640 1 -- 172.21.15.32:0/4212854708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6774030050 con 0x560d7d5624a0 2024-04-16T05:44:11.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 10 client.17733 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:11.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 1 -- 172.21.15.32:0/4212854708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d7d5e6690 con 0x560d7d5624a0 2024-04-16T05:44:11.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f67627fc640 1 -- 172.21.15.32:0/4212854708 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560d7d4cea10 con 0x560d7d5624a0 2024-04-16T05:44:11.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f67627fc640 1 -- 172.21.15.32:0/4212854708 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6774021420 con 0x560d7d5624a0 2024-04-16T05:44:11.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 10 client.17733 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:11.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:11.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 20 client.17733 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:11.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560d7d56f760 0x560d7d5e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:11.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 4 client.17733 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:44:11.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.931+0000 7f6784032740 1 -- 172.21.15.32:0/4212854708 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d7d44a560 con 0x560d7d56f760 2024-04-16T05:44:11.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f677affd640 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560d7d56f760 0x560d7d5e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:11.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f677affd640 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560d7d56f760 0x560d7d5e8e80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51dae5fc2cf7c967 server_cookie=7eef400e9ef1fbc9 in_seq=0 out_seq=0 2024-04-16T05:44:11.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f67627fc640 10 client.17733 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f67627fc640 1 -- 172.21.15.32:0/4212854708 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560d7d44a560 con 0x560d7d56f760 2024-04-16T05:44:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f67627fc640 10 client.17733 handle_command_reply: tid=0 2024-04-16T05:44:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 client.17733 shutdown 2024-04-16T05:44:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 -- 172.21.15.32:0/4212854708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d5624a0 msgr2=0x560d7d5e4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d5624a0 0x560d7d5e4f00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6774011bf0 tx=0x7f6774004a40 comp rx=0 tx=0).stop 2024-04-16T05:44:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 -- 172.21.15.32:0/4212854708 shutdown_connections 2024-04-16T05:44:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560d7d5624a0 0x560d7d5e4f00 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d7d561b80 0x560d7d575fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x560d7d56f760 0x560d7d5e8e80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 --2- 172.21.15.32:0/4212854708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7d55c580 0x560d7d575a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 -- 172.21.15.32:0/4212854708 >> 172.21.15.32:0/4212854708 conn(0x560d7d56b330 msgr2=0x560d7d56c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f67627fc640 0 client.17733 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 -- 172.21.15.32:0/4212854708 shutdown_connections 2024-04-16T05:44:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 1 -- 172.21.15.32:0/4212854708 wait complete. 2024-04-16T05:44:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.932+0000 7f6784032740 20 client.17733 trim_cache size 0 max 16384 2024-04-16T05:44:11.957 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:44:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:44:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:44:11.958 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:44:11.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 -- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67680a3d20 msgr2=0x7f67680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67680a3d20 0x7f67680a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6774014750 tx=0x7f6774004ab0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 -- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 msgr2=0x7f677c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:11.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 0x7f677c19e620 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f676c00d770 tx=0x7f676c00dc30 comp rx=0 tx=0).stop 2024-04-16T05:44:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 -- 172.21.15.32:0/2020879822 shutdown_connections 2024-04-16T05:44:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c1070e0 0x7f677c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c101bf0 0x7f677c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f67680a3d20 0x7f67680a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 --2- 172.21.15.32:0/2020879822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677c101250 0x7f677c19e620 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 -- 172.21.15.32:0/2020879822 >> 172.21.15.32:0/2020879822 conn(0x7f677c076d00 msgr2=0x7f677c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:11.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 -- 172.21.15.32:0/2020879822 shutdown_connections 2024-04-16T05:44:11.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:11.933+0000 7f6781372640 1 -- 172.21.15.32:0/2020879822 wait complete. 2024-04-16T05:44:11.962 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:44:11.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:44:12.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 -- 172.21.15.32:0/2630890101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec101140 msgr2=0x7fdeec101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2630890101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec101140 0x7fdeec101520 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009900 tx=0x7fdee802bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:12.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 -- 172.21.15.32:0/2630890101 shutdown_connections 2024-04-16T05:44:12.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2630890101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeec106210 0x7fdeec10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2630890101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeec101af0 0x7fdeec105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2630890101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec101140 0x7fdeec101520 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 -- 172.21.15.32:0/2630890101 >> 172.21.15.32:0/2630890101 conn(0x7fdeec077280 msgr2=0x7fdeec077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 -- 172.21.15.32:0/2630890101 shutdown_connections 2024-04-16T05:44:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.070+0000 7fdef2b59640 1 -- 172.21.15.32:0/2630890101 wait complete. 2024-04-16T05:44:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef2b59640 1 Processor -- start 2024-04-16T05:44:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef2b59640 1 -- start start 2024-04-16T05:44:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef2b59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeec101140 0x7fdeec19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef2b59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeec106210 0x7fdeec19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef2b59640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec198640 0x7fdeec198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef2b59640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec1123e0 con 0x7fdeec198640 2024-04-16T05:44:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef2b59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec112260 con 0x7fdeec106210 2024-04-16T05:44:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef2b59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec112560 con 0x7fdeec101140 2024-04-16T05:44:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec198640 0x7fdeec198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec198640 0x7fdeec198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58466/0 (socket says 172.21.15.32:58466) 2024-04-16T05:44:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 -- 172.21.15.32:0/2440811133 learned_addr learned my addr 172.21.15.32:0/2440811133 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef08f5640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeec101140 0x7fdeec19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdee3fff640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeec106210 0x7fdeec19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 -- 172.21.15.32:0/2440811133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeec101140 msgr2=0x7fdeec19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeec101140 0x7fdeec19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 -- 172.21.15.32:0/2440811133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeec106210 msgr2=0x7fdeec19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeec106210 0x7fdeec19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 -- 172.21.15.32:0/2440811133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec199240 con 0x7fdeec198640 2024-04-16T05:44:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdee3fff640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeec106210 0x7fdeec19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.071+0000 7fdef10f6640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec198640 0x7fdeec198ac0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdee400eb10 tx=0x7fdee400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.072+0000 7fdee1ffb640 1 -- 172.21.15.32:0/2440811133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4004b20 con 0x7fdeec198640 2024-04-16T05:44:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.072+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec1a5320 con 0x7fdeec198640 2024-04-16T05:44:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.072+0000 7fdee1ffb640 1 -- 172.21.15.32:0/2440811133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee4004cc0 con 0x7fdeec198640 2024-04-16T05:44:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.072+0000 7fdee1ffb640 1 -- 172.21.15.32:0/2440811133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4010760 con 0x7fdeec198640 2024-04-16T05:44:12.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.072+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec1a56d0 con 0x7fdeec198640 2024-04-16T05:44:12.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.073+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeec102610 con 0x7fdeec198640 2024-04-16T05:44:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.074+0000 7fdee1ffb640 1 -- 172.21.15.32:0/2440811133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdee4005020 con 0x7fdeec198640 2024-04-16T05:44:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.074+0000 7fdee1ffb640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fded40a3d20 0x7fded40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.075+0000 7fdef08f5640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fded40a3d20 0x7fded40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.075+0000 7fdee1ffb640 1 -- 172.21.15.32:0/2440811133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7fdee40d1260 con 0x7fdeec198640 2024-04-16T05:44:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.075+0000 7fdef08f5640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fded40a3d20 0x7fded40a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdee8006010 tx=0x7fdee8000bc0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.079+0000 7fdee1ffb640 1 -- 172.21.15.32:0/2440811133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdee409d510 con 0x7fdeec198640 2024-04-16T05:44:12.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.305+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdeec069270 con 0x7fdeec198640 2024-04-16T05:44:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.306+0000 7fdee1ffb640 1 -- 172.21.15.32:0/2440811133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7fdee409ce10 con 0x7fdeec198640 2024-04-16T05:44:12.304 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:44:12.305 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:44:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:44:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fded40a3d20 msgr2=0x7fded40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fded40a3d20 0x7fded40a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdee8006010 tx=0x7fdee8000bc0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec198640 msgr2=0x7fdeec198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec198640 0x7fdeec198ac0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdee400eb10 tx=0x7fdee400c2d0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 shutdown_connections 2024-04-16T05:44:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fded40a3d20 0x7fded40a61e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeec198640 0x7fdeec198ac0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeec106210 0x7fdeec19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 --2- 172.21.15.32:0/2440811133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeec101140 0x7fdeec19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 >> 172.21.15.32:0/2440811133 conn(0x7fdeec077280 msgr2=0x7fdeec0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 shutdown_connections 2024-04-16T05:44:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.308+0000 7fdef2b59640 1 -- 172.21.15.32:0/2440811133 wait complete. 2024-04-16T05:44:12.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:44:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/1761780385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 msgr2=0x7fd7bc10eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/1761780385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 0x7fd7bc10eef0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009ca0 tx=0x7fd7b402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/1761780385 shutdown_connections 2024-04-16T05:44:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/1761780385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 0x7fd7bc10eef0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/1761780385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7bc102c00 0x7fd7bc106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/1761780385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7bc102250 0x7fd7bc102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/1761780385 >> 172.21.15.32:0/1761780385 conn(0x7fd7bc079490 msgr2=0x7fd7bc0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/1761780385 shutdown_connections 2024-04-16T05:44:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.427+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/1761780385 wait complete. 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c2b7c640 1 Processor -- start 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c2b7c640 1 -- start start 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c2b7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7bc102250 0x7fd7bc19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c2b7c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 0x7fd7bc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c2b7c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7bc1986a0 0x7fd7bc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c2b7c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc1123e0 con 0x7fd7bc107320 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c2b7c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc112260 con 0x7fd7bc1986a0 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c2b7c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc112560 con 0x7fd7bc102250 2024-04-16T05:44:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 0x7fd7bc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 0x7fd7bc19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58478/0 (socket says 172.21.15.32:58478) 2024-04-16T05:44:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 -- 172.21.15.32:0/285950643 learned_addr learned my addr 172.21.15.32:0/285950643 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c0918640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7bc102250 0x7fd7bc19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c1119640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7bc1986a0 0x7fd7bc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 -- 172.21.15.32:0/285950643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7bc102250 msgr2=0x7fd7bc19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7bc102250 0x7fd7bc19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 -- 172.21.15.32:0/285950643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7bc1986a0 msgr2=0x7fd7bc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7bc1986a0 0x7fd7bc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 -- 172.21.15.32:0/285950643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc1992a0 con 0x7fd7bc107320 2024-04-16T05:44:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c1119640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7bc1986a0 0x7fd7bc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7c0918640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7bc102250 0x7fd7bc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.428+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 0x7fd7bc19eb20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00c970 tx=0x7fd7ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.429+0000 7fd7b1ffb640 1 -- 172.21.15.32:0/285950643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac007bb0 con 0x7fd7bc107320 2024-04-16T05:44:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.429+0000 7fd7b1ffb640 1 -- 172.21.15.32:0/285950643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ac007d50 con 0x7fd7bc107320 2024-04-16T05:44:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.429+0000 7fd7b1ffb640 1 -- 172.21.15.32:0/285950643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00f450 con 0x7fd7bc107320 2024-04-16T05:44:12.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.429+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/285950643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7bc1a5380 con 0x7fd7bc107320 2024-04-16T05:44:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.429+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/285950643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7bc1a57e0 con 0x7fd7bc107320 2024-04-16T05:44:12.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7b1ffb640 1 -- 172.21.15.32:0/285950643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd7ac0046c0 con 0x7fd7bc107320 2024-04-16T05:44:12.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c5766740 1 Processor -- start 2024-04-16T05:44:12.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c5766740 1 -- start start 2024-04-16T05:44:12.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0326e60 0x55bfc0327240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7b1ffb640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd7a40a3da0 0x7fd7a40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0327780 0x55bfc032fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0332750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c5766740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bfc028c0e0 con 0x55bfc0326e60 2024-04-16T05:44:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c5766740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bfc0229520 con 0x55bfc0327780 2024-04-16T05:44:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c5766740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bfc02880a0 con 0x55bfc0330360 2024-04-16T05:44:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c0918640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd7a40a3da0 0x7fd7a40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c1119640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0326e60 0x55bfc0327240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c1119640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0326e60 0x55bfc0327240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58480/0 (socket says 172.21.15.32:58480) 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7c1119640 1 -- 172.21.15.32:0/1027556016 learned_addr learned my addr 172.21.15.32:0/1027556016 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.432+0000 7fd7b1ffb640 1 -- 172.21.15.32:0/285950643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7fd7ac0d0be0 con 0x7fd7bc107320 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/1027556016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0327780 0x55bfc032fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7c1119640 1 -- 172.21.15.32:0/1027556016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 msgr2=0x55bfc0332750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7c1119640 1 --2- 172.21.15.32:0/1027556016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0332750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7c1119640 1 -- 172.21.15.32:0/1027556016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0327780 msgr2=0x55bfc032fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7c1119640 1 --2- 172.21.15.32:0/1027556016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0327780 0x55bfc032fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7c1119640 1 -- 172.21.15.32:0/1027556016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc02518d0 con 0x55bfc0326e60 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7c0918640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd7a40a3da0 0x7fd7a40a6260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80058c0 tx=0x7fd7b8005850 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7c1119640 1 --2- 172.21.15.32:0/1027556016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0326e60 0x55bfc0327240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4005fe0 tx=0x7fd7b40098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6073a4a3114ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7a2ffd640 1 -- 172.21.15.32:0/1027556016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4045070 con 0x55bfc0326e60 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7a2ffd640 1 -- 172.21.15.32:0/1027556016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7b4003970 con 0x55bfc0326e60 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.433+0000 7fd7a2ffd640 1 -- 172.21.15.32:0/1027556016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4039040 con 0x55bfc0326e60 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 -- 172.21.15.32:0/1027556016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0326e60 msgr2=0x55bfc0327240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1027556016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0326e60 0x55bfc0327240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4005fe0 tx=0x7fd7b40098d0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 -- 172.21.15.32:0/1027556016 shutdown_connections 2024-04-16T05:44:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1027556016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0332750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1027556016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0327780 0x55bfc032fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1027556016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0326e60 0x55bfc0327240 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 -- 172.21.15.32:0/1027556016 >> 172.21.15.32:0/1027556016 conn(0x55bfc0321590 msgr2=0x55bfc0321970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 -- 172.21.15.32:0/1027556016 shutdown_connections 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 -- 172.21.15.32:0/1027556016 wait complete. 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 -- start start 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.434+0000 7fd7c5766740 1 Processor -- start 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc032d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc032de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bfc0333000 con 0x55bfc0327780 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bfc02454d0 con 0x55bfc0326e60 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bfc0333180 con 0x55bfc0330360 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 10 client.0 _ll_register_callbacks cb 0x55bfc022a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc032d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc032d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58484/0 (socket says 172.21.15.32:58484) 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 -- 172.21.15.32:0/1691185039 learned_addr learned my addr 172.21.15.32:0/1691185039 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c1119640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc032de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c0918640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 -- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 msgr2=0x55bfc032de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc032de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 -- 172.21.15.32:0/1691185039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 msgr2=0x55bfc032d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 -- 172.21.15.32:0/1691185039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfbff9e3f0 con 0x55bfc0327780 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c0918640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c1119640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc032de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc032d8c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac098bf0 tx=0x7fd7ac09c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729768801a3f72c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd79bfff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd79bfff640 1 -- 172.21.15.32:0/1691185039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac01d030 con 0x55bfc0327780 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd79bfff640 1 -- 172.21.15.32:0/1691185039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7fd7ac09aa90 con 0x55bfc0327780 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 1 -- 172.21.15.32:0/1691185039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b4003970 con 0x55bfc0327780 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd79bfff640 1 -- 172.21.15.32:0/1691185039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7ac0ba050 con 0x55bfc0327780 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 10 client.17751 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.435+0000 7fd7c5766740 1 -- 172.21.15.32:0/1691185039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfc00687d0 con 0x55bfc0327780 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd79bfff640 1 -- 172.21.15.32:0/1691185039 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7b4003970 con 0x55bfc0327780 2024-04-16T05:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd79bfff640 1 -- 172.21.15.32:0/1691185039 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd7ac098e00 con 0x55bfc0327780 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd7c5766740 10 client.17751 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd7c5766740 10 client.17751 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd7c5766740 20 client.17751 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55bfc0340260 0x55bfc03426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd7c5766740 4 client.17751 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd7c5766740 1 -- 172.21.15.32:0/1691185039 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfc0299b00 con 0x55bfc0340260 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.436+0000 7fd7c0918640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55bfc0340260 0x55bfc03426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd79bfff640 10 client.17751 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:44:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c0918640 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55bfc0340260 0x55bfc03426b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de3fb1e8d6771cd0 server_cookie=4c824641479ce6dd in_seq=0 out_seq=0 2024-04-16T05:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd79bfff640 1 -- 172.21.15.32:0/1691185039 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfc0299b00 con 0x55bfc0340260 2024-04-16T05:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd79bfff640 10 client.17751 handle_command_reply: tid=0 2024-04-16T05:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 client.17751 shutdown 2024-04-16T05:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 -- 172.21.15.32:0/1691185039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 msgr2=0x55bfc032d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc032d8c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac098bf0 tx=0x7fd7ac09c400 comp rx=0 tx=0).stop 2024-04-16T05:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 -- 172.21.15.32:0/1691185039 shutdown_connections 2024-04-16T05:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55bfc0340260 0x55bfc03426b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc032de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd79bfff640 0 client.17751 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc032d8c0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1691185039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.437+0000 7fd7c5766740 1 -- 172.21.15.32:0/1691185039 >> 172.21.15.32:0/1691185039 conn(0x55bfc0321590 msgr2=0x55bfc03232e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.438+0000 7fd7c5766740 1 -- 172.21.15.32:0/1691185039 shutdown_connections 2024-04-16T05:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.438+0000 7fd7c5766740 1 -- 172.21.15.32:0/1691185039 wait complete. 2024-04-16T05:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.438+0000 7fd7c5766740 20 client.17751 trim_cache size 0 max 16384 2024-04-16T05:44:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c5766740 1 Processor -- start 2024-04-16T05:44:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c5766740 1 -- start start 2024-04-16T05:44:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc033bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0347f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c5766740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc1b1d40 con 0x55bfc0327780 2024-04-16T05:44:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c5766740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc1af610 con 0x55bfc0326e60 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c5766740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc1ad480 con 0x55bfc0330360 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc033bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc033bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58488/0 (socket says 172.21.15.32:58488) 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7b3fff640 1 -- 172.21.15.32:0/4189601023 learned_addr learned my addr 172.21.15.32:0/4189601023 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7c1119640 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0347f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.451+0000 7fd7b3fff640 1 -- 172.21.15.32:0/4189601023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 msgr2=0x55bfc0347f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c0918640 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0347f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7b3fff640 1 -- 172.21.15.32:0/4189601023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 msgr2=0x55bfc032fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7b3fff640 1 -- 172.21.15.32:0/4189601023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc0332750 con 0x55bfc0327780 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c1119640 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0347f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c0918640 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc033bfd0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac000c00 tx=0x7fd7ac087050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476f08c3f7962487 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7a27fc640 1 -- 172.21.15.32:0/4189601023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac01d070 con 0x55bfc0327780 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7a27fc640 1 -- 172.21.15.32:0/4189601023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ac080020 con 0x55bfc0327780 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7a27fc640 1 -- 172.21.15.32:0/4189601023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac098fa0 con 0x55bfc0327780 2024-04-16T05:44:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c5766740 1 -- 172.21.15.32:0/4189601023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 msgr2=0x55bfc033bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c5766740 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc033bfd0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac000c00 tx=0x7fd7ac087050 comp rx=0 tx=0).stop 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c5766740 1 -- 172.21.15.32:0/4189601023 shutdown_connections 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c5766740 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0347f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c5766740 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc033bfd0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.452+0000 7fd7c5766740 1 --2- 172.21.15.32:0/4189601023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc032fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 -- 172.21.15.32:0/4189601023 >> 172.21.15.32:0/4189601023 conn(0x55bfc0321590 msgr2=0x55bfc031c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 -- 172.21.15.32:0/4189601023 shutdown_connections 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 -- 172.21.15.32:0/4189601023 wait complete. 2024-04-16T05:44:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 -- start start 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 Processor -- start 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc03385b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc0333ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0334000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc1adad0 con 0x55bfc0327780 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc1aef40 con 0x55bfc0326e60 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc1ae1a0 con 0x55bfc0330360 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7c5766740 10 client.0 _ll_register_callbacks cb 0x7fd7ac007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc0333ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc0333ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58498/0 (socket says 172.21.15.32:58498) 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.453+0000 7fd7b3fff640 1 -- 172.21.15.32:0/1686401626 learned_addr learned my addr 172.21.15.32:0/1686401626 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c0918640 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc03385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c1119640 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0334000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7b3fff640 1 -- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 msgr2=0x55bfc0334000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0334000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7b3fff640 1 -- 172.21.15.32:0/1686401626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 msgr2=0x55bfc03385b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc03385b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7b3fff640 1 -- 172.21.15.32:0/1686401626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc0334540 con 0x55bfc0327780 2024-04-16T05:44:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c1119640 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0334000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc0333ac0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac09c490 tx=0x7fd7ac00a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487f93e63d90c372 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd79bfff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd79bfff640 1 -- 172.21.15.32:0/1686401626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac01d070 con 0x55bfc0327780 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd79bfff640 1 -- 172.21.15.32:0/1686401626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7fd7ac09bbe0 con 0x55bfc0327780 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 1 -- 172.21.15.32:0/1686401626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7ac080020 con 0x55bfc0327780 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd79bfff640 1 -- 172.21.15.32:0/1686401626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7ac09c790 con 0x55bfc0327780 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 10 client.17757 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 1 -- 172.21.15.32:0/1686401626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfc03a6490 con 0x55bfc0327780 2024-04-16T05:44:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd79bfff640 1 -- 172.21.15.32:0/1686401626 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ac080020 con 0x55bfc0327780 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd79bfff640 1 -- 172.21.15.32:0/1686401626 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd7ac09b650 con 0x55bfc0327780 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 10 client.17757 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 10 client.17757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 20 client.17757 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55bfc03a6820 0x55bfc03a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 4 client.17757 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.454+0000 7fd7c5766740 1 -- 172.21.15.32:0/1686401626 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfc0293970 con 0x55bfc03a6820 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.455+0000 7fd7c0918640 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55bfc03a6820 0x55bfc03a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.455+0000 7fd7c0918640 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55bfc03a6820 0x55bfc03a8c40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=889a6b5c01957442 server_cookie=3557110d78e5906a in_seq=0 out_seq=0 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.455+0000 7fd79bfff640 10 client.17757 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.455+0000 7fd79bfff640 1 -- 172.21.15.32:0/1686401626 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfc0293970 con 0x55bfc03a6820 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.455+0000 7fd79bfff640 10 client.17757 handle_command_reply: tid=0 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.455+0000 7fd7c5766740 1 client.17757 shutdown 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 -- 172.21.15.32:0/1686401626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 msgr2=0x55bfc0333ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc0333ac0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac09c490 tx=0x7fd7ac00a4a0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 -- 172.21.15.32:0/1686401626 shutdown_connections 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55bfc03a6820 0x55bfc03a8c40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfc0330360 0x55bfc0334000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd79bfff640 0 client.17757 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:44:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bfc0327780 0x55bfc0333ac0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 --2- 172.21.15.32:0/1686401626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfc0326e60 0x55bfc03385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 -- 172.21.15.32:0/1686401626 >> 172.21.15.32:0/1686401626 conn(0x55bfc0321590 msgr2=0x55bfc03229c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 -- 172.21.15.32:0/1686401626 shutdown_connections 2024-04-16T05:44:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 1 -- 172.21.15.32:0/1686401626 wait complete. 2024-04-16T05:44:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c5766740 20 client.17757 trim_cache size 0 max 16384 2024-04-16T05:44:12.464 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:44:12.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd7a40a3da0 msgr2=0x7fd7a40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd7a40a3da0 0x7fd7a40a6260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80058c0 tx=0x7fd7b8005850 comp rx=0 tx=0).stop 2024-04-16T05:44:12.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 msgr2=0x7fd7bc19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.456+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 0x7fd7bc19eb20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00c970 tx=0x7fd7ac00ce30 comp rx=0 tx=0).stop 2024-04-16T05:44:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.457+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/285950643 shutdown_connections 2024-04-16T05:44:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.457+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7bc1986a0 0x7fd7bc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.457+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd7a40a3da0 0x7fd7a40a6260 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.457+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7bc107320 0x7fd7bc19eb20 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.457+0000 7fd7c2b7c640 1 --2- 172.21.15.32:0/285950643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7bc102250 0x7fd7bc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.457+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/285950643 >> 172.21.15.32:0/285950643 conn(0x7fd7bc079490 msgr2=0x7fd7bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.457+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/285950643 shutdown_connections 2024-04-16T05:44:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.457+0000 7fd7c2b7c640 1 -- 172.21.15.32:0/285950643 wait complete. 2024-04-16T05:44:12.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 -- 172.21.15.32:0/586972492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0107320 msgr2=0x7f29f010eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/586972492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0107320 0x7f29f010eef0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009ca0 tx=0x7f29e802c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 -- 172.21.15.32:0/586972492 shutdown_connections 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/586972492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0107320 0x7f29f010eef0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/586972492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29f0102c00 0x7f29f0106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/586972492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29f0102250 0x7f29f0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 -- 172.21.15.32:0/586972492 >> 172.21.15.32:0/586972492 conn(0x7f29f0079490 msgr2=0x7f29f00798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 -- 172.21.15.32:0/586972492 shutdown_connections 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.576+0000 7f29f7f1b640 1 -- 172.21.15.32:0/586972492 wait complete. 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f7f1b640 1 Processor -- start 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f7f1b640 1 -- start start 2024-04-16T05:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f7f1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29f0102250 0x7f29f019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f7f1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29f0107320 0x7f29f019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f7f1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0198660 0x7f29f0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f7f1b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01123e0 con 0x7f29f0198660 2024-04-16T05:44:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f7f1b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0112260 con 0x7f29f0107320 2024-04-16T05:44:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f7f1b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0112560 con 0x7f29f0102250 2024-04-16T05:44:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f64b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0198660 0x7f29f0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f64b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0198660 0x7f29f0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58522/0 (socket says 172.21.15.32:58522) 2024-04-16T05:44:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.577+0000 7f29f64b8640 1 -- 172.21.15.32:0/818681631 learned_addr learned my addr 172.21.15.32:0/818681631 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29f0102250 0x7f29f019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f54b6640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29f0107320 0x7f29f019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f64b8640 1 -- 172.21.15.32:0/818681631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29f0102250 msgr2=0x7f29f019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f64b8640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29f0102250 0x7f29f019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f64b8640 1 -- 172.21.15.32:0/818681631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29f0107320 msgr2=0x7f29f019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f64b8640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29f0107320 0x7f29f019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f64b8640 1 -- 172.21.15.32:0/818681631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f0199260 con 0x7f29f0198660 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29f0102250 0x7f29f019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f64b8640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0198660 0x7f29f0198ae0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f29e802c700 tx=0x7f29e8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29e6ffd640 1 -- 172.21.15.32:0/818681631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8041020 con 0x7f29f0198660 2024-04-16T05:44:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29e6ffd640 1 -- 172.21.15.32:0/818681631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e8034cd0 con 0x7f29f0198660 2024-04-16T05:44:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f7f1b640 1 -- 172.21.15.32:0/818681631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f01994f0 con 0x7f29f0198660 2024-04-16T05:44:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29e6ffd640 1 -- 172.21.15.32:0/818681631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8039040 con 0x7f29f0198660 2024-04-16T05:44:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.578+0000 7f29f7f1b640 1 -- 172.21.15.32:0/818681631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f01a5630 con 0x7f29f0198660 2024-04-16T05:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.580+0000 7f29e6ffd640 1 -- 172.21.15.32:0/818681631 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29e8044020 con 0x7f29f0198660 2024-04-16T05:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29fabd7740 1 Processor -- start 2024-04-16T05:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29fabd7740 1 -- start start 2024-04-16T05:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 0x5626c8897cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c8898700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29fabd7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5626c87fd0e0 con 0x5626c88a9a90 2024-04-16T05:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29fabd7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5626c879a520 con 0x5626c8898280 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29fabd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5626c87f90a0 con 0x5626c88978d0 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29e6ffd640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29dc0a3d20 0x7f29dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29f54b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29f64b8640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29dc0a3d20 0x7f29dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29f54b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58526/0 (socket says 172.21.15.32:58526) 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29f54b6640 1 -- 172.21.15.32:0/1199676521 learned_addr learned my addr 172.21.15.32:0/1199676521 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29f54b6640 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 0x5626c8897cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29e6ffd640 1 -- 172.21.15.32:0/818681631 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f29e80f2a40 con 0x7f29f0198660 2024-04-16T05:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.581+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c8898700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29f54b6640 1 -- 172.21.15.32:0/1199676521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 msgr2=0x5626c8897cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29f54b6640 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 0x5626c8897cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29f54b6640 1 -- 172.21.15.32:0/1199676521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 msgr2=0x5626c8898700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29f54b6640 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c8898700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29f54b6640 1 -- 172.21.15.32:0/1199676521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c87c28d0 con 0x5626c88a9a90 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29f64b8640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29dc0a3d20 0x7f29dc0a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f29e80efeb0 tx=0x7f29e808d050 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29f54b6640 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88abe80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f29e0014790 tx=0x7f29e0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac20a9ec7872e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29dbfff640 1 -- 172.21.15.32:0/1199676521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e00140d0 con 0x5626c88a9a90 2024-04-16T05:44:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29dbfff640 1 -- 172.21.15.32:0/1199676521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e0014270 con 0x5626c88a9a90 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.582+0000 7f29dbfff640 1 -- 172.21.15.32:0/1199676521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e00047a0 con 0x5626c88a9a90 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 -- 172.21.15.32:0/1199676521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 msgr2=0x5626c88abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88abe80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f29e0014790 tx=0x7f29e0009650 comp rx=0 tx=0).stop 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 -- 172.21.15.32:0/1199676521 shutdown_connections 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88abe80 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c8898700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 --2- 172.21.15.32:0/1199676521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 0x5626c8897cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 -- 172.21.15.32:0/1199676521 >> 172.21.15.32:0/1199676521 conn(0x5626c8892590 msgr2=0x5626c8892970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 -- 172.21.15.32:0/1199676521 shutdown_connections 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 -- 172.21.15.32:0/1199676521 wait complete. 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 -- start start 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.583+0000 7f29fabd7740 1 Processor -- start 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c8899e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c889a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c88a9a90 0x5626c889a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29fabd7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5626c88ac8e0 con 0x5626c88978d0 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29fabd7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5626c87b64d0 con 0x5626c88a9a90 2024-04-16T05:44:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29fabd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5626c88aca60 con 0x5626c8898280 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29fabd7740 10 client.0 _ll_register_callbacks cb 0x5626c879b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c8899e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c8899e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58542/0 (socket says 172.21.15.32:58542) 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 -- 172.21.15.32:0/766978848 learned_addr learned my addr 172.21.15.32:0/766978848 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c889a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 -- 172.21.15.32:0/766978848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 msgr2=0x5626c889a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c889a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 -- 172.21.15.32:0/766978848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c88a9a90 msgr2=0x5626c889a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c88a9a90 0x5626c889a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 -- 172.21.15.32:0/766978848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c850f3f0 con 0x5626c88978d0 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29f54b6640 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c8899e80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f29e000a010 tx=0x7f29e0009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e9ad9add659bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29d97fa640 1 -- 172.21.15.32:0/766978848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0026030 con 0x5626c88978d0 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29d97fa640 1 -- 172.21.15.32:0/766978848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f29e000fde0 con 0x5626c88978d0 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.584+0000 7f29fabd7740 1 -- 172.21.15.32:0/766978848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e0014270 con 0x5626c88978d0 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29d97fa640 1 -- 172.21.15.32:0/766978848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29e0014420 con 0x5626c88978d0 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29fabd7740 10 client.17769 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29fabd7740 1 -- 172.21.15.32:0/766978848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5626c85d97d0 con 0x5626c88978d0 2024-04-16T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29d97fa640 1 -- 172.21.15.32:0/766978848 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e0014270 con 0x5626c88978d0 2024-04-16T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29d97fa640 1 -- 172.21.15.32:0/766978848 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f29e00046c0 con 0x5626c88978d0 2024-04-16T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29fabd7740 10 client.17769 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29fabd7740 10 client.17769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29fabd7740 20 client.17769 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29fabd7740 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5626c88b9a10 0x5626c88bbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29fabd7740 4 client.17769 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29fabd7740 1 -- 172.21.15.32:0/766978848 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626c880ab00 con 0x5626c88b9a10 2024-04-16T05:44:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.585+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5626c88b9a10 0x5626c88bbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.586+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5626c88b9a10 0x5626c88bbe00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24c031f2791e3098 server_cookie=5ccf48e7e63b30dc in_seq=0 out_seq=0 2024-04-16T05:44:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.586+0000 7f29d97fa640 10 client.17769 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:44:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.586+0000 7f29d97fa640 1 -- 172.21.15.32:0/766978848 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626c880ab00 con 0x5626c88b9a10 2024-04-16T05:44:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.586+0000 7f29d97fa640 10 client.17769 handle_command_reply: tid=0 2024-04-16T05:44:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.586+0000 7f29fabd7740 1 client.17769 shutdown 2024-04-16T05:44:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.586+0000 7f29fabd7740 1 -- 172.21.15.32:0/766978848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 msgr2=0x5626c8899e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c8899e80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f29e000a010 tx=0x7f29e0009a40 comp rx=0 tx=0).stop 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 -- 172.21.15.32:0/766978848 shutdown_connections 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5626c88b9a10 0x5626c88bbe00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c88a9a90 0x5626c889a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c889a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 --2- 172.21.15.32:0/766978848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c8899e80 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29d97fa640 0 client.17769 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 -- 172.21.15.32:0/766978848 >> 172.21.15.32:0/766978848 conn(0x5626c8892590 msgr2=0x5626c88942e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 -- 172.21.15.32:0/766978848 shutdown_connections 2024-04-16T05:44:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 1 -- 172.21.15.32:0/766978848 wait complete. 2024-04-16T05:44:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.587+0000 7f29fabd7740 20 client.17769 trim_cache size 0 max 16384 2024-04-16T05:44:12.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29fabd7740 1 Processor -- start 2024-04-16T05:44:12.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29fabd7740 1 -- start start 2024-04-16T05:44:12.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 0x5626c88b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c88c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29fabd7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01afb90 con 0x5626c88a9a90 2024-04-16T05:44:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29fabd7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01ad460 con 0x5626c8898280 2024-04-16T05:44:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29fabd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01ab2d0 con 0x5626c88978d0 2024-04-16T05:44:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29f54b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29f54b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88c1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58550/0 (socket says 172.21.15.32:58550) 2024-04-16T05:44:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29f54b6640 1 -- 172.21.15.32:0/104377953 learned_addr learned my addr 172.21.15.32:0/104377953 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.600+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c88c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29f54b6640 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 0x5626c88b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29f54b6640 1 -- 172.21.15.32:0/104377953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 msgr2=0x5626c88b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29f54b6640 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 0x5626c88b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29f54b6640 1 -- 172.21.15.32:0/104377953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 msgr2=0x5626c88c13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29f54b6640 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c88c13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29f54b6640 1 -- 172.21.15.32:0/104377953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c8896f50 con 0x5626c88a9a90 2024-04-16T05:44:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c88c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29f54b6640 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88c1910 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f29e000fb30 tx=0x7f29e0009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bec4a3afc4b0af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29db7fe640 1 -- 172.21.15.32:0/104377953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0026030 con 0x5626c88a9a90 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29db7fe640 1 -- 172.21.15.32:0/104377953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e0009530 con 0x5626c88a9a90 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29db7fe640 1 -- 172.21.15.32:0/104377953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0004640 con 0x5626c88a9a90 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29fabd7740 1 -- 172.21.15.32:0/104377953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 msgr2=0x5626c88c1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29fabd7740 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88c1910 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f29e000fb30 tx=0x7f29e0009ab0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29fabd7740 1 -- 172.21.15.32:0/104377953 shutdown_connections 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29fabd7740 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88a9a90 0x5626c88c1910 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29fabd7740 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c8898280 0x5626c88c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29fabd7740 1 --2- 172.21.15.32:0/104377953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c88978d0 0x5626c88b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29fabd7740 1 -- 172.21.15.32:0/104377953 >> 172.21.15.32:0/104377953 conn(0x5626c8892590 msgr2=0x5626c88a9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.601+0000 7f29fabd7740 1 -- 172.21.15.32:0/104377953 shutdown_connections 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 -- 172.21.15.32:0/104377953 wait complete. 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 -- start start 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 Processor -- start 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c88aa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c88aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c88a9a90 0x5626c88ab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01ab920 con 0x5626c88978d0 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01acd90 con 0x5626c88a9a90 2024-04-16T05:44:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01abff0 con 0x5626c8898280 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29fabd7740 10 client.0 _ll_register_callbacks cb 0x7f29ec00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29f54b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c88aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29f54b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c88aa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58560/0 (socket says 172.21.15.32:58560) 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.602+0000 7f29f54b6640 1 -- 172.21.15.32:0/2257043358 learned_addr learned my addr 172.21.15.32:0/2257043358 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c88aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29f54b6640 1 -- 172.21.15.32:0/2257043358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 msgr2=0x5626c88aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29f54b6640 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c88aacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29f54b6640 1 -- 172.21.15.32:0/2257043358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c88a9a90 msgr2=0x5626c88ab200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29f54b6640 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c88a9a90 0x5626c88ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29f54b6640 1 -- 172.21.15.32:0/2257043358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c88aff90 con 0x5626c88978d0 2024-04-16T05:44:12.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c88aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:12.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29f54b6640 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c88aa780 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f29e0004c00 tx=0x7f29e00099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28060b502989e17a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:12.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29d97fa640 1 -- 172.21.15.32:0/2257043358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0026040 con 0x5626c88978d0 2024-04-16T05:44:12.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29d97fa640 1 -- 172.21.15.32:0/2257043358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f29e0004640 con 0x5626c88978d0 2024-04-16T05:44:12.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29fabd7740 1 -- 172.21.15.32:0/2257043358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e0009530 con 0x5626c88978d0 2024-04-16T05:44:12.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29d97fa640 1 -- 172.21.15.32:0/2257043358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29e0004860 con 0x5626c88978d0 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29fabd7740 10 client.17775 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29fabd7740 1 -- 172.21.15.32:0/2257043358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5626c891ff50 con 0x5626c88978d0 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.603+0000 7f29d97fa640 1 -- 172.21.15.32:0/2257043358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e0009530 con 0x5626c88978d0 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29d97fa640 1 -- 172.21.15.32:0/2257043358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f29e0012570 con 0x5626c88978d0 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 10 client.17775 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 10 client.17775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 20 client.17775 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5626c8920470 0x5626c8922860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 4 client.17775 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 1 -- 172.21.15.32:0/2257043358 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626c8804970 con 0x5626c8920470 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5626c8920470 0x5626c8922860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29d97fa640 10 client.17775 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29f5cb7640 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5626c8920470 0x5626c8922860 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a94f5341088b0ea6 server_cookie=dc9e77f25d333663 in_seq=0 out_seq=0 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29d97fa640 1 -- 172.21.15.32:0/2257043358 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626c8804970 con 0x5626c8920470 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29d97fa640 10 client.17775 handle_command_reply: tid=0 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 1 client.17775 shutdown 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 1 -- 172.21.15.32:0/2257043358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 msgr2=0x5626c88aa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.604+0000 7f29fabd7740 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c88aa780 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f29e0004c00 tx=0x7f29e00099f0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 1 -- 172.21.15.32:0/2257043358 shutdown_connections 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5626c8920470 0x5626c8922860 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626c88a9a90 0x5626c88ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c8898280 0x5626c88aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 1 --2- 172.21.15.32:0/2257043358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626c88978d0 0x5626c88aa780 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29d97fa640 0 client.17775 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 1 -- 172.21.15.32:0/2257043358 >> 172.21.15.32:0/2257043358 conn(0x5626c8892590 msgr2=0x5626c8893960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 1 -- 172.21.15.32:0/2257043358 shutdown_connections 2024-04-16T05:44:12.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 1 -- 172.21.15.32:0/2257043358 wait complete. 2024-04-16T05:44:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29fabd7740 20 client.17775 trim_cache size 0 max 16384 2024-04-16T05:44:12.614 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:44:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 -- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29dc0a3d20 msgr2=0x7f29dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29dc0a3d20 0x7f29dc0a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f29e80efeb0 tx=0x7f29e808d050 comp rx=0 tx=0).stop 2024-04-16T05:44:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 -- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0198660 msgr2=0x7f29f0198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0198660 0x7f29f0198ae0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f29e802c700 tx=0x7f29e8032f20 comp rx=0 tx=0).stop 2024-04-16T05:44:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 -- 172.21.15.32:0/818681631 shutdown_connections 2024-04-16T05:44:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f29dc0a3d20 0x7f29dc0a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0198660 0x7f29f0198ae0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29f0107320 0x7f29f019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 --2- 172.21.15.32:0/818681631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29f0102250 0x7f29f019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 -- 172.21.15.32:0/818681631 >> 172.21.15.32:0/818681631 conn(0x7f29f0079490 msgr2=0x7f29f00ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 -- 172.21.15.32:0/818681631 shutdown_connections 2024-04-16T05:44:12.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.605+0000 7f29f7f1b640 1 -- 172.21.15.32:0/818681631 wait complete. 2024-04-16T05:44:12.616 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:12 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2440811133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:12.617 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:44:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:12 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2440811133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:12.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:12 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2440811133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.726+0000 7f929ab38640 1 -- 172.21.15.32:0/270554460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9294103bc0 msgr2=0x7f929410c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.726+0000 7f929ab38640 1 --2- 172.21.15.32:0/270554460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9294103bc0 0x7f929410c0a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f9288009900 tx=0x7f928802bd60 comp rx=0 tx=0).stop 2024-04-16T05:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.726+0000 7f929ab38640 1 -- 172.21.15.32:0/270554460 shutdown_connections 2024-04-16T05:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.726+0000 7f929ab38640 1 --2- 172.21.15.32:0/270554460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f929410c5e0 0x7f929410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.726+0000 7f929ab38640 1 --2- 172.21.15.32:0/270554460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9294103bc0 0x7f929410c0a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.726+0000 7f929ab38640 1 --2- 172.21.15.32:0/270554460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92941032a0 0x7f9294103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.726+0000 7f929ab38640 1 -- 172.21.15.32:0/270554460 >> 172.21.15.32:0/270554460 conn(0x7f9294077280 msgr2=0x7f9294077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 -- 172.21.15.32:0/270554460 shutdown_connections 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 -- 172.21.15.32:0/270554460 wait complete. 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 Processor -- start 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 -- start start 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92941032a0 0x7f929419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9294103bc0 0x7f929419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f929410c5e0 0x7f92941986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92941123f0 con 0x7f92941032a0 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9294112270 con 0x7f9294103bc0 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f929ab38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9294112570 con 0x7f929410c5e0 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f92988d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92941032a0 0x7f929419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f92988d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92941032a0 0x7f929419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58568/0 (socket says 172.21.15.32:58568) 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.727+0000 7f92988d4640 1 -- 172.21.15.32:0/3024500052 learned_addr learned my addr 172.21.15.32:0/3024500052 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f9293fff640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9294103bc0 0x7f929419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f92990d5640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f929410c5e0 0x7f92941986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f92988d4640 1 -- 172.21.15.32:0/3024500052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f929410c5e0 msgr2=0x7f92941986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f92988d4640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f929410c5e0 0x7f92941986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f92988d4640 1 -- 172.21.15.32:0/3024500052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9294103bc0 msgr2=0x7f929419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f92988d4640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9294103bc0 0x7f929419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f92988d4640 1 -- 172.21.15.32:0/3024500052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9294198fb0 con 0x7f92941032a0 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f92988d4640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92941032a0 0x7f929419e5e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f928400d950 tx=0x7f928400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f9291ffb640 1 -- 172.21.15.32:0/3024500052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284017070 con 0x7f92941032a0 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f9291ffb640 1 -- 172.21.15.32:0/3024500052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9284009c30 con 0x7f92941032a0 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f929ab38640 1 -- 172.21.15.32:0/3024500052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92941992a0 con 0x7f92941032a0 2024-04-16T05:44:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f9291ffb640 1 -- 172.21.15.32:0/3024500052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928400f6b0 con 0x7f92941032a0 2024-04-16T05:44:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.728+0000 7f929ab38640 1 -- 172.21.15.32:0/3024500052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92941a5560 con 0x7f92941032a0 2024-04-16T05:44:12.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f9291ffb640 1 -- 172.21.15.32:0/3024500052 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f92840109d0 con 0x7f92941032a0 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f929d7f3740 1 Processor -- start 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f929d7f3740 1 -- start start 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f9291ffb640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f927c0a3d20 0x7f927c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f92990d5640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f927c0a3d20 0x7f927c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f9291ffb640 1 -- 172.21.15.32:0/3024500052 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f92840d0770 con 0x7f92941032a0 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d0490240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0498e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 0x5602d049b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f929d7f3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5602d03f50e0 con 0x5602d0490780 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f929d7f3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5602d0392520 con 0x5602d048fe60 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f929d7f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602d03f10a0 con 0x5602d0499360 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.731+0000 7f9293fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0498e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f9293fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0498e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58580/0 (socket says 172.21.15.32:58580) 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f9293fff640 1 -- 172.21.15.32:0/3949391270 learned_addr learned my addr 172.21.15.32:0/3949391270 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f92990d5640 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 0x5602d049b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f92988d4640 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d0490240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f9293fff640 1 -- 172.21.15.32:0/3949391270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 msgr2=0x5602d049b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f9293fff640 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 0x5602d049b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f9293fff640 1 -- 172.21.15.32:0/3949391270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 msgr2=0x5602d0490240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f9293fff640 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d0490240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f9293fff640 1 -- 172.21.15.32:0/3949391270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602d03ba8d0 con 0x5602d0490780 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f92990d5640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f927c0a3d20 0x7f927c0a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f928c005fd0 tx=0x7f928c0107a0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f9293fff640 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0498e20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f928802c370 tx=0x7f928802c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be4a2e70f73aa09 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f927affd640 1 -- 172.21.15.32:0/3949391270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288043070 con 0x5602d0490780 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f927affd640 1 -- 172.21.15.32:0/3949391270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f928802c9b0 con 0x5602d0490780 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.732+0000 7f927affd640 1 -- 172.21.15.32:0/3949391270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288004c80 con 0x5602d0490780 2024-04-16T05:44:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 -- 172.21.15.32:0/3949391270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 msgr2=0x5602d0498e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0498e20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f928802c370 tx=0x7f928802c700 comp rx=0 tx=0).stop 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 -- 172.21.15.32:0/3949391270 shutdown_connections 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 0x5602d049b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0498e20 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3949391270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d0490240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 -- 172.21.15.32:0/3949391270 >> 172.21.15.32:0/3949391270 conn(0x5602d048a590 msgr2=0x5602d048a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 -- 172.21.15.32:0/3949391270 shutdown_connections 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 -- 172.21.15.32:0/3949391270 wait complete. 2024-04-16T05:44:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 -- start start 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.733+0000 7f929d7f3740 1 Processor -- start 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d04962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0496830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 0x5602d0496d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f929d7f3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5602d049c000 con 0x5602d0490780 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f929d7f3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5602d03ae4d0 con 0x5602d048fe60 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f929d7f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602d049c180 con 0x5602d0499360 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f929d7f3740 10 client.0 _ll_register_callbacks cb 0x5602d03932c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0496830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0496830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58594/0 (socket says 172.21.15.32:58594) 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 -- 172.21.15.32:0/3192338049 learned_addr learned my addr 172.21.15.32:0/3192338049 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f92990d5640 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 0x5602d0496d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f92988d4640 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d04962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 -- 172.21.15.32:0/3192338049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 msgr2=0x5602d0496d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 0x5602d0496d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 -- 172.21.15.32:0/3192338049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 msgr2=0x5602d04962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d04962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 -- 172.21.15.32:0/3192338049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602d01073f0 con 0x5602d0490780 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9293fff640 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0496830 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f928802c2a0 tx=0x7f9288004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930eaf5305f94e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.734+0000 7f9273fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f9273fff640 1 -- 172.21.15.32:0/3192338049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288043070 con 0x5602d0490780 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 1 -- 172.21.15.32:0/3192338049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928802c9b0 con 0x5602d0490780 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f9273fff640 1 -- 172.21.15.32:0/3192338049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f9288004ec0 con 0x5602d0490780 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f9273fff640 1 -- 172.21.15.32:0/3192338049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9288056050 con 0x5602d0490780 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 10 client.17787 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 1 -- 172.21.15.32:0/3192338049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5602d01d17d0 con 0x5602d0490780 2024-04-16T05:44:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f9273fff640 1 -- 172.21.15.32:0/3192338049 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f928802c9b0 con 0x5602d0490780 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f9273fff640 1 -- 172.21.15.32:0/3192338049 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9288003bc0 con 0x5602d0490780 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 10 client.17787 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 20 client.17787 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5602d04a90c0 0x5602d04ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 4 client.17787 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f929d7f3740 1 -- 172.21.15.32:0/3192338049 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602d0402b00 con 0x5602d04a90c0 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.735+0000 7f92988d4640 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5602d04a90c0 0x5602d04ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.736+0000 7f92988d4640 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5602d04a90c0 0x5602d04ab4b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c5629a5883f681 server_cookie=d4452652be9cf738 in_seq=0 out_seq=0 2024-04-16T05:44:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.736+0000 7f9273fff640 10 client.17787 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.736+0000 7f9273fff640 1 -- 172.21.15.32:0/3192338049 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602d0402b00 con 0x5602d04a90c0 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.736+0000 7f9273fff640 10 client.17787 handle_command_reply: tid=0 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.736+0000 7f929d7f3740 1 client.17787 shutdown 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 -- 172.21.15.32:0/3192338049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 msgr2=0x5602d0496830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0496830 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f928802c2a0 tx=0x7f9288004bd0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 -- 172.21.15.32:0/3192338049 shutdown_connections 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5602d04a90c0 0x5602d04ab4b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0499360 0x5602d0496d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0490780 0x5602d0496830 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f9273fff640 0 client.17787 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:12.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 --2- 172.21.15.32:0/3192338049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d04962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 -- 172.21.15.32:0/3192338049 >> 172.21.15.32:0/3192338049 conn(0x5602d048a590 msgr2=0x5602d048c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 -- 172.21.15.32:0/3192338049 shutdown_connections 2024-04-16T05:44:12.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 1 -- 172.21.15.32:0/3192338049 wait complete. 2024-04-16T05:44:12.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.737+0000 7f929d7f3740 20 client.17787 trim_cache size 0 max 16384 2024-04-16T05:44:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f929d7f3740 1 Processor -- start 2024-04-16T05:44:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f929d7f3740 1 -- start start 2024-04-16T05:44:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d048fe60 0x5602d0498a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 0x5602d04a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d0499360 0x5602d04b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f929d7f3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92941b1de0 con 0x5602d048fe60 2024-04-16T05:44:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f929d7f3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92941af6b0 con 0x5602d0499360 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f929d7f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92941ad520 con 0x5602d0490780 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f92988d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d048fe60 0x5602d0498a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f92988d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d048fe60 0x5602d0498a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58610/0 (socket says 172.21.15.32:58610) 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.750+0000 7f92988d4640 1 -- 172.21.15.32:0/1676908822 learned_addr learned my addr 172.21.15.32:0/1676908822 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f92988d4640 1 -- 172.21.15.32:0/1676908822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 msgr2=0x5602d04a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f92988d4640 1 --2- 172.21.15.32:0/1676908822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 0x5602d04a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f92988d4640 1 -- 172.21.15.32:0/1676908822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d0499360 msgr2=0x5602d04b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f92988d4640 1 --2- 172.21.15.32:0/1676908822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d0499360 0x5602d04b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f92988d4640 1 -- 172.21.15.32:0/1676908822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602d0485e60 con 0x5602d048fe60 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f92988d4640 1 --2- 172.21.15.32:0/1676908822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d048fe60 0x5602d0498a20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9284010fd0 tx=0x7f9284005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a41f0a295579e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f927a7fc640 1 -- 172.21.15.32:0/1676908822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284020040 con 0x5602d048fe60 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f927a7fc640 1 -- 172.21.15.32:0/1676908822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f928409a6d0 con 0x5602d048fe60 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.751+0000 7f927a7fc640 1 -- 172.21.15.32:0/1676908822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928409b820 con 0x5602d048fe60 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- 172.21.15.32:0/1676908822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d048fe60 msgr2=0x5602d0498a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 --2- 172.21.15.32:0/1676908822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d048fe60 0x5602d0498a20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9284010fd0 tx=0x7f9284005880 comp rx=0 tx=0).stop 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- 172.21.15.32:0/1676908822 shutdown_connections 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 --2- 172.21.15.32:0/1676908822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d0499360 0x5602d04b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 --2- 172.21.15.32:0/1676908822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 0x5602d04a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 --2- 172.21.15.32:0/1676908822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d048fe60 0x5602d0498a20 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- 172.21.15.32:0/1676908822 >> 172.21.15.32:0/1676908822 conn(0x5602d048a590 msgr2=0x5602d04855c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- 172.21.15.32:0/1676908822 shutdown_connections 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- 172.21.15.32:0/1676908822 wait complete. 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- start start 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 Processor -- start 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d049c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 0x5602d049cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0499360 0x5602d05072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92941adb70 con 0x5602d0499360 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92941aefe0 con 0x5602d048fe60 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92941ae240 con 0x5602d0490780 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.752+0000 7f929d7f3740 10 client.0 _ll_register_callbacks cb 0x7f9288009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0499360 0x5602d05072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0499360 0x5602d05072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58618/0 (socket says 172.21.15.32:58618) 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 -- 172.21.15.32:0/2212538858 learned_addr learned my addr 172.21.15.32:0/2212538858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f9293fff640 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 0x5602d049cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92988d4640 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d049c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 -- 172.21.15.32:0/2212538858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 msgr2=0x5602d049cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 0x5602d049cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 -- 172.21.15.32:0/2212538858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 msgr2=0x5602d049c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d049c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 -- 172.21.15.32:0/2212538858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602d0507930 con 0x5602d0499360 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f92990d5640 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0499360 0x5602d05072f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f928c013ab0 tx=0x7f928c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb2709c3b3618d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f9273fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f9273fff640 1 -- 172.21.15.32:0/2212538858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c01fbf0 con 0x5602d0499360 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f9273fff640 1 -- 172.21.15.32:0/2212538858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9526+0+0 (secure 0 0 0) 0x7f928c01fd90 con 0x5602d0499360 2024-04-16T05:44:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f929d7f3740 1 -- 172.21.15.32:0/2212538858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928409a6d0 con 0x5602d0499360 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f9273fff640 1 -- 172.21.15.32:0/2212538858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f928c01e450 con 0x5602d0499360 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f929d7f3740 10 client.17793 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f929d7f3740 1 -- 172.21.15.32:0/2212538858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5602d050ed40 con 0x5602d0499360 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.753+0000 7f9273fff640 1 -- 172.21.15.32:0/2212538858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f928409a6d0 con 0x5602d0499360 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f9273fff640 1 -- 172.21.15.32:0/2212538858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f928c011680 con 0x5602d0499360 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f929d7f3740 10 client.17793 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f929d7f3740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f929d7f3740 20 client.17793 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f929d7f3740 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5602d050f540 0x5602d0511960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f929d7f3740 4 client.17793 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f929d7f3740 1 -- 172.21.15.32:0/2212538858 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602d03fc970 con 0x5602d050f540 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f92988d4640 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5602d050f540 0x5602d0511960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f9273fff640 10 client.17793 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f92988d4640 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5602d050f540 0x5602d0511960 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4e905d6098548a server_cookie=edad16d5c6bba025 in_seq=0 out_seq=0 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f9273fff640 1 -- 172.21.15.32:0/2212538858 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602d03fc970 con 0x5602d050f540 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f9273fff640 10 client.17793 handle_command_reply: tid=0 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.754+0000 7f929d7f3740 1 client.17793 shutdown 2024-04-16T05:44:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 -- 172.21.15.32:0/2212538858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0499360 msgr2=0x5602d05072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0499360 0x5602d05072f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f928c013ab0 tx=0x7f928c004ab0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 -- 172.21.15.32:0/2212538858 shutdown_connections 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602d0499360 0x5602d05072f0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5602d050f540 0x5602d0511960 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602d0490780 0x5602d049cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 --2- 172.21.15.32:0/2212538858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602d048fe60 0x5602d049c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 -- 172.21.15.32:0/2212538858 >> 172.21.15.32:0/2212538858 conn(0x5602d048a590 msgr2=0x5602d0488ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f9273fff640 0 client.17793 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 -- 172.21.15.32:0/2212538858 shutdown_connections 2024-04-16T05:44:12.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 1 -- 172.21.15.32:0/2212538858 wait complete. 2024-04-16T05:44:12.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929d7f3740 20 client.17793 trim_cache size 0 max 16384 2024-04-16T05:44:12.763 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:44:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 -- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f927c0a3d20 msgr2=0x7f927c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f927c0a3d20 0x7f927c0a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f928c005fd0 tx=0x7f928c0107a0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 -- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92941032a0 msgr2=0x7f929419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92941032a0 0x7f929419e5e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f928400d950 tx=0x7f928400de10 comp rx=0 tx=0).stop 2024-04-16T05:44:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 -- 172.21.15.32:0/3024500052 shutdown_connections 2024-04-16T05:44:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f929410c5e0 0x7f92941986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9294103bc0 0x7f929419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f927c0a3d20 0x7f927c0a61e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 --2- 172.21.15.32:0/3024500052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92941032a0 0x7f929419e5e0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 -- 172.21.15.32:0/3024500052 >> 172.21.15.32:0/3024500052 conn(0x7f9294077280 msgr2=0x7f92940fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:12.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 -- 172.21.15.32:0/3024500052 shutdown_connections 2024-04-16T05:44:12.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:12.755+0000 7f929ab38640 1 -- 172.21.15.32:0/3024500052 wait complete. 2024-04-16T05:44:13.767 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:44:13.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:44:13.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:44:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 -- 172.21.15.32:0/1635762320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f8107310 msgr2=0x7f56f810eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 --2- 172.21.15.32:0/1635762320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f8107310 0x7f56f810eee0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009ca0 tx=0x7f56f002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 -- 172.21.15.32:0/1635762320 shutdown_connections 2024-04-16T05:44:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 --2- 172.21.15.32:0/1635762320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f8107310 0x7f56f810eee0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 --2- 172.21.15.32:0/1635762320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f8102bf0 0x7f56f8106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 --2- 172.21.15.32:0/1635762320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f8102240 0x7f56f8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 -- 172.21.15.32:0/1635762320 >> 172.21.15.32:0/1635762320 conn(0x7f56f80fda40 msgr2=0x7f56f80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 -- 172.21.15.32:0/1635762320 shutdown_connections 2024-04-16T05:44:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.892+0000 7f56fd847640 1 -- 172.21.15.32:0/1635762320 wait complete. 2024-04-16T05:44:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56fd847640 1 Processor -- start 2024-04-16T05:44:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56fd847640 1 -- start start 2024-04-16T05:44:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56fd847640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f8102240 0x7f56f819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56fd847640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f8107310 0x7f56f819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56fd847640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f81986d0 0x7f56f8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56fd847640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81123d0 con 0x7f56f81986d0 2024-04-16T05:44:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56fd847640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8112250 con 0x7f56f8107310 2024-04-16T05:44:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56fd847640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8112550 con 0x7f56f8102240 2024-04-16T05:44:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56f77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f81986d0 0x7f56f8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56f77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f81986d0 0x7f56f8198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58636/0 (socket says 172.21.15.32:58636) 2024-04-16T05:44:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56f77fe640 1 -- 172.21.15.32:0/3229848082 learned_addr learned my addr 172.21.15.32:0/3229848082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f8102240 0x7f56f819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56effff640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f8107310 0x7f56f819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56f77fe640 1 -- 172.21.15.32:0/3229848082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f8102240 msgr2=0x7f56f819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56f77fe640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f8102240 0x7f56f819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56f77fe640 1 -- 172.21.15.32:0/3229848082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f8107310 msgr2=0x7f56f819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.893+0000 7f56f77fe640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f8107310 0x7f56f819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.894+0000 7f56f77fe640 1 -- 172.21.15.32:0/3229848082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f81992d0 con 0x7f56f81986d0 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.894+0000 7f56f77fe640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f81986d0 0x7f56f8198b50 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f56f002c700 tx=0x7f56f0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.894+0000 7f56fc845640 1 -- 172.21.15.32:0/3229848082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0041020 con 0x7f56f81986d0 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.894+0000 7f56fc845640 1 -- 172.21.15.32:0/3229848082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f0034cd0 con 0x7f56f81986d0 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.894+0000 7f56fd847640 1 -- 172.21.15.32:0/3229848082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f8199560 con 0x7f56f81986d0 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.894+0000 7f56fc845640 1 -- 172.21.15.32:0/3229848082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0039040 con 0x7f56f81986d0 2024-04-16T05:44:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.894+0000 7f56fd847640 1 -- 172.21.15.32:0/3229848082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f81a56a0 con 0x7f56f81986d0 2024-04-16T05:44:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.896+0000 7f56fc845640 1 -- 172.21.15.32:0/3229848082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f56f0044020 con 0x7f56f81986d0 2024-04-16T05:44:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.896+0000 7f570042f740 1 Processor -- start 2024-04-16T05:44:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f570042f740 1 -- start start 2024-04-16T05:44:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f570042f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 0x55f16d048e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f570042f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d051a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f570042f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d0543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f570042f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f16cfade90 con 0x55f16d0493d0 2024-04-16T05:44:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f570042f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f16cf4b4c0 con 0x55f16d048ab0 2024-04-16T05:44:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f570042f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f16cfa9e50 con 0x55f16d051fb0 2024-04-16T05:44:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56fc845640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f56e40a3d20 0x7f56e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d051a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d051a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58640/0 (socket says 172.21.15.32:58640) 2024-04-16T05:44:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f6ffd640 1 -- 172.21.15.32:0/1445718955 learned_addr learned my addr 172.21.15.32:0/1445718955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f77fe640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f56e40a3d20 0x7f56e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56effff640 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 0x55f16d048e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56effff640 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d0543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56fc845640 1 -- 172.21.15.32:0/3229848082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f56f00f2980 con 0x7f56f81986d0 2024-04-16T05:44:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f6ffd640 1 -- 172.21.15.32:0/1445718955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 msgr2=0x55f16d0543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d0543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f6ffd640 1 -- 172.21.15.32:0/1445718955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 msgr2=0x55f16d048e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 0x55f16d048e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.897+0000 7f56f6ffd640 1 -- 172.21.15.32:0/1445718955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f16cd887e0 con 0x55f16d0493d0 2024-04-16T05:44:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f56f77fe640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f56e40a3d20 0x7f56e40a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56f0004af0 tx=0x7f56f0090040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d051a70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f56e800bcb0 tx=0x7f56e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bd617805a15e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f56edffb640 1 -- 172.21.15.32:0/1445718955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8015070 con 0x55f16d0493d0 2024-04-16T05:44:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f56edffb640 1 -- 172.21.15.32:0/1445718955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e801a940 con 0x55f16d0493d0 2024-04-16T05:44:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f56edffb640 1 -- 172.21.15.32:0/1445718955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8011990 con 0x55f16d0493d0 2024-04-16T05:44:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 -- 172.21.15.32:0/1445718955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 msgr2=0x55f16d051a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d051a70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f56e800bcb0 tx=0x7f56e800f730 comp rx=0 tx=0).stop 2024-04-16T05:44:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 -- 172.21.15.32:0/1445718955 shutdown_connections 2024-04-16T05:44:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d0543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d051a70 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 --2- 172.21.15.32:0/1445718955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 0x55f16d048e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 -- 172.21.15.32:0/1445718955 >> 172.21.15.32:0/1445718955 conn(0x55f16d0431e0 msgr2=0x55f16d0435c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 -- 172.21.15.32:0/1445718955 shutdown_connections 2024-04-16T05:44:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.898+0000 7f570042f740 1 -- 172.21.15.32:0/1445718955 wait complete. 2024-04-16T05:44:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 1 -- start start 2024-04-16T05:44:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 1 Processor -- start 2024-04-16T05:44:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 0x55f16d04efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d04f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d04fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f16d054da0 con 0x55f16d0493d0 2024-04-16T05:44:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f16cf36410 con 0x55f16d048ab0 2024-04-16T05:44:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f16d054f20 con 0x55f16d051fb0 2024-04-16T05:44:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f56f6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d04f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f570042f740 10 client.0 _ll_register_callbacks cb 0x55f16cf31b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f56f6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d04f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58650/0 (socket says 172.21.15.32:58650) 2024-04-16T05:44:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.899+0000 7f56f6ffd640 1 -- 172.21.15.32:0/3139968795 learned_addr learned my addr 172.21.15.32:0/3139968795 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56effff640 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 0x55f16d04efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56effff640 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d04fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56f6ffd640 1 -- 172.21.15.32:0/3139968795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 msgr2=0x55f16d04fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d04fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56f6ffd640 1 -- 172.21.15.32:0/3139968795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 msgr2=0x55f16d04efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 0x55f16d04efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56f6ffd640 1 -- 172.21.15.32:0/3139968795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f16cde8f70 con 0x55f16d0493d0 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56effff640 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d04fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d04f510 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f56e800bfd0 tx=0x7f56e800ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefc19a9d604cdf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3139968795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8015070 con 0x55f16d0493d0 2024-04-16T05:44:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3139968795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f56e801ad00 con 0x55f16d0493d0 2024-04-16T05:44:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f570042f740 1 -- 172.21.15.32:0/3139968795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e801a940 con 0x55f16d0493d0 2024-04-16T05:44:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3139968795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f56e801a460 con 0x55f16d0493d0 2024-04-16T05:44:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f570042f740 10 client.17805 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f570042f740 1 -- 172.21.15.32:0/3139968795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f16cc80170 con 0x55f16d0493d0 2024-04-16T05:44:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.900+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3139968795 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e801a940 con 0x55f16d0493d0 2024-04-16T05:44:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3139968795 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f56e8005330 con 0x55f16d0493d0 2024-04-16T05:44:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f570042f740 10 client.17805 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f570042f740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f570042f740 20 client.17805 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f570042f740 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f16d0622b0 0x55f16d0646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f570042f740 4 client.17805 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f570042f740 1 -- 172.21.15.32:0/3139968795 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f16cfbb8b0 con 0x55f16d0622b0 2024-04-16T05:44:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f56effff640 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f16d0622b0 0x55f16d0646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f56effff640 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f16d0622b0 0x55f16d0646a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71623c380b6ce20 server_cookie=ad37632d69c5b285 in_seq=0 out_seq=0 2024-04-16T05:44:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.901+0000 7f56d6ffd640 10 client.17805 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3139968795 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f16cfbb8b0 con 0x55f16d0622b0 2024-04-16T05:44:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f56d6ffd640 10 client.17805 handle_command_reply: tid=0 2024-04-16T05:44:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 client.17805 shutdown 2024-04-16T05:44:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 -- 172.21.15.32:0/3139968795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 msgr2=0x55f16d04f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d04f510 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f56e800bfd0 tx=0x7f56e800ff50 comp rx=0 tx=0).stop 2024-04-16T05:44:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 -- 172.21.15.32:0/3139968795 shutdown_connections 2024-04-16T05:44:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f16d0622b0 0x55f16d0646a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d04fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d0493d0 0x55f16d04f510 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 --2- 172.21.15.32:0/3139968795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d048ab0 0x55f16d04efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f56d6ffd640 0 client.17805 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 -- 172.21.15.32:0/3139968795 >> 172.21.15.32:0/3139968795 conn(0x55f16d0431e0 msgr2=0x55f16d044f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.902+0000 7f570042f740 1 -- 172.21.15.32:0/3139968795 shutdown_connections 2024-04-16T05:44:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.903+0000 7f570042f740 1 -- 172.21.15.32:0/3139968795 wait complete. 2024-04-16T05:44:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.903+0000 7f570042f740 20 client.17805 trim_cache size 0 max 16384 2024-04-16T05:44:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.920+0000 7f570042f740 1 Processor -- start 2024-04-16T05:44:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.920+0000 7f570042f740 1 -- start start 2024-04-16T05:44:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.920+0000 7f570042f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d048ab0 0x55f16d051700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.920+0000 7f570042f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d05dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.920+0000 7f570042f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d051fb0 0x55f16d069e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f570042f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81b1d60 con 0x55f16d051fb0 2024-04-16T05:44:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f570042f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81af630 con 0x55f16d0493d0 2024-04-16T05:44:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f570042f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81ad4a0 con 0x55f16d048ab0 2024-04-16T05:44:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d051fb0 0x55f16d069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d051fb0 0x55f16d069e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58662/0 (socket says 172.21.15.32:58662) 2024-04-16T05:44:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 -- 172.21.15.32:0/1635328712 learned_addr learned my addr 172.21.15.32:0/1635328712 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d05dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d048ab0 0x55f16d051700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 -- 172.21.15.32:0/1635328712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d048ab0 msgr2=0x55f16d051700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d048ab0 0x55f16d051700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 -- 172.21.15.32:0/1635328712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 msgr2=0x55f16d05dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d05dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 -- 172.21.15.32:0/1635328712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f16d048630 con 0x55f16d051fb0 2024-04-16T05:44:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d05dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56effff640 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d051fb0 0x55f16d069e10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f56e0011740 tx=0x7f56e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97900e1f06e058f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56ed7fa640 1 -- 172.21.15.32:0/1635328712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0011e00 con 0x55f16d051fb0 2024-04-16T05:44:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56ed7fa640 1 -- 172.21.15.32:0/1635328712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e0013930 con 0x55f16d051fb0 2024-04-16T05:44:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.921+0000 7f56ed7fa640 1 -- 172.21.15.32:0/1635328712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0020490 con 0x55f16d051fb0 2024-04-16T05:44:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 -- 172.21.15.32:0/1635328712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d051fb0 msgr2=0x55f16d069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d051fb0 0x55f16d069e10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f56e0011740 tx=0x7f56e00046c0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 -- 172.21.15.32:0/1635328712 shutdown_connections 2024-04-16T05:44:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d051fb0 0x55f16d069e10 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d05dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 --2- 172.21.15.32:0/1635328712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d048ab0 0x55f16d051700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 -- 172.21.15.32:0/1635328712 >> 172.21.15.32:0/1635328712 conn(0x55f16d0431e0 msgr2=0x55f16d04f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 -- 172.21.15.32:0/1635328712 shutdown_connections 2024-04-16T05:44:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 -- 172.21.15.32:0/1635328712 wait complete. 2024-04-16T05:44:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.922+0000 7f570042f740 1 -- start start 2024-04-16T05:44:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f570042f740 1 Processor -- start 2024-04-16T05:44:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f570042f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d048ab0 0x55f16d05a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f570042f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d057140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f570042f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d057680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f570042f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81adaf0 con 0x55f16d048ab0 2024-04-16T05:44:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f570042f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81aef60 con 0x55f16d0493d0 2024-04-16T05:44:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f570042f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81ae1c0 con 0x55f16d051fb0 2024-04-16T05:44:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f570042f740 10 client.0 _ll_register_callbacks cb 0x7f56e0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d048ab0 0x55f16d05a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d048ab0 0x55f16d05a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58664/0 (socket says 172.21.15.32:58664) 2024-04-16T05:44:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 -- 172.21.15.32:0/3848111961 learned_addr learned my addr 172.21.15.32:0/3848111961 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d057140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 -- 172.21.15.32:0/3848111961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 msgr2=0x55f16d057680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d057680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 -- 172.21.15.32:0/3848111961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 msgr2=0x55f16d057140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d057140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 -- 172.21.15.32:0/3848111961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f16d057c70 con 0x55f16d048ab0 2024-04-16T05:44:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56effff640 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d048ab0 0x55f16d05a360 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f56e0022010 tx=0x7f56e0009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90930cd60a1597b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.923+0000 7f56d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3848111961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0013ce0 con 0x55f16d048ab0 2024-04-16T05:44:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3848111961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f56e0004040 con 0x55f16d048ab0 2024-04-16T05:44:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 1 -- 172.21.15.32:0/3848111961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e0013930 con 0x55f16d048ab0 2024-04-16T05:44:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3848111961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f56e0004260 con 0x55f16d048ab0 2024-04-16T05:44:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 10 client.17811 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 1 -- 172.21.15.32:0/3848111961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f16d0c94f0 con 0x55f16d048ab0 2024-04-16T05:44:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3848111961 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e0013930 con 0x55f16d048ab0 2024-04-16T05:44:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3848111961 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f56e001d070 con 0x55f16d048ab0 2024-04-16T05:44:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 10 client.17811 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 20 client.17811 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f16d0c9b90 0x55f16d0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 4 client.17811 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:44:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f570042f740 1 -- 172.21.15.32:0/3848111961 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f16cfb5720 con 0x55f16d0c9b90 2024-04-16T05:44:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.924+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f16d0c9b90 0x55f16d0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.925+0000 7f56f6ffd640 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f16d0c9b90 0x55f16d0cbfb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ce0f742d7c29ca server_cookie=cccaf5d88e7c3ca3 in_seq=0 out_seq=0 2024-04-16T05:44:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.925+0000 7f56d6ffd640 10 client.17811 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f56d6ffd640 1 -- 172.21.15.32:0/3848111961 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f16cfb5720 con 0x55f16d0c9b90 2024-04-16T05:44:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f56d6ffd640 10 client.17811 handle_command_reply: tid=0 2024-04-16T05:44:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 client.17811 shutdown 2024-04-16T05:44:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 -- 172.21.15.32:0/3848111961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d048ab0 msgr2=0x55f16d05a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d048ab0 0x55f16d05a360 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f56e0022010 tx=0x7f56e0009c70 comp rx=0 tx=0).stop 2024-04-16T05:44:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 -- 172.21.15.32:0/3848111961 shutdown_connections 2024-04-16T05:44:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55f16d0c9b90 0x55f16d0cbfb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d051fb0 0x55f16d057680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f16d0493d0 0x55f16d057140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 --2- 172.21.15.32:0/3848111961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f16d048ab0 0x55f16d05a360 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 -- 172.21.15.32:0/3848111961 >> 172.21.15.32:0/3848111961 conn(0x55f16d0431e0 msgr2=0x55f16d0419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f56d6ffd640 0 client.17811 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 -- 172.21.15.32:0/3848111961 shutdown_connections 2024-04-16T05:44:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 1 -- 172.21.15.32:0/3848111961 wait complete. 2024-04-16T05:44:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.926+0000 7f570042f740 20 client.17811 trim_cache size 0 max 16384 2024-04-16T05:44:13.937 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:44:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:44:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "6ac250c2-37f0-48bf-8342-aa18e3d0068f", 2024-04-16T05:44:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:44:13.938 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:44:13.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 -- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f56e40a3d20 msgr2=0x7f56e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f56e40a3d20 0x7f56e40a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56f0004af0 tx=0x7f56f0090040 comp rx=0 tx=0).stop 2024-04-16T05:44:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 -- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f81986d0 msgr2=0x7f56f8198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f81986d0 0x7f56f8198b50 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f56f002c700 tx=0x7f56f0032f20 comp rx=0 tx=0).stop 2024-04-16T05:44:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 -- 172.21.15.32:0/3229848082 shutdown_connections 2024-04-16T05:44:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f56e40a3d20 0x7f56e40a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56f81986d0 0x7f56f8198b50 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f8107310 0x7f56f819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 --2- 172.21.15.32:0/3229848082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f8102240 0x7f56f819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 -- 172.21.15.32:0/3229848082 >> 172.21.15.32:0/3229848082 conn(0x7f56f80fda40 msgr2=0x7f56f80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:13.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 -- 172.21.15.32:0/3229848082 shutdown_connections 2024-04-16T05:44:13.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:13.927+0000 7f56fd847640 1 -- 172.21.15.32:0/3229848082 wait complete. 2024-04-16T05:44:13.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:44:13.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:44:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 -- 172.21.15.32:0/3199594811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 msgr2=0x7f14e0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 --2- 172.21.15.32:0/3199594811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 0x7f14e0101ad0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009900 tx=0x7f14c802bc80 comp rx=0 tx=0).stop 2024-04-16T05:44:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 -- 172.21.15.32:0/3199594811 shutdown_connections 2024-04-16T05:44:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 --2- 172.21.15.32:0/3199594811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e0102010 0x7f14e010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 --2- 172.21.15.32:0/3199594811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 0x7f14e0101ad0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 --2- 172.21.15.32:0/3199594811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e0105ed0 0x7f14e01062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 -- 172.21.15.32:0/3199594811 >> 172.21.15.32:0/3199594811 conn(0x7f14e0077490 msgr2=0x7f14e00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 -- 172.21.15.32:0/3199594811 shutdown_connections 2024-04-16T05:44:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.051+0000 7f14e4b40640 1 -- 172.21.15.32:0/3199594811 wait complete. 2024-04-16T05:44:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14e4b40640 1 Processor -- start 2024-04-16T05:44:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14e4b40640 1 -- start start 2024-04-16T05:44:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14e4b40640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 0x7f14e019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14e4b40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e0102010 0x7f14e019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14e4b40640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e0105ed0 0x7f14e01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14e4b40640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01124f0 con 0x7f14e0101670 2024-04-16T05:44:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14e4b40640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0112370 con 0x7f14e0102010 2024-04-16T05:44:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14e4b40640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0112670 con 0x7f14e0105ed0 2024-04-16T05:44:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14de59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 0x7f14e019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14de59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 0x7f14e019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58680/0 (socket says 172.21.15.32:58680) 2024-04-16T05:44:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14de59c640 1 -- 172.21.15.32:0/1605357638 learned_addr learned my addr 172.21.15.32:0/1605357638 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.052+0000 7f14d7fff640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e0102010 0x7f14e019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14ded9d640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e0105ed0 0x7f14e01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14de59c640 1 -- 172.21.15.32:0/1605357638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e0105ed0 msgr2=0x7f14e01987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14de59c640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e0105ed0 0x7f14e01987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14de59c640 1 -- 172.21.15.32:0/1605357638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e0102010 msgr2=0x7f14e019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14de59c640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e0102010 0x7f14e019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14de59c640 1 -- 172.21.15.32:0/1605357638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e0198f80 con 0x7f14e0101670 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14ded9d640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e0105ed0 0x7f14e01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14de59c640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 0x7f14e019e640 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d950 tx=0x7f14d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14d77fe640 1 -- 172.21.15.32:0/1605357638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0017070 con 0x7f14e0101670 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14d77fe640 1 -- 172.21.15.32:0/1605357638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d0009c30 con 0x7f14e0101670 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14e4b40640 1 -- 172.21.15.32:0/1605357638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e0199270 con 0x7f14e0101670 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14d77fe640 1 -- 172.21.15.32:0/1605357638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d000f6b0 con 0x7f14e0101670 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.053+0000 7f14e4b40640 1 -- 172.21.15.32:0/1605357638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e01a5490 con 0x7f14e0101670 2024-04-16T05:44:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.055+0000 7f14d77fe640 1 -- 172.21.15.32:0/1605357638 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14d00109d0 con 0x7f14e0101670 2024-04-16T05:44:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.055+0000 7f14e7809740 1 Processor -- start 2024-04-16T05:44:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14e7809740 1 -- start start 2024-04-16T05:44:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14e7809740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a7e50 0x5630382a8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14e7809740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14e7809740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382b1350 0x5630382b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14e7809740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56303820d180 con 0x5630382a8770 2024-04-16T05:44:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14e7809740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5630381aa360 con 0x5630382b1350 2024-04-16T05:44:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14e7809740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563038209140 con 0x5630382a7e50 2024-04-16T05:44:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14de59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a7e50 0x5630382a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14de59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58688/0 (socket says 172.21.15.32:58688) 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14de59c640 1 -- 172.21.15.32:0/3851947045 learned_addr learned my addr 172.21.15.32:0/3851947045 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14d77fe640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14cc0a3da0 0x7f14cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14d7fff640 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382b1350 0x5630382b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14de59c640 1 -- 172.21.15.32:0/3851947045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a7e50 msgr2=0x5630382a8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14de59c640 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a7e50 0x5630382a8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14de59c640 1 -- 172.21.15.32:0/3851947045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382b1350 msgr2=0x5630382b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.056+0000 7f14de59c640 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382b1350 0x5630382b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14de59c640 1 -- 172.21.15.32:0/3851947045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563037fe97d0 con 0x5630382a8770 2024-04-16T05:44:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14ded9d640 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a7e50 0x5630382a8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T05:44:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14ded9d640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14cc0a3da0 0x7f14cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14d77fe640 1 -- 172.21.15.32:0/1605357638 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f14d00d0740 con 0x7f14e0101670 2024-04-16T05:44:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14de59c640 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382b0e10 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f14d00cda70 tx=0x7f14d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668ece4ab8d9727a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14ded9d640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14cc0a3da0 0x7f14cc0a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f14d8005fd0 tx=0x7f14d8004a40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14d4ff9640 1 -- 172.21.15.32:0/3851947045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0020020 con 0x5630382a8770 2024-04-16T05:44:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14d4ff9640 1 -- 172.21.15.32:0/3851947045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d0009de0 con 0x5630382a8770 2024-04-16T05:44:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.057+0000 7f14d4ff9640 1 -- 172.21.15.32:0/3851947045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d009ca50 con 0x5630382a8770 2024-04-16T05:44:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 -- 172.21.15.32:0/3851947045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 msgr2=0x5630382b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382b0e10 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f14d00cda70 tx=0x7f14d0005f90 comp rx=0 tx=0).stop 2024-04-16T05:44:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 -- 172.21.15.32:0/3851947045 shutdown_connections 2024-04-16T05:44:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382b1350 0x5630382b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382b0e10 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 --2- 172.21.15.32:0/3851947045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a7e50 0x5630382a8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 -- 172.21.15.32:0/3851947045 >> 172.21.15.32:0/3851947045 conn(0x5630382a2580 msgr2=0x5630382a2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 -- 172.21.15.32:0/3851947045 shutdown_connections 2024-04-16T05:44:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 -- 172.21.15.32:0/3851947045 wait complete. 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 -- start start 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 Processor -- start 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382b1350 0x5630382aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5630382b4080 con 0x5630382a8770 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.058+0000 7f14e7809740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5630381c62c0 con 0x5630382a7e50 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14e7809740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5630382b4200 con 0x5630382b1350 2024-04-16T05:44:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14e7809740 10 client.0 _ll_register_callbacks cb 0x5630381ad090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58704/0 (socket says 172.21.15.32:58704) 2024-04-16T05:44:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 -- 172.21.15.32:0/1784795388 learned_addr learned my addr 172.21.15.32:0/1784795388 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14ded9d640 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382b1350 0x5630382aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14de59c640 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 -- 172.21.15.32:0/1784795388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382b1350 msgr2=0x5630382aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382b1350 0x5630382aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 -- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 msgr2=0x5630382ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 -- 172.21.15.32:0/1784795388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563037f1f3f0 con 0x5630382a8770 2024-04-16T05:44:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14de59c640 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14d7fff640 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382ae8f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f14c802c1c0 tx=0x7f14c802c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c248ff5103c39831 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14c5ffb640 1 -- 172.21.15.32:0/1784795388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c802caa0 con 0x5630382a8770 2024-04-16T05:44:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14c5ffb640 1 -- 172.21.15.32:0/1784795388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f14c802cc40 con 0x5630382a8770 2024-04-16T05:44:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14e7809740 1 -- 172.21.15.32:0/1784795388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d0009de0 con 0x5630382a8770 2024-04-16T05:44:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14c5ffb640 1 -- 172.21.15.32:0/1784795388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f14c80046c0 con 0x5630382a8770 2024-04-16T05:44:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14e7809740 10 client.17823 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.059+0000 7f14e7809740 1 -- 172.21.15.32:0/1784795388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563037edf0f0 con 0x5630382a8770 2024-04-16T05:44:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14c5ffb640 1 -- 172.21.15.32:0/1784795388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d0009de0 con 0x5630382a8770 2024-04-16T05:44:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14c5ffb640 1 -- 172.21.15.32:0/1784795388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f14c80035f0 con 0x5630382a8770 2024-04-16T05:44:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14e7809740 10 client.17823 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14e7809740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14e7809740 20 client.17823 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:14.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14e7809740 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5630382c1640 0x5630382c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14e7809740 4 client.17823 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:14.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14e7809740 1 -- 172.21.15.32:0/1784795388 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56303821aba0 con 0x5630382c1640 2024-04-16T05:44:14.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14de59c640 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5630382c1640 0x5630382c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14de59c640 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5630382c1640 0x5630382c3a90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115e8165e188ecf4 server_cookie=214fd1df79786ced in_seq=0 out_seq=0 2024-04-16T05:44:14.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.060+0000 7f14c5ffb640 10 client.17823 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:14.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14c5ffb640 1 -- 172.21.15.32:0/1784795388 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56303821aba0 con 0x5630382c1640 2024-04-16T05:44:14.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14c5ffb640 10 client.17823 handle_command_reply: tid=0 2024-04-16T05:44:14.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 client.17823 shutdown 2024-04-16T05:44:14.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 -- 172.21.15.32:0/1784795388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 msgr2=0x5630382ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382ae8f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f14c802c1c0 tx=0x7f14c802c7f0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 -- 172.21.15.32:0/1784795388 shutdown_connections 2024-04-16T05:44:14.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5630382c1640 0x5630382c3a90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382b1350 0x5630382aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382a8770 0x5630382ae8f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 --2- 172.21.15.32:0/1784795388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14c5ffb640 0 client.17823 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:14.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 -- 172.21.15.32:0/1784795388 >> 172.21.15.32:0/1784795388 conn(0x5630382a2580 msgr2=0x5630382a42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:14.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 -- 172.21.15.32:0/1784795388 shutdown_connections 2024-04-16T05:44:14.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 1 -- 172.21.15.32:0/1784795388 wait complete. 2024-04-16T05:44:14.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.061+0000 7f14e7809740 20 client.17823 trim_cache size 0 max 16384 2024-04-16T05:44:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14e7809740 1 Processor -- start 2024-04-16T05:44:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14e7809740 1 -- start start 2024-04-16T05:44:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14e7809740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14e7809740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 0x5630382a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14e7809740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x5630382bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14e7809740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01b1de0 con 0x5630382b1350 2024-04-16T05:44:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14e7809740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01af6b0 con 0x5630382a7e50 2024-04-16T05:44:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14e7809740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01ad520 con 0x5630382a8770 2024-04-16T05:44:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x5630382bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x5630382bcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58710/0 (socket says 172.21.15.32:58710) 2024-04-16T05:44:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14ded9d640 1 -- 172.21.15.32:0/1960034280 learned_addr learned my addr 172.21.15.32:0/1960034280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14de59c640 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14ded9d640 1 -- 172.21.15.32:0/1960034280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 msgr2=0x5630382a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14d7fff640 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 0x5630382a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14ded9d640 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 0x5630382a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14ded9d640 1 -- 172.21.15.32:0/1960034280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 msgr2=0x5630382b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14ded9d640 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.079+0000 7f14ded9d640 1 -- 172.21.15.32:0/1960034280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630382b3d10 con 0x5630382b1350 2024-04-16T05:44:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14ded9d640 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x5630382bcf40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f14d80108e0 tx=0x7f14d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e21af8872c6136 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14c7fff640 1 -- 172.21.15.32:0/1960034280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8010e60 con 0x5630382b1350 2024-04-16T05:44:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14c7fff640 1 -- 172.21.15.32:0/1960034280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d802b070 con 0x5630382b1350 2024-04-16T05:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14c7fff640 1 -- 172.21.15.32:0/1960034280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8026e30 con 0x5630382b1350 2024-04-16T05:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 -- 172.21.15.32:0/1960034280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 msgr2=0x5630382bcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x5630382bcf40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f14d80108e0 tx=0x7f14d8016000 comp rx=0 tx=0).stop 2024-04-16T05:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 -- 172.21.15.32:0/1960034280 shutdown_connections 2024-04-16T05:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x5630382bcf40 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 0x5630382a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 --2- 172.21.15.32:0/1960034280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 -- 172.21.15.32:0/1960034280 >> 172.21.15.32:0/1960034280 conn(0x5630382a2580 msgr2=0x5630382a0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 -- 172.21.15.32:0/1960034280 shutdown_connections 2024-04-16T05:44:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.080+0000 7f14e7809740 1 -- 172.21.15.32:0/1960034280 wait complete. 2024-04-16T05:44:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 1 -- start start 2024-04-16T05:44:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 1 Processor -- start 2024-04-16T05:44:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 0x5630382b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x563038326c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01adb70 con 0x5630382b1350 2024-04-16T05:44:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01aefe0 con 0x5630382a7e50 2024-04-16T05:44:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01ae240 con 0x5630382a8770 2024-04-16T05:44:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14e7809740 10 client.0 _ll_register_callbacks cb 0x7f14c8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x563038326c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x563038326c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58726/0 (socket says 172.21.15.32:58726) 2024-04-16T05:44:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14ded9d640 1 -- 172.21.15.32:0/828701066 learned_addr learned my addr 172.21.15.32:0/828701066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14d7fff640 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 0x5630382b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14de59c640 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14ded9d640 1 -- 172.21.15.32:0/828701066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 msgr2=0x5630382b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14ded9d640 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 0x5630382b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14ded9d640 1 -- 172.21.15.32:0/828701066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 msgr2=0x5630382b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14ded9d640 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.081+0000 7f14ded9d640 1 -- 172.21.15.32:0/828701066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563038327190 con 0x5630382b1350 2024-04-16T05:44:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14ded9d640 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x563038326c20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f14d8026890 tx=0x7f14d80266a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee66df190dea0ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14c5ffb640 1 -- 172.21.15.32:0/828701066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8033020 con 0x5630382b1350 2024-04-16T05:44:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14c5ffb640 1 -- 172.21.15.32:0/828701066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f14d801b9a0 con 0x5630382b1350 2024-04-16T05:44:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 1 -- 172.21.15.32:0/828701066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d802b070 con 0x5630382b1350 2024-04-16T05:44:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14c5ffb640 1 -- 172.21.15.32:0/828701066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f14d803e050 con 0x5630382b1350 2024-04-16T05:44:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 10 client.17829 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 1 -- 172.21.15.32:0/828701066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630383284a0 con 0x5630382b1350 2024-04-16T05:44:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14c5ffb640 1 -- 172.21.15.32:0/828701066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d802b070 con 0x5630382b1350 2024-04-16T05:44:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14c5ffb640 1 -- 172.21.15.32:0/828701066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f14d800f040 con 0x5630382b1350 2024-04-16T05:44:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 10 client.17829 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 10 client.17829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 20 client.17829 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563038328c30 0x56303832b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 4 client.17829 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:44:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.082+0000 7f14e7809740 1 -- 172.21.15.32:0/828701066 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563038214a10 con 0x563038328c30 2024-04-16T05:44:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14de59c640 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563038328c30 0x56303832b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14c5ffb640 10 client.17829 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14de59c640 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563038328c30 0x56303832b050 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3afb79cbbf834c server_cookie=fdf7115a57502a4d in_seq=0 out_seq=0 2024-04-16T05:44:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14c5ffb640 1 -- 172.21.15.32:0/828701066 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563038214a10 con 0x563038328c30 2024-04-16T05:44:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14c5ffb640 10 client.17829 handle_command_reply: tid=0 2024-04-16T05:44:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 client.17829 shutdown 2024-04-16T05:44:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 -- 172.21.15.32:0/828701066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 msgr2=0x563038326c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x563038326c20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f14d8026890 tx=0x7f14d80266a0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 -- 172.21.15.32:0/828701066 shutdown_connections 2024-04-16T05:44:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5630382b1350 0x563038326c20 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x563038328c30 0x56303832b050 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630382a8770 0x5630382b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 --2- 172.21.15.32:0/828701066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630382a7e50 0x5630382b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14c5ffb640 0 client.17829 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:14.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.083+0000 7f14e7809740 1 -- 172.21.15.32:0/828701066 >> 172.21.15.32:0/828701066 conn(0x5630382a2580 msgr2=0x5630382aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:14.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e7809740 1 -- 172.21.15.32:0/828701066 shutdown_connections 2024-04-16T05:44:14.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e7809740 1 -- 172.21.15.32:0/828701066 wait complete. 2024-04-16T05:44:14.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e7809740 20 client.17829 trim_cache size 0 max 16384 2024-04-16T05:44:14.094 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:44:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:44:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:44:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "6ac250c2-37f0-48bf-8342-aa18e3d0068f": { 2024-04-16T05:44:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:44:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "6ac250c2-37f0-48bf-8342-aa18e3d0068f", 2024-04-16T05:44:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:44:14.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:44:14.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:44:14.095 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:44:14.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 -- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14cc0a3da0 msgr2=0x7f14cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14cc0a3da0 0x7f14cc0a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f14d8005fd0 tx=0x7f14d8004a40 comp rx=0 tx=0).stop 2024-04-16T05:44:14.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 -- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 msgr2=0x7f14e019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:14.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 0x7f14e019e640 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d950 tx=0x7f14d000de10 comp rx=0 tx=0).stop 2024-04-16T05:44:14.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 -- 172.21.15.32:0/1605357638 shutdown_connections 2024-04-16T05:44:14.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e0105ed0 0x7f14e01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e0102010 0x7f14e019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f14cc0a3da0 0x7f14cc0a6260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 --2- 172.21.15.32:0/1605357638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e0101670 0x7f14e019e640 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 -- 172.21.15.32:0/1605357638 >> 172.21.15.32:0/1605357638 conn(0x7f14e0077490 msgr2=0x7f14e0103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 -- 172.21.15.32:0/1605357638 shutdown_connections 2024-04-16T05:44:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:14.084+0000 7f14e4b40640 1 -- 172.21.15.32:0/1605357638 wait complete. 2024-04-16T05:44:14.097 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:13 smithi032 ceph-mon[32351]: pgmap v286: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:44:14.098 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:13 smithi032 ceph-mon[32351]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T05:44:14.098 INFO:tasks.cephfs.filesystem:scrub status for tag:6ac250c2-37f0-48bf-8342-aa18e3d0068f - {'path': '/', 'tag': '6ac250c2-37f0-48bf-8342-aa18e3d0068f', 'options': 'recursive,force'} 2024-04-16T05:44:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:13 smithi134 ceph-mon[36153]: pgmap v286: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:44:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:13 smithi134 ceph-mon[36153]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T05:44:14.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:13 smithi082 ceph-mon[36195]: pgmap v286: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:44:14.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:13 smithi082 ceph-mon[36195]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T05:44:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:14 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:44:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:14 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:14 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:44:15.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:14 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:15.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:14 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:44:15.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:14 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:15.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:14 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:44:15.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:14 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:15.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:14 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:44:15.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:14 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:15.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:14 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:44:15.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:14 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:16.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:15 smithi134 ceph-mon[36153]: pgmap v288: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 246 B/s wr, 7 op/s 2024-04-16T05:44:16.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:15 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:44:16.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:15 smithi082 ceph-mon[36195]: pgmap v288: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 246 B/s wr, 7 op/s 2024-04-16T05:44:16.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:15 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:44:16.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:15 smithi032 ceph-mon[32351]: pgmap v288: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 246 B/s wr, 7 op/s 2024-04-16T05:44:16.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:15 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:44:18.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:17 smithi134 ceph-mon[36153]: pgmap v289: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:44:18.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:17 smithi082 ceph-mon[36195]: pgmap v289: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:44:18.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:17 smithi032 ceph-mon[32351]: pgmap v289: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:44:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:19 smithi134 ceph-mon[36153]: pgmap v290: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:44:20.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:19 smithi082 ceph-mon[36195]: pgmap v290: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:44:20.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:19 smithi032 ceph-mon[32351]: pgmap v290: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:44:22.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:21 smithi134 ceph-mon[36153]: pgmap v291: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:44:22.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:21 smithi082 ceph-mon[36195]: pgmap v291: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:44:22.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:21 smithi032 ceph-mon[32351]: pgmap v291: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-16T05:44:24.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:23 smithi134 ceph-mon[36153]: pgmap v292: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-04-16T05:44:24.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:23 smithi082 ceph-mon[36195]: pgmap v292: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-04-16T05:44:24.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:23 smithi032 ceph-mon[32351]: pgmap v292: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-04-16T05:44:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:25 smithi134 ceph-mon[36153]: pgmap v293: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-16T05:44:26.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:25 smithi082 ceph-mon[36195]: pgmap v293: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-16T05:44:26.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:25 smithi032 ceph-mon[32351]: pgmap v293: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-16T05:44:28.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:27 smithi134 ceph-mon[36153]: pgmap v294: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:28.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:27 smithi082 ceph-mon[36195]: pgmap v294: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:28.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:27 smithi032 ceph-mon[32351]: pgmap v294: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:30.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:29 smithi134 ceph-mon[36153]: pgmap v295: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:30.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:29 smithi082 ceph-mon[36195]: pgmap v295: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:30.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:29 smithi032 ceph-mon[32351]: pgmap v295: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:32.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:31 smithi134 ceph-mon[36153]: pgmap v296: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:32.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:31 smithi082 ceph-mon[36195]: pgmap v296: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:32.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:31 smithi032 ceph-mon[32351]: pgmap v296: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:34.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:33 smithi134 ceph-mon[36153]: pgmap v297: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:34.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:33 smithi082 ceph-mon[36195]: pgmap v297: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:34.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:33 smithi032 ceph-mon[32351]: pgmap v297: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:35 smithi134 ceph-mon[36153]: pgmap v298: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:36.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:35 smithi082 ceph-mon[36195]: pgmap v298: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:36.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:35 smithi032 ceph-mon[32351]: pgmap v298: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:37 smithi134 ceph-mon[36153]: pgmap v299: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:38.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:37 smithi082 ceph-mon[36195]: pgmap v299: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:38.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:37 smithi032 ceph-mon[32351]: pgmap v299: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:39 smithi134 ceph-mon[36153]: pgmap v300: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:40.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:39 smithi082 ceph-mon[36195]: pgmap v300: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:40.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:39 smithi032 ceph-mon[32351]: pgmap v300: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:41 smithi134 ceph-mon[36153]: pgmap v301: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:42.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:41 smithi082 ceph-mon[36195]: pgmap v301: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:41 smithi032 ceph-mon[32351]: pgmap v301: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:44.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:44:44.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:44:44.130 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:43 smithi032 ceph-mon[32351]: pgmap v302: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:43 smithi134 ceph-mon[36153]: pgmap v302: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:44.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:43 smithi082 ceph-mon[36195]: pgmap v302: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:44:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 -- 172.21.15.32:0/3650769208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 msgr2=0x7f445810c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 --2- 172.21.15.32:0/3650769208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 0x7f445810c080 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f4448009900 tx=0x7f444802bd60 comp rx=0 tx=0).stop 2024-04-16T05:44:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 -- 172.21.15.32:0/3650769208 shutdown_connections 2024-04-16T05:44:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 --2- 172.21.15.32:0/3650769208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445810c5c0 0x7f445810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 --2- 172.21.15.32:0/3650769208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 0x7f445810c080 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 --2- 172.21.15.32:0/3650769208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4458103280 0x7f4458103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 -- 172.21.15.32:0/3650769208 >> 172.21.15.32:0/3650769208 conn(0x7f4458078000 msgr2=0x7f4458078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 -- 172.21.15.32:0/3650769208 shutdown_connections 2024-04-16T05:44:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.222+0000 7f445fe74640 1 -- 172.21.15.32:0/3650769208 wait complete. 2024-04-16T05:44:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445fe74640 1 Processor -- start 2024-04-16T05:44:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445fe74640 1 -- start start 2024-04-16T05:44:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445fe74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4458103280 0x7f4458100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445fe74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 0x7f4458101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445fe74640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445810c5c0 0x7f4458101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445fe74640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44581123d0 con 0x7f4458103ba0 2024-04-16T05:44:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445fe74640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4458112250 con 0x7f445810c5c0 2024-04-16T05:44:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445fe74640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4458112550 con 0x7f4458103280 2024-04-16T05:44:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445d40f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 0x7f4458101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445d40f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 0x7f4458101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34714/0 (socket says 172.21.15.32:34714) 2024-04-16T05:44:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445d40f640 1 -- 172.21.15.32:0/1295539117 learned_addr learned my addr 172.21.15.32:0/1295539117 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445e411640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445810c5c0 0x7f4458101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445dc10640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4458103280 0x7f4458100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.223+0000 7f445d40f640 1 -- 172.21.15.32:0/1295539117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4458103280 msgr2=0x7f4458100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f445d40f640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4458103280 0x7f4458100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f445d40f640 1 -- 172.21.15.32:0/1295539117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445810c5c0 msgr2=0x7f4458101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f445d40f640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445810c5c0 0x7f4458101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f445d40f640 1 -- 172.21.15.32:0/1295539117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44581a6df0 con 0x7f4458103ba0 2024-04-16T05:44:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f445dc10640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4458103280 0x7f4458100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f445d40f640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 0x7f4458101410 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4448009a20 tx=0x7f444802df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f444effd640 1 -- 172.21.15.32:0/1295539117 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448043020 con 0x7f4458103ba0 2024-04-16T05:44:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f444effd640 1 -- 172.21.15.32:0/1295539117 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44480054e0 con 0x7f4458103ba0 2024-04-16T05:44:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f445fe74640 1 -- 172.21.15.32:0/1295539117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44581a7080 con 0x7f4458103ba0 2024-04-16T05:44:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f444effd640 1 -- 172.21.15.32:0/1295539117 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448003ce0 con 0x7f4458103ba0 2024-04-16T05:44:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.224+0000 7f445fe74640 1 -- 172.21.15.32:0/1295539117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44581a75d0 con 0x7f4458103ba0 2024-04-16T05:44:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.226+0000 7f444effd640 1 -- 172.21.15.32:0/1295539117 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4448046020 con 0x7f4458103ba0 2024-04-16T05:44:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f4462a64740 1 Processor -- start 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f4462a64740 1 -- start start 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f4462a64740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f29230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f4462a64740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f31e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f4462a64740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38f34740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f4462a64740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d38e8e180 con 0x564d38f28e50 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f4462a64740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564d38e2b360 con 0x564d38f32350 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f4462a64740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d38e8a140 con 0x564d38f29770 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f444effd640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44440a3d20 0x7f44440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f445e411640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f29230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f445e411640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f29230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34722/0 (socket says 172.21.15.32:34722) 2024-04-16T05:44:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f445e411640 1 -- 172.21.15.32:0/2536277285 learned_addr learned my addr 172.21.15.32:0/2536277285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f445e411640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44440a3d20 0x7f44440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f445dc10640 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f31e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f445d40f640 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38f34740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.227+0000 7f444effd640 1 -- 172.21.15.32:0/1295539117 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f44480f2cb0 con 0x7f4458103ba0 2024-04-16T05:44:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445e411640 1 -- 172.21.15.32:0/2536277285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 msgr2=0x564d38f31e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445e411640 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f31e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445e411640 1 -- 172.21.15.32:0/2536277285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 msgr2=0x564d38f34740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445e411640 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38f34740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445e411640 1 -- 172.21.15.32:0/2536277285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d38c6a7d0 con 0x564d38f28e50 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445d40f640 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38f34740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445dc10640 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f31e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445e411640 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f29230 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f4450012d00 tx=0x7f4450010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb54a3d3964ff6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f445e411640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44440a3d20 0x7f44440a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4450010c40 tx=0x7f4450017000 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f4443fff640 1 -- 172.21.15.32:0/2536277285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450024070 con 0x564d38f28e50 2024-04-16T05:44:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f4443fff640 1 -- 172.21.15.32:0/2536277285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4450009d00 con 0x564d38f28e50 2024-04-16T05:44:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.228+0000 7f4443fff640 1 -- 172.21.15.32:0/2536277285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450005220 con 0x564d38f28e50 2024-04-16T05:44:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 -- 172.21.15.32:0/2536277285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 msgr2=0x564d38f29230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f29230 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f4450012d00 tx=0x7f4450010740 comp rx=0 tx=0).stop 2024-04-16T05:44:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 -- 172.21.15.32:0/2536277285 shutdown_connections 2024-04-16T05:44:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38f34740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f31e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 --2- 172.21.15.32:0/2536277285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f29230 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 -- 172.21.15.32:0/2536277285 >> 172.21.15.32:0/2536277285 conn(0x564d38f23580 msgr2=0x564d38f23960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 -- 172.21.15.32:0/2536277285 shutdown_connections 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 -- 172.21.15.32:0/2536277285 wait complete. 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 -- start start 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.229+0000 7f4462a64740 1 Processor -- start 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f4462a64740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f2f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f4462a64740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f4462a64740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f32350 0x564d38f2fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f4462a64740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d38f350e0 con 0x564d38f28e50 2024-04-16T05:44:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f4462a64740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564d38e472c0 con 0x564d38f29770 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f4462a64740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d38f35260 con 0x564d38f32350 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f4462a64740 10 client.0 _ll_register_callbacks cb 0x564d38e2e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f2f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f2f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34728/0 (socket says 172.21.15.32:34728) 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 -- 172.21.15.32:0/3285423648 learned_addr learned my addr 172.21.15.32:0/3285423648 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445e411640 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f32350 0x564d38f2fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445d40f640 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 -- 172.21.15.32:0/3285423648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f32350 msgr2=0x564d38f2fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f32350 0x564d38f2fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 -- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 msgr2=0x564d38f2f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 -- 172.21.15.32:0/3285423648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d38ba03f0 con 0x564d38f28e50 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445d40f640 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f445dc10640 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f2f440 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f44540106a0 tx=0x7f4454010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4a693b3f5f63a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.230+0000 7f44417fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f44417fa640 1 -- 172.21.15.32:0/3285423648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454010ea0 con 0x564d38f28e50 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f44417fa640 1 -- 172.21.15.32:0/3285423648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f4454004220 con 0x564d38f28e50 2024-04-16T05:44:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 1 -- 172.21.15.32:0/3285423648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4450009d00 con 0x564d38f28e50 2024-04-16T05:44:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f44417fa640 1 -- 172.21.15.32:0/3285423648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4454023050 con 0x564d38f28e50 2024-04-16T05:44:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 10 client.17841 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 1 -- 172.21.15.32:0/3285423648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d38b600f0 con 0x564d38f28e50 2024-04-16T05:44:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f44417fa640 1 -- 172.21.15.32:0/3285423648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4450009d00 con 0x564d38f28e50 2024-04-16T05:44:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f44417fa640 1 -- 172.21.15.32:0/3285423648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f445400f040 con 0x564d38f28e50 2024-04-16T05:44:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 10 client.17841 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 20 client.17841 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564d38f42730 0x564d38f44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 4 client.17841 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.231+0000 7f4462a64740 1 -- 172.21.15.32:0/3285423648 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d38e9bba0 con 0x564d38f42730 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.232+0000 7f445d40f640 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564d38f42730 0x564d38f44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.232+0000 7f445d40f640 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564d38f42730 0x564d38f44bc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62e77df50c87b1c server_cookie=f6ec84da685e0614 in_seq=0 out_seq=0 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.232+0000 7f44417fa640 10 client.17841 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f44417fa640 1 -- 172.21.15.32:0/3285423648 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d38e9bba0 con 0x564d38f42730 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f44417fa640 10 client.17841 handle_command_reply: tid=0 2024-04-16T05:44:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 client.17841 shutdown 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 -- 172.21.15.32:0/3285423648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 msgr2=0x564d38f2f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f2f440 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f44540106a0 tx=0x7f4454010b60 comp rx=0 tx=0).stop 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 -- 172.21.15.32:0/3285423648 shutdown_connections 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564d38f42730 0x564d38f44bc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f32350 0x564d38f2fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f44417fa640 0 client.17841 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 --2- 172.21.15.32:0/3285423648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f2f440 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 -- 172.21.15.32:0/3285423648 >> 172.21.15.32:0/3285423648 conn(0x564d38f23580 msgr2=0x564d38f252d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 -- 172.21.15.32:0/3285423648 shutdown_connections 2024-04-16T05:44:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 1 -- 172.21.15.32:0/3285423648 wait complete. 2024-04-16T05:44:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.233+0000 7f4462a64740 20 client.17841 trim_cache size 0 max 16384 2024-04-16T05:44:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f4462a64740 1 Processor -- start 2024-04-16T05:44:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f4462a64740 1 -- start start 2024-04-16T05:44:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f4462a64740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f28e50 0x564d38f3dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f4462a64740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f4462a64740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f32350 0x564d38f4a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f4462a64740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44581b3ed0 con 0x564d38f32350 2024-04-16T05:44:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f4462a64740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f44581b17a0 con 0x564d38f29770 2024-04-16T05:44:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f4462a64740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44581af610 con 0x564d38f28e50 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f32350 0x564d38f4a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f32350 0x564d38f4a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34740/0 (socket says 172.21.15.32:34740) 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 -- 172.21.15.32:0/3351340434 learned_addr learned my addr 172.21.15.32:0/3351340434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445d40f640 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 -- 172.21.15.32:0/3351340434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f28e50 msgr2=0x564d38f3dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445dc10640 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f28e50 0x564d38f3dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f28e50 0x564d38f3dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 -- 172.21.15.32:0/3351340434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 msgr2=0x564d38f2fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 -- 172.21.15.32:0/3351340434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d38f25870 con 0x564d38f32350 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f445e411640 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f32350 0x564d38f4a490 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4450012470 tx=0x7f4450012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3ec57a27342d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.251+0000 7f44437fe640 1 -- 172.21.15.32:0/3351340434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450024050 con 0x564d38f32350 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f44437fe640 1 -- 172.21.15.32:0/3351340434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445002c070 con 0x564d38f32350 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f44437fe640 1 -- 172.21.15.32:0/3351340434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44500052a0 con 0x564d38f32350 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 -- 172.21.15.32:0/3351340434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f32350 msgr2=0x564d38f4a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f32350 0x564d38f4a490 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4450012470 tx=0x7f4450012f30 comp rx=0 tx=0).stop 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 -- 172.21.15.32:0/3351340434 shutdown_connections 2024-04-16T05:44:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f32350 0x564d38f4a490 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f29770 0x564d38f2fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 --2- 172.21.15.32:0/3351340434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f28e50 0x564d38f3dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 -- 172.21.15.32:0/3351340434 >> 172.21.15.32:0/3351340434 conn(0x564d38f23580 msgr2=0x564d38f22760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 -- 172.21.15.32:0/3351340434 shutdown_connections 2024-04-16T05:44:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 -- 172.21.15.32:0/3351340434 wait complete. 2024-04-16T05:44:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.252+0000 7f4462a64740 1 -- start start 2024-04-16T05:44:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f4462a64740 1 Processor -- start 2024-04-16T05:44:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f4462a64740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f38660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f4462a64740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f38ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f4462a64740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38fa7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f4462a64740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44581afc60 con 0x564d38f28e50 2024-04-16T05:44:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f4462a64740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f44581b10d0 con 0x564d38f32350 2024-04-16T05:44:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f4462a64740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44581b0330 con 0x564d38f29770 2024-04-16T05:44:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f4462a64740 10 client.0 _ll_register_callbacks cb 0x7f4454009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445dc10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f38660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445dc10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f38660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34756/0 (socket says 172.21.15.32:34756) 2024-04-16T05:44:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445dc10640 1 -- 172.21.15.32:0/1790843220 learned_addr learned my addr 172.21.15.32:0/1790843220 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445e411640 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38fa7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445d40f640 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f38ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445dc10640 1 -- 172.21.15.32:0/1790843220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 msgr2=0x564d38f38ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445dc10640 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f38ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445dc10640 1 -- 172.21.15.32:0/1790843220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 msgr2=0x564d38fa7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445dc10640 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38fa7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445dc10640 1 -- 172.21.15.32:0/1790843220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d38fa82a0 con 0x564d38f28e50 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.253+0000 7f445d40f640 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f38ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f445dc10640 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f38660 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4454009760 tx=0x7f4454012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dac5f048873875c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f44417fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f44417fa640 1 -- 172.21.15.32:0/1790843220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44540037f0 con 0x564d38f28e50 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f44417fa640 1 -- 172.21.15.32:0/1790843220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f445400e8d0 con 0x564d38f28e50 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 1 -- 172.21.15.32:0/1790843220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445002c070 con 0x564d38f28e50 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f44417fa640 1 -- 172.21.15.32:0/1790843220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4454032050 con 0x564d38f28e50 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 10 client.17847 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 1 -- 172.21.15.32:0/1790843220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d38fa9610 con 0x564d38f28e50 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f44417fa640 1 -- 172.21.15.32:0/1790843220 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445002c070 con 0x564d38f28e50 2024-04-16T05:44:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f44417fa640 1 -- 172.21.15.32:0/1790843220 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4454005070 con 0x564d38f28e50 2024-04-16T05:44:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 10 client.17847 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 20 client.17847 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564d38fa9d90 0x564d38fac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 4 client.17847 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:44:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.254+0000 7f4462a64740 1 -- 172.21.15.32:0/1790843220 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d38e95a10 con 0x564d38fa9d90 2024-04-16T05:44:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f445d40f640 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564d38fa9d90 0x564d38fac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f445d40f640 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564d38fa9d90 0x564d38fac1b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24861f9131287a2 server_cookie=ad422c0171a83bd8 in_seq=0 out_seq=0 2024-04-16T05:44:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f44417fa640 10 client.17847 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f44417fa640 1 -- 172.21.15.32:0/1790843220 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564d38e95a10 con 0x564d38fa9d90 2024-04-16T05:44:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f44417fa640 10 client.17847 handle_command_reply: tid=0 2024-04-16T05:44:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f4462a64740 1 client.17847 shutdown 2024-04-16T05:44:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f4462a64740 1 -- 172.21.15.32:0/1790843220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 msgr2=0x564d38f38660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f4462a64740 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f38660 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4454009760 tx=0x7f4454012450 comp rx=0 tx=0).stop 2024-04-16T05:44:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f4462a64740 1 -- 172.21.15.32:0/1790843220 shutdown_connections 2024-04-16T05:44:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f4462a64740 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d38f32350 0x564d38fa7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f4462a64740 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x564d38fa9d90 0x564d38fac1b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f4462a64740 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d38f29770 0x564d38f38ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f4462a64740 1 --2- 172.21.15.32:0/1790843220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d38f28e50 0x564d38f38660 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.255+0000 7f44417fa640 0 client.17847 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f4462a64740 1 -- 172.21.15.32:0/1790843220 >> 172.21.15.32:0/1790843220 conn(0x564d38f23580 msgr2=0x564d38f24860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f4462a64740 1 -- 172.21.15.32:0/1790843220 shutdown_connections 2024-04-16T05:44:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f4462a64740 1 -- 172.21.15.32:0/1790843220 wait complete. 2024-04-16T05:44:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f4462a64740 20 client.17847 trim_cache size 0 max 16384 2024-04-16T05:44:44.265 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:44:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:44:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:44:44.265 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:44:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 -- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44440a3d20 msgr2=0x7f44440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44440a3d20 0x7f44440a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4450010c40 tx=0x7f4450017000 comp rx=0 tx=0).stop 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 -- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 msgr2=0x7f4458101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 0x7f4458101410 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4448009a20 tx=0x7f444802df60 comp rx=0 tx=0).stop 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 -- 172.21.15.32:0/1295539117 shutdown_connections 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445810c5c0 0x7f4458101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f44440a3d20 0x7f44440a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4458103ba0 0x7f4458101410 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 --2- 172.21.15.32:0/1295539117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4458103280 0x7f4458100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 -- 172.21.15.32:0/1295539117 >> 172.21.15.32:0/1295539117 conn(0x7f4458078000 msgr2=0x7f44580fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 -- 172.21.15.32:0/1295539117 shutdown_connections 2024-04-16T05:44:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.256+0000 7f445fe74640 1 -- 172.21.15.32:0/1295539117 wait complete. 2024-04-16T05:44:44.268 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:44:44.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:44:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 -- 172.21.15.32:0/1567416854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a94101b00 msgr2=0x7f3a94105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 --2- 172.21.15.32:0/1567416854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a94101b00 0x7f3a94105b40 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009900 tx=0x7f3a7c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 -- 172.21.15.32:0/1567416854 shutdown_connections 2024-04-16T05:44:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 --2- 172.21.15.32:0/1567416854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94106220 0x7f3a9410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 --2- 172.21.15.32:0/1567416854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a94101b00 0x7f3a94105b40 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 --2- 172.21.15.32:0/1567416854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a94101150 0x7f3a94101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 -- 172.21.15.32:0/1567416854 >> 172.21.15.32:0/1567416854 conn(0x7f3a94077fd0 msgr2=0x7f3a940783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 -- 172.21.15.32:0/1567416854 shutdown_connections 2024-04-16T05:44:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.414+0000 7f3a993d4640 1 -- 172.21.15.32:0/1567416854 wait complete. 2024-04-16T05:44:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a993d4640 1 Processor -- start 2024-04-16T05:44:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a993d4640 1 -- start start 2024-04-16T05:44:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a993d4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94101150 0x7f3a94199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a993d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a94106220 0x7f3a94199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a993d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9419a1e0 0x7f3a941a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a993d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3a941112b0 con 0x7f3a9419a1e0 2024-04-16T05:44:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a993d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94111130 con 0x7f3a94106220 2024-04-16T05:44:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a993d4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94111430 con 0x7f3a94101150 2024-04-16T05:44:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a937fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9419a1e0 0x7f3a941a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a937fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9419a1e0 0x7f3a941a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34770/0 (socket says 172.21.15.32:34770) 2024-04-16T05:44:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.415+0000 7f3a937fe640 1 -- 172.21.15.32:0/3059180940 learned_addr learned my addr 172.21.15.32:0/3059180940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a92ffd640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94101150 0x7f3a94199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a937fe640 1 -- 172.21.15.32:0/3059180940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94101150 msgr2=0x7f3a94199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a937fe640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94101150 0x7f3a94199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a937fe640 1 -- 172.21.15.32:0/3059180940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a94106220 msgr2=0x7f3a94199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a927fc640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a94106220 0x7f3a94199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a937fe640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a94106220 0x7f3a94199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a937fe640 1 -- 172.21.15.32:0/3059180940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a941a4f90 con 0x7f3a9419a1e0 2024-04-16T05:44:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a927fc640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a94106220 0x7f3a94199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a937fe640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9419a1e0 0x7f3a941a4790 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00e970 tx=0x7f3a8c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a8bfff640 1 -- 172.21.15.32:0/3059180940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c00ccc0 con 0x7f3a9419a1e0 2024-04-16T05:44:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a941a5280 con 0x7f3a9419a1e0 2024-04-16T05:44:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a8bfff640 1 -- 172.21.15.32:0/3059180940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8c00ce60 con 0x7f3a9419a1e0 2024-04-16T05:44:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a8bfff640 1 -- 172.21.15.32:0/3059180940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c010450 con 0x7f3a9419a1e0 2024-04-16T05:44:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.416+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a941a57d0 con 0x7f3a9419a1e0 2024-04-16T05:44:44.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.417+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a94069270 con 0x7f3a9419a1e0 2024-04-16T05:44:44.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.418+0000 7f3a8bfff640 1 -- 172.21.15.32:0/3059180940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a8c005020 con 0x7f3a9419a1e0 2024-04-16T05:44:44.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.419+0000 7f3a8bfff640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3a800a3d20 0x7f3a800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.419+0000 7f3a92ffd640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3a800a3d20 0x7f3a800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.419+0000 7f3a8bfff640 1 -- 172.21.15.32:0/3059180940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f3a8c0d1090 con 0x7f3a9419a1e0 2024-04-16T05:44:44.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.420+0000 7f3a92ffd640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3a800a3d20 0x7f3a800a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005c40 tx=0x7f3a84005bd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.424+0000 7f3a8bfff640 1 -- 172.21.15.32:0/3059180940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a8c09d3a0 con 0x7f3a9419a1e0 2024-04-16T05:44:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.651+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a94101b00 con 0x7f3a9419a1e0 2024-04-16T05:44:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.652+0000 7f3a8bfff640 1 -- 172.21.15.32:0/3059180940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f3a8c09cca0 con 0x7f3a9419a1e0 2024-04-16T05:44:44.651 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:44:44.651 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:44:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:44:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3a800a3d20 msgr2=0x7f3a800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3a800a3d20 0x7f3a800a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005c40 tx=0x7f3a84005bd0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9419a1e0 msgr2=0x7f3a941a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9419a1e0 0x7f3a941a4790 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00e970 tx=0x7f3a8c00ed70 comp rx=0 tx=0).stop 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 shutdown_connections 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3a800a3d20 0x7f3a800a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9419a1e0 0x7f3a941a4790 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a94106220 0x7f3a94199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 --2- 172.21.15.32:0/3059180940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94101150 0x7f3a94199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 >> 172.21.15.32:0/3059180940 conn(0x7f3a94077fd0 msgr2=0x7f3a940ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 shutdown_connections 2024-04-16T05:44:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.654+0000 7f3a993d4640 1 -- 172.21.15.32:0/3059180940 wait complete. 2024-04-16T05:44:44.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:44:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 -- 172.21.15.32:0/2380990628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c108110 msgr2=0x7f9c0c1084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2380990628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c108110 0x7f9c0c1084f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009930 tx=0x7f9bfc02bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 -- 172.21.15.32:0/2380990628 shutdown_connections 2024-04-16T05:44:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2380990628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0c06b020 0x7f9c0c10ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2380990628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c0c06a660 0x7f9c0c06aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2380990628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c108110 0x7f9c0c1084f0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 -- 172.21.15.32:0/2380990628 >> 172.21.15.32:0/2380990628 conn(0x7f9c0c077cf0 msgr2=0x7f9c0c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 -- 172.21.15.32:0/2380990628 shutdown_connections 2024-04-16T05:44:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 -- 172.21.15.32:0/2380990628 wait complete. 2024-04-16T05:44:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 Processor -- start 2024-04-16T05:44:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.772+0000 7f9c131e1640 1 -- start start 2024-04-16T05:44:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c131e1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c0c06a660 0x7f9c0c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c131e1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c06b020 0x7f9c0c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c131e1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0c108110 0x7f9c0c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c131e1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1123a0 con 0x7f9c0c06b020 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c131e1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c112220 con 0x7f9c0c108110 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c131e1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c112520 con 0x7f9c0c06a660 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c06b020 0x7f9c0c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c06b020 0x7f9c0c19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34798/0 (socket says 172.21.15.32:34798) 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 -- 172.21.15.32:0/2795241942 learned_addr learned my addr 172.21.15.32:0/2795241942 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c1177e640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0c108110 0x7f9c0c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c0c06a660 0x7f9c0c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 -- 172.21.15.32:0/2795241942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c0c06a660 msgr2=0x7f9c0c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c0c06a660 0x7f9c0c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 -- 172.21.15.32:0/2795241942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0c108110 msgr2=0x7f9c0c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0c108110 0x7f9c0c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 -- 172.21.15.32:0/2795241942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c198f00 con 0x7f9c0c06b020 2024-04-16T05:44:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c0bfff640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c06b020 0x7f9c0c19eac0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c970 tx=0x7f9bf400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c09ffb640 1 -- 172.21.15.32:0/2795241942 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4007bb0 con 0x7f9c0c06b020 2024-04-16T05:44:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c09ffb640 1 -- 172.21.15.32:0/2795241942 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf4007d50 con 0x7f9c0c06b020 2024-04-16T05:44:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c131e1640 1 -- 172.21.15.32:0/2795241942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1991f0 con 0x7f9c0c06b020 2024-04-16T05:44:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.773+0000 7f9c09ffb640 1 -- 172.21.15.32:0/2795241942 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400f450 con 0x7f9c0c06b020 2024-04-16T05:44:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.774+0000 7f9c131e1640 1 -- 172.21.15.32:0/2795241942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c1a5500 con 0x7f9c0c06b020 2024-04-16T05:44:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c09ffb640 1 -- 172.21.15.32:0/2795241942 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9bf4005020 con 0x7f9c0c06b020 2024-04-16T05:44:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c15dc5740 1 Processor -- start 2024-04-16T05:44:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c15dc5740 1 -- start start 2024-04-16T05:44:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fdeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6fec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c09ffb640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9bf80a3d20 0x7f9bf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6feeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f6d6f440e0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f6d6ee1520 con 0x55f6d6fec770 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f6d6f400a0 con 0x55f6d6fdf430 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c1177e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fdeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c1177e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fdeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34806/0 (socket says 172.21.15.32:34806) 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c0bfff640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9bf80a3d20 0x7f9bf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.776+0000 7f9c1177e640 1 -- 172.21.15.32:0/564661727 learned_addr learned my addr 172.21.15.32:0/564661727 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6fec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c09ffb640 1 -- 172.21.15.32:0/2795241942 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f9bf40d09a0 con 0x7f9c0c06b020 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c1177e640 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6feeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c1177e640 1 -- 172.21.15.32:0/564661727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 msgr2=0x55f6d6fec230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c1177e640 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6fec230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c1177e640 1 -- 172.21.15.32:0/564661727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 msgr2=0x55f6d6feeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c1177e640 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6feeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c1177e640 1 -- 172.21.15.32:0/564661727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d6f098d0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c1177e640 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fdeef0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9c04014930 tx=0x7f9c04009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374e270b7a92e7da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c0bfff640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9bf80a3d20 0x7f9bf80a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cde10 tx=0x7f9bf40870e0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c02ffd640 1 -- 172.21.15.32:0/564661727 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04014050 con 0x55f6d6fdeb10 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c02ffd640 1 -- 172.21.15.32:0/564661727 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c040141f0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.777+0000 7f9c02ffd640 1 -- 172.21.15.32:0/564661727 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04005220 con 0x55f6d6fdeb10 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 -- 172.21.15.32:0/564661727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 msgr2=0x55f6d6fdeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fdeef0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9c04014930 tx=0x7f9c04009f90 comp rx=0 tx=0).stop 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 -- 172.21.15.32:0/564661727 shutdown_connections 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6feeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6fec230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/564661727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fdeef0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 -- 172.21.15.32:0/564661727 >> 172.21.15.32:0/564661727 conn(0x55f6d6fd9590 msgr2=0x55f6d6fd9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 -- 172.21.15.32:0/564661727 shutdown_connections 2024-04-16T05:44:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 -- 172.21.15.32:0/564661727 wait complete. 2024-04-16T05:44:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 -- start start 2024-04-16T05:44:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.778+0000 7f9c15dc5740 1 Processor -- start 2024-04-16T05:44:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fe96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6fe9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f6d6fef5b0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f6d6efd4d0 con 0x55f6d6fec770 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f6d6fef730 con 0x55f6d6fdf430 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c15dc5740 10 client.0 _ll_register_callbacks cb 0x55f6d6ee22c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fe96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fe96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34818/0 (socket says 172.21.15.32:34818) 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 -- 172.21.15.32:0/3398531146 learned_addr learned my addr 172.21.15.32:0/3398531146 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6fe9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 -- 172.21.15.32:0/3398531146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 msgr2=0x55f6d6fe9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6fe9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 -- 172.21.15.32:0/3398531146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 msgr2=0x55f6d6fea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 -- 172.21.15.32:0/3398531146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d6c563f0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c1177e640 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fe96d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f9c04005f40 tx=0x7f9c04012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a42743e18c9483 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9bebfff640 1 -- 172.21.15.32:0/3398531146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0401f430 con 0x55f6d6fdeb10 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9bebfff640 1 -- 172.21.15.32:0/3398531146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f9c0401f5d0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.779+0000 7f9c15dc5740 1 -- 172.21.15.32:0/3398531146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c040141f0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9bebfff640 1 -- 172.21.15.32:0/3398531146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c04032050 con 0x55f6d6fdeb10 2024-04-16T05:44:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c15dc5740 10 client.17865 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c15dc5740 1 -- 172.21.15.32:0/3398531146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6d6d207d0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9bebfff640 1 -- 172.21.15.32:0/3398531146 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c040141f0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9bebfff640 1 -- 172.21.15.32:0/3398531146 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c0400f040 con 0x55f6d6fdeb10 2024-04-16T05:44:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c15dc5740 10 client.17865 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c15dc5740 10 client.17865 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:44:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c15dc5740 20 client.17865 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f6d6ffc720 0x55f6d6ffeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c15dc5740 4 client.17865 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c15dc5740 1 -- 172.21.15.32:0/3398531146 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6d6f51b00 con 0x55f6d6ffc720 2024-04-16T05:44:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.780+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f6d6ffc720 0x55f6d6ffeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.781+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f6d6ffc720 0x55f6d6ffeb10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62760cd05eafd15d server_cookie=bfee8a918bdf9645 in_seq=0 out_seq=0 2024-04-16T05:44:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.781+0000 7f9bebfff640 10 client.17865 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:44:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.781+0000 7f9bebfff640 1 -- 172.21.15.32:0/3398531146 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6d6f51b00 con 0x55f6d6ffc720 2024-04-16T05:44:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.781+0000 7f9bebfff640 10 client.17865 handle_command_reply: tid=0 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.781+0000 7f9c15dc5740 1 client.17865 shutdown 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 -- 172.21.15.32:0/3398531146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 msgr2=0x55f6d6fe96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fe96d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f9c04005f40 tx=0x7f9c04012f40 comp rx=0 tx=0).stop 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 -- 172.21.15.32:0/3398531146 shutdown_connections 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f6d6ffc720 0x55f6d6ffeb10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6fe9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9bebfff640 0 client.17865 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:44:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/3398531146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6fe96d0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 -- 172.21.15.32:0/3398531146 >> 172.21.15.32:0/3398531146 conn(0x55f6d6fd9590 msgr2=0x55f6d6fdb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 -- 172.21.15.32:0/3398531146 shutdown_connections 2024-04-16T05:44:44.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 1 -- 172.21.15.32:0/3398531146 wait complete. 2024-04-16T05:44:44.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.782+0000 7f9c15dc5740 20 client.17865 trim_cache size 0 max 16384 2024-04-16T05:44:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c15dc5740 1 Processor -- start 2024-04-16T05:44:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c15dc5740 1 -- start start 2024-04-16T05:44:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdeb10 0x55f6d6febe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdf430 0x55f6d6ff8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fe8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1afac0 con 0x55f6d6fdf430 2024-04-16T05:44:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1ad390 con 0x55f6d6fec770 2024-04-16T05:44:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1ab200 con 0x55f6d6fdeb10 2024-04-16T05:44:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c10f7d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdf430 0x55f6d6ff8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c10f7d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdf430 0x55f6d6ff8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34826/0 (socket says 172.21.15.32:34826) 2024-04-16T05:44:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c10f7d640 1 -- 172.21.15.32:0/2639484354 learned_addr learned my addr 172.21.15.32:0/2639484354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.795+0000 7f9c1177e640 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdeb10 0x55f6d6febe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c10f7d640 1 -- 172.21.15.32:0/2639484354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdeb10 msgr2=0x55f6d6febe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdeb10 0x55f6d6febe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c10f7d640 1 -- 172.21.15.32:0/2639484354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 msgr2=0x55f6d6fe8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c1177e640 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fe8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fe8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c10f7d640 1 -- 172.21.15.32:0/2639484354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d6fff0a0 con 0x55f6d6fdf430 2024-04-16T05:44:44.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c1177e640 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fe8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:44.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdf430 0x55f6d6ff8510 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc006010 tx=0x7f9bfc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b81edb070347ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c027fc640 1 -- 172.21.15.32:0/2639484354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc02cae0 con 0x55f6d6fdf430 2024-04-16T05:44:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c027fc640 1 -- 172.21.15.32:0/2639484354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bfc02cc80 con 0x55f6d6fdf430 2024-04-16T05:44:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c027fc640 1 -- 172.21.15.32:0/2639484354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0052a0 con 0x55f6d6fdf430 2024-04-16T05:44:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c15dc5740 1 -- 172.21.15.32:0/2639484354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdf430 msgr2=0x55f6d6ff8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.796+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdf430 0x55f6d6ff8510 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc006010 tx=0x7f9bfc005a20 comp rx=0 tx=0).stop 2024-04-16T05:44:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 -- 172.21.15.32:0/2639484354 shutdown_connections 2024-04-16T05:44:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6fe8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdf430 0x55f6d6ff8510 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/2639484354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdeb10 0x55f6d6febe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 -- 172.21.15.32:0/2639484354 >> 172.21.15.32:0/2639484354 conn(0x55f6d6fd9590 msgr2=0x55f6d6fd8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 -- 172.21.15.32:0/2639484354 shutdown_connections 2024-04-16T05:44:44.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 -- 172.21.15.32:0/2639484354 wait complete. 2024-04-16T05:44:44.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 -- start start 2024-04-16T05:44:44.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 Processor -- start 2024-04-16T05:44:44.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6ff4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6feffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6ff0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1ab850 con 0x55f6d6fdeb10 2024-04-16T05:44:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1accc0 con 0x55f6d6fec770 2024-04-16T05:44:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1abf20 con 0x55f6d6fdf430 2024-04-16T05:44:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c15dc5740 10 client.0 _ll_register_callbacks cb 0x7f9bfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c1177e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6ff4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c1177e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6ff4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34834/0 (socket says 172.21.15.32:34834) 2024-04-16T05:44:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.797+0000 7f9c1177e640 1 -- 172.21.15.32:0/331667328 learned_addr learned my addr 172.21.15.32:0/331667328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c1177e640 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6ff0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6feffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c1177e640 1 -- 172.21.15.32:0/331667328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 msgr2=0x55f6d6feffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c1177e640 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6feffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c1177e640 1 -- 172.21.15.32:0/331667328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 msgr2=0x55f6d6ff0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c1177e640 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6ff0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c1177e640 1 -- 172.21.15.32:0/331667328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d6fd93c0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c1177e640 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6ff4c10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9c04012b90 tx=0x7f9c04012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b95a4b30c8992b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9bebfff640 1 -- 172.21.15.32:0/331667328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0403c020 con 0x55f6d6fdeb10 2024-04-16T05:44:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c15dc5740 1 -- 172.21.15.32:0/331667328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bfc02cc80 con 0x55f6d6fdeb10 2024-04-16T05:44:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9bebfff640 1 -- 172.21.15.32:0/331667328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f9c04002030 con 0x55f6d6fdeb10 2024-04-16T05:44:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9bebfff640 1 -- 172.21.15.32:0/331667328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c040144d0 con 0x55f6d6fdeb10 2024-04-16T05:44:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c15dc5740 10 client.17871 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9c15dc5740 1 -- 172.21.15.32:0/331667328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6d7062710 con 0x55f6d6fdeb10 2024-04-16T05:44:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.798+0000 7f9bebfff640 1 -- 172.21.15.32:0/331667328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bfc02cc80 con 0x55f6d6fdeb10 2024-04-16T05:44:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9bebfff640 1 -- 172.21.15.32:0/331667328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c0401f910 con 0x55f6d6fdeb10 2024-04-16T05:44:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c15dc5740 10 client.17871 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c15dc5740 10 client.17871 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:44:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c15dc5740 20 client.17871 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f6d7062dd0 0x55f6d70651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c15dc5740 4 client.17871 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c15dc5740 1 -- 172.21.15.32:0/331667328 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6d6f4b970 con 0x55f6d7062dd0 2024-04-16T05:44:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f6d7062dd0 0x55f6d70651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c10f7d640 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f6d7062dd0 0x55f6d70651c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=980e2b8b34636057 server_cookie=5890934d152c5cb3 in_seq=0 out_seq=0 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9bebfff640 10 client.17871 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9bebfff640 1 -- 172.21.15.32:0/331667328 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6d6f4b970 con 0x55f6d7062dd0 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9bebfff640 10 client.17871 handle_command_reply: tid=0 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.799+0000 7f9c15dc5740 1 client.17871 shutdown 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 -- 172.21.15.32:0/331667328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 msgr2=0x55f6d6ff4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6ff4c10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9c04012b90 tx=0x7f9c04012f40 comp rx=0 tx=0).stop 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 -- 172.21.15.32:0/331667328 shutdown_connections 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x55f6d7062dd0 0x55f6d70651c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6d6fec770 0x55f6d6ff0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6d6fdf430 0x55f6d6feffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9bebfff640 0 client.17871 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:44:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 --2- 172.21.15.32:0/331667328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6d6fdeb10 0x55f6d6ff4c10 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 -- 172.21.15.32:0/331667328 >> 172.21.15.32:0/331667328 conn(0x55f6d6fd9590 msgr2=0x55f6d6fe9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 -- 172.21.15.32:0/331667328 shutdown_connections 2024-04-16T05:44:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 1 -- 172.21.15.32:0/331667328 wait complete. 2024-04-16T05:44:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c15dc5740 20 client.17871 trim_cache size 0 max 16384 2024-04-16T05:44:44.810 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:44:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 -- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9bf80a3d20 msgr2=0x7f9bf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9bf80a3d20 0x7f9bf80a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cde10 tx=0x7f9bf40870e0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 -- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c06b020 msgr2=0x7f9c0c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c06b020 0x7f9c0c19eac0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c970 tx=0x7f9bf400ce30 comp rx=0 tx=0).stop 2024-04-16T05:44:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 -- 172.21.15.32:0/2795241942 shutdown_connections 2024-04-16T05:44:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0c108110 0x7f9c0c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9bf80a3d20 0x7f9bf80a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c0c06b020 0x7f9c0c19eac0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 --2- 172.21.15.32:0/2795241942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c0c06a660 0x7f9c0c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 -- 172.21.15.32:0/2795241942 >> 172.21.15.32:0/2795241942 conn(0x7f9c0c077cf0 msgr2=0x7f9c0c0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 -- 172.21.15.32:0/2795241942 shutdown_connections 2024-04-16T05:44:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.800+0000 7f9c131e1640 1 -- 172.21.15.32:0/2795241942 wait complete. 2024-04-16T05:44:44.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:44:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 -- 172.21.15.32:0/118492730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc101ae0 msgr2=0x7fd5fc105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 --2- 172.21.15.32:0/118492730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc101ae0 0x7fd5fc105b20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009900 tx=0x7fd5f002bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 -- 172.21.15.32:0/118492730 shutdown_connections 2024-04-16T05:44:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 --2- 172.21.15.32:0/118492730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5fc106200 0x7fd5fc10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 --2- 172.21.15.32:0/118492730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc101ae0 0x7fd5fc105b20 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 --2- 172.21.15.32:0/118492730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5fc101130 0x7fd5fc101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 -- 172.21.15.32:0/118492730 >> 172.21.15.32:0/118492730 conn(0x7fd5fc0780b0 msgr2=0x7fd5fc0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 -- 172.21.15.32:0/118492730 shutdown_connections 2024-04-16T05:44:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.922+0000 7fd601104640 1 -- 172.21.15.32:0/118492730 wait complete. 2024-04-16T05:44:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.923+0000 7fd601104640 1 Processor -- start 2024-04-16T05:44:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.923+0000 7fd601104640 1 -- start start 2024-04-16T05:44:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd601104640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5fc101130 0x7fd5fc199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd601104640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc106200 0x7fd5fc199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd601104640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5fc19a0e0 0x7fd5fc1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd601104640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc111290 con 0x7fd5fc106200 2024-04-16T05:44:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd601104640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc111110 con 0x7fd5fc19a0e0 2024-04-16T05:44:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd601104640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc111410 con 0x7fd5fc101130 2024-04-16T05:44:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc106200 0x7fd5fc199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc106200 0x7fd5fc199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34856/0 (socket says 172.21.15.32:34856) 2024-04-16T05:44:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 -- 172.21.15.32:0/2061610318 learned_addr learned my addr 172.21.15.32:0/2061610318 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5fc101130 0x7fd5fc199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5fc19a0e0 0x7fd5fc1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 -- 172.21.15.32:0/2061610318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5fc101130 msgr2=0x7fd5fc199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5fc101130 0x7fd5fc199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 -- 172.21.15.32:0/2061610318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5fc19a0e0 msgr2=0x7fd5fc1a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5fc19a0e0 0x7fd5fc1a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 -- 172.21.15.32:0/2061610318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc1a4e90 con 0x7fd5fc106200 2024-04-16T05:44:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5fa59c640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc106200 0x7fd5fc199b50 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0000c00 tx=0x7fd5f002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5e3fff640 1 -- 172.21.15.32:0/2061610318 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0041070 con 0x7fd5fc106200 2024-04-16T05:44:44.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd601104640 1 -- 172.21.15.32:0/2061610318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc1a5120 con 0x7fd5fc106200 2024-04-16T05:44:44.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5e3fff640 1 -- 172.21.15.32:0/2061610318 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f0004450 con 0x7fd5fc106200 2024-04-16T05:44:44.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd5e3fff640 1 -- 172.21.15.32:0/2061610318 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0034440 con 0x7fd5fc106200 2024-04-16T05:44:44.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.924+0000 7fd601104640 1 -- 172.21.15.32:0/2061610318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc1a5660 con 0x7fd5fc106200 2024-04-16T05:44:44.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd5e3fff640 1 -- 172.21.15.32:0/2061610318 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd5f0044070 con 0x7fd5fc106200 2024-04-16T05:44:44.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd603dca740 1 Processor -- start 2024-04-16T05:44:44.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd603dca740 1 -- start start 2024-04-16T05:44:44.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd5e3fff640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5e40a3d20 0x7fd5e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd603dca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb0c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd603dca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb14e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5e40a3d20 0x7fd5e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd5e3fff640 1 -- 172.21.15.32:0/2061610318 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fd5f00f2810 con 0x7fd5fc106200 2024-04-16T05:44:44.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd603dca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 0x56221cb17750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd603dca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56221ca710e0 con 0x56221cb0be60 2024-04-16T05:44:44.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd603dca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56221ca0e520 con 0x56221cb15360 2024-04-16T05:44:44.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd603dca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56221ca6d0a0 con 0x56221cb0c780 2024-04-16T05:44:44.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.927+0000 7fd5fad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb0c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb0c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34866/0 (socket says 172.21.15.32:34866) 2024-04-16T05:44:44.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fad9d640 1 -- 172.21.15.32:0/1773593510 learned_addr learned my addr 172.21.15.32:0/1773593510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fa59c640 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb14e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 0x56221cb17750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fad9d640 1 -- 172.21.15.32:0/1773593510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 msgr2=0x56221cb14e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb14e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fad9d640 1 -- 172.21.15.32:0/1773593510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 msgr2=0x56221cb17750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 0x56221cb17750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fad9d640 1 -- 172.21.15.32:0/1773593510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221ca368d0 con 0x56221cb0be60 2024-04-16T05:44:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 0x56221cb17750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5e40a3d20 0x7fd5e40a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f4008f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb0c240 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00cc60 tx=0x7fd5ec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febcf6ab54dce782 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5e17fa640 1 -- 172.21.15.32:0/1773593510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec014070 con 0x56221cb0be60 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5e17fa640 1 -- 172.21.15.32:0/1773593510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec0047a0 con 0x56221cb0be60 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.928+0000 7fd5e17fa640 1 -- 172.21.15.32:0/1773593510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec010790 con 0x56221cb0be60 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 -- 172.21.15.32:0/1773593510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 msgr2=0x56221cb0c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb0c240 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00cc60 tx=0x7fd5ec00e720 comp rx=0 tx=0).stop 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 -- 172.21.15.32:0/1773593510 shutdown_connections 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 0x56221cb17750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb14e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 --2- 172.21.15.32:0/1773593510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb0c240 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 -- 172.21.15.32:0/1773593510 >> 172.21.15.32:0/1773593510 conn(0x56221cb06590 msgr2=0x56221cb06970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 -- 172.21.15.32:0/1773593510 shutdown_connections 2024-04-16T05:44:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 -- 172.21.15.32:0/1773593510 wait complete. 2024-04-16T05:44:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 -- start start 2024-04-16T05:44:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.929+0000 7fd603dca740 1 Processor -- start 2024-04-16T05:44:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd603dca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0be60 0x56221cb12400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd603dca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb12940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd603dca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb12e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd603dca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56221cb18060 con 0x56221cb15360 2024-04-16T05:44:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd603dca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56221ca2a4d0 con 0x56221cb0be60 2024-04-16T05:44:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd603dca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56221cb181e0 con 0x56221cb0c780 2024-04-16T05:44:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd603dca740 10 client.0 _ll_register_callbacks cb 0x56221ca0f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb12e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb12e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34882/0 (socket says 172.21.15.32:34882) 2024-04-16T05:44:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 -- 172.21.15.32:0/1741533154 learned_addr learned my addr 172.21.15.32:0/1741533154 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fa59c640 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb12940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0be60 0x56221cb12400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 -- 172.21.15.32:0/1741533154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 msgr2=0x56221cb12940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb12940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 -- 172.21.15.32:0/1741533154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0be60 msgr2=0x56221cb12400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0be60 0x56221cb12400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 -- 172.21.15.32:0/1741533154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221c7833f0 con 0x56221cb15360 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb12e80 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400fa60 tx=0x7fd5f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb98021d032890c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5deff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5deff5640 1 -- 172.21.15.32:0/1741533154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4022020 con 0x56221cb15360 2024-04-16T05:44:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd5deff5640 1 -- 172.21.15.32:0/1741533154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fd5f4011dd0 con 0x56221cb15360 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.930+0000 7fd603dca740 1 -- 172.21.15.32:0/1741533154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ec0047a0 con 0x56221cb15360 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd5deff5640 1 -- 172.21.15.32:0/1741533154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd5f402d050 con 0x56221cb15360 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd603dca740 10 client.17883 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd603dca740 1 -- 172.21.15.32:0/1741533154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56221c84d7d0 con 0x56221cb15360 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd5deff5640 1 -- 172.21.15.32:0/1741533154 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec0047a0 con 0x56221cb15360 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd5deff5640 1 -- 172.21.15.32:0/1741533154 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd5f401d620 con 0x56221cb15360 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd603dca740 10 client.17883 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd603dca740 10 client.17883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:44:44.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd603dca740 20 client.17883 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:44.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd603dca740 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56221cb25340 0x56221cb27790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd603dca740 4 client.17883 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:44.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd603dca740 1 -- 172.21.15.32:0/1741533154 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56221ca7eb00 con 0x56221cb25340 2024-04-16T05:44:44.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.931+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56221cb25340 0x56221cb27790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd5deff5640 10 client.17883 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:44:44.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56221cb25340 0x56221cb27790 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cee4b34cd5e3ba6 server_cookie=fa2438d857930acd in_seq=0 out_seq=0 2024-04-16T05:44:44.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd5deff5640 1 -- 172.21.15.32:0/1741533154 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56221ca7eb00 con 0x56221cb25340 2024-04-16T05:44:44.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd5deff5640 10 client.17883 handle_command_reply: tid=0 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd603dca740 1 client.17883 shutdown 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd603dca740 1 -- 172.21.15.32:0/1741533154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 msgr2=0x56221cb12e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd603dca740 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb12e80 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400fa60 tx=0x7fd5f4004ab0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd603dca740 1 -- 172.21.15.32:0/1741533154 shutdown_connections 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd603dca740 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56221cb25340 0x56221cb27790 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd603dca740 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb12e80 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd603dca740 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb12940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd5deff5640 0 client.17883 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:44:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.932+0000 7fd603dca740 1 --2- 172.21.15.32:0/1741533154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0be60 0x56221cb12400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.933+0000 7fd603dca740 1 -- 172.21.15.32:0/1741533154 >> 172.21.15.32:0/1741533154 conn(0x56221cb06590 msgr2=0x56221cb082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.933+0000 7fd603dca740 1 -- 172.21.15.32:0/1741533154 shutdown_connections 2024-04-16T05:44:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.933+0000 7fd603dca740 1 -- 172.21.15.32:0/1741533154 wait complete. 2024-04-16T05:44:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.933+0000 7fd603dca740 20 client.17883 trim_cache size 0 max 16384 2024-04-16T05:44:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd603dca740 1 Processor -- start 2024-04-16T05:44:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd603dca740 1 -- start start 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd603dca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb14b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd603dca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb21000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd603dca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 0x56221cb2d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd603dca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1b1cc0 con 0x56221cb0be60 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd603dca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1af590 con 0x56221cb15360 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd603dca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ad400 con 0x56221cb0c780 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb14b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb14b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34898/0 (socket says 172.21.15.32:34898) 2024-04-16T05:44:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fad9d640 1 -- 172.21.15.32:0/3451913726 learned_addr learned my addr 172.21.15.32:0/3451913726 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fa59c640 1 --2- 172.21.15.32:0/3451913726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb21000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fad9d640 1 -- 172.21.15.32:0/3451913726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 msgr2=0x56221cb21000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/3451913726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb21000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fad9d640 1 -- 172.21.15.32:0/3451913726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 msgr2=0x56221cb2d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/3451913726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 0x56221cb2d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.946+0000 7fd5fad9d640 1 -- 172.21.15.32:0/3451913726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221cb060c0 con 0x56221cb0be60 2024-04-16T05:44:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/3451913726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb14b30 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00c7e0 tx=0x7fd5ec010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6892dfae62eca774 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd5e0ff9640 1 -- 172.21.15.32:0/3451913726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec014040 con 0x56221cb0be60 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd5e0ff9640 1 -- 172.21.15.32:0/3451913726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec027150 con 0x56221cb0be60 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd5e0ff9640 1 -- 172.21.15.32:0/3451913726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec003850 con 0x56221cb0be60 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 -- 172.21.15.32:0/3451913726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 msgr2=0x56221cb14b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 --2- 172.21.15.32:0/3451913726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb14b30 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00c7e0 tx=0x7fd5ec010430 comp rx=0 tx=0).stop 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 -- 172.21.15.32:0/3451913726 shutdown_connections 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 --2- 172.21.15.32:0/3451913726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb15360 0x56221cb2d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 --2- 172.21.15.32:0/3451913726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0c780 0x56221cb21000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 --2- 172.21.15.32:0/3451913726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb0be60 0x56221cb14b30 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 -- 172.21.15.32:0/3451913726 >> 172.21.15.32:0/3451913726 conn(0x56221cb06590 msgr2=0x56221cb05730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 -- 172.21.15.32:0/3451913726 shutdown_connections 2024-04-16T05:44:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.947+0000 7fd603dca740 1 -- 172.21.15.32:0/3451913726 wait complete. 2024-04-16T05:44:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 1 -- start start 2024-04-16T05:44:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 1 Processor -- start 2024-04-16T05:44:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0be60 0x56221cb18ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0c780 0x56221cb19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb19950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ada50 con 0x56221cb15360 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1aeec0 con 0x56221cb0c780 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ae120 con 0x56221cb0be60 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd603dca740 10 client.0 _ll_register_callbacks cb 0x7fd5f4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fb59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb19950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fb59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb19950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34908/0 (socket says 172.21.15.32:34908) 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fb59e640 1 -- 172.21.15.32:0/2316802148 learned_addr learned my addr 172.21.15.32:0/2316802148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0be60 0x56221cb18ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fa59c640 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0c780 0x56221cb19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fb59e640 1 -- 172.21.15.32:0/2316802148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0be60 msgr2=0x56221cb18ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0be60 0x56221cb18ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fb59e640 1 -- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0c780 msgr2=0x56221cb19410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0c780 0x56221cb19410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fb59e640 1 -- 172.21.15.32:0/2316802148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221cb8a8d0 con 0x56221cb15360 2024-04-16T05:44:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.948+0000 7fd5fa59c640 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0c780 0x56221cb19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd5fb59e640 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb19950 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd5f401df10 tx=0x7fd5f401df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58218904a3076c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd5deff5640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd5deff5640 1 -- 172.21.15.32:0/2316802148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4022040 con 0x56221cb15360 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd5deff5640 1 -- 172.21.15.32:0/2316802148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fd5f4004a00 con 0x56221cb15360 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 1 -- 172.21.15.32:0/2316802148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ec027150 con 0x56221cb15360 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd5deff5640 1 -- 172.21.15.32:0/2316802148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd5f403f050 con 0x56221cb15360 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 10 client.17889 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 1 -- 172.21.15.32:0/2316802148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56221cb8bb70 con 0x56221cb15360 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd5deff5640 1 -- 172.21.15.32:0/2316802148 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec027150 con 0x56221cb15360 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd5deff5640 1 -- 172.21.15.32:0/2316802148 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd5f40052a0 con 0x56221cb15360 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 10 client.17889 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 10 client.17889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:44:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 20 client.17889 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56221cb8c2b0 0x56221cb8e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 4 client.17889 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.949+0000 7fd603dca740 1 -- 172.21.15.32:0/2316802148 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56221ca78970 con 0x56221cb8c2b0 2024-04-16T05:44:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.950+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56221cb8c2b0 0x56221cb8e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.950+0000 7fd5fad9d640 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56221cb8c2b0 0x56221cb8e6d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b64736d1a1a5d538 server_cookie=cacdf968062db094 in_seq=0 out_seq=0 2024-04-16T05:44:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.950+0000 7fd5deff5640 10 client.17889 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:44:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.950+0000 7fd5deff5640 1 -- 172.21.15.32:0/2316802148 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56221ca78970 con 0x56221cb8c2b0 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.950+0000 7fd5deff5640 10 client.17889 handle_command_reply: tid=0 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.950+0000 7fd603dca740 1 client.17889 shutdown 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.950+0000 7fd603dca740 1 -- 172.21.15.32:0/2316802148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 msgr2=0x56221cb19950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.950+0000 7fd603dca740 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb19950 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd5f401df10 tx=0x7fd5f401df40 comp rx=0 tx=0).stop 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 1 -- 172.21.15.32:0/2316802148 shutdown_connections 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x56221cb8c2b0 0x56221cb8e6d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56221cb15360 0x56221cb19950 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56221cb0c780 0x56221cb19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 1 --2- 172.21.15.32:0/2316802148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56221cb0be60 0x56221cb18ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 1 -- 172.21.15.32:0/2316802148 >> 172.21.15.32:0/2316802148 conn(0x56221cb06590 msgr2=0x56221cb123c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd5deff5640 0 client.17889 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:44:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 1 -- 172.21.15.32:0/2316802148 shutdown_connections 2024-04-16T05:44:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 1 -- 172.21.15.32:0/2316802148 wait complete. 2024-04-16T05:44:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd603dca740 20 client.17889 trim_cache size 0 max 16384 2024-04-16T05:44:44.960 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:44:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 -- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5e40a3d20 msgr2=0x7fd5e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5e40a3d20 0x7fd5e40a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f4008f90 comp rx=0 tx=0).stop 2024-04-16T05:44:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 -- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc106200 msgr2=0x7fd5fc199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc106200 0x7fd5fc199b50 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0000c00 tx=0x7fd5f002cc70 comp rx=0 tx=0).stop 2024-04-16T05:44:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 -- 172.21.15.32:0/2061610318 shutdown_connections 2024-04-16T05:44:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5fc19a0e0 0x7fd5fc1a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd5e40a3d20 0x7fd5e40a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc106200 0x7fd5fc199b50 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 --2- 172.21.15.32:0/2061610318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5fc101130 0x7fd5fc199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 -- 172.21.15.32:0/2061610318 >> 172.21.15.32:0/2061610318 conn(0x7fd5fc0780b0 msgr2=0x7fd5fc0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 -- 172.21.15.32:0/2061610318 shutdown_connections 2024-04-16T05:44:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:44.951+0000 7fd601104640 1 -- 172.21.15.32:0/2061610318 wait complete. 2024-04-16T05:44:44.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:44 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3059180940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:44.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:44:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 -- 172.21.15.32:0/4115876692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1550101710 msgr2=0x7f155010e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 --2- 172.21.15.32:0/4115876692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1550101710 0x7f155010e460 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f1540009900 tx=0x7f154002bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 -- 172.21.15.32:0/4115876692 shutdown_connections 2024-04-16T05:44:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 --2- 172.21.15.32:0/4115876692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f155010e9a0 0x7f1550110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 --2- 172.21.15.32:0/4115876692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1550101710 0x7f155010e460 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 --2- 172.21.15.32:0/4115876692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1550100df0 0x7f15501011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 -- 172.21.15.32:0/4115876692 >> 172.21.15.32:0/4115876692 conn(0x7f15500780b0 msgr2=0x7f15500784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 -- 172.21.15.32:0/4115876692 shutdown_connections 2024-04-16T05:44:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 -- 172.21.15.32:0/4115876692 wait complete. 2024-04-16T05:44:45.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 Processor -- start 2024-04-16T05:44:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 -- start start 2024-04-16T05:44:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.074+0000 7f1557aeb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1550100df0 0x7f15501a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1557aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1550101710 0x7f15501a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1557aeb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155010e9a0 0x7f155019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1557aeb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15501145b0 con 0x7f155010e9a0 2024-04-16T05:44:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1557aeb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1550114430 con 0x7f1550101710 2024-04-16T05:44:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1557aeb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1550114730 con 0x7f1550100df0 2024-04-16T05:44:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155010e9a0 0x7f155019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155010e9a0 0x7f155019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34934/0 (socket says 172.21.15.32:34934) 2024-04-16T05:44:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 -- 172.21.15.32:0/1373547097 learned_addr learned my addr 172.21.15.32:0/1373547097 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1555887640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1550100df0 0x7f15501a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1555086640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1550101710 0x7f15501a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 -- 172.21.15.32:0/1373547097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1550100df0 msgr2=0x7f15501a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1550100df0 0x7f15501a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 -- 172.21.15.32:0/1373547097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1550101710 msgr2=0x7f15501a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1550101710 0x7f15501a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 -- 172.21.15.32:0/1373547097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155019b050 con 0x7f155010e9a0 2024-04-16T05:44:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1555086640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1550101710 0x7f15501a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1556088640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155010e9a0 0x7f155019a8d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f154800e9d0 tx=0x7f154800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1546ffd640 1 -- 172.21.15.32:0/1373547097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154800cd80 con 0x7f155010e9a0 2024-04-16T05:44:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1557aeb640 1 -- 172.21.15.32:0/1373547097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155019b340 con 0x7f155010e9a0 2024-04-16T05:44:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1546ffd640 1 -- 172.21.15.32:0/1373547097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1548005020 con 0x7f155010e9a0 2024-04-16T05:44:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1546ffd640 1 -- 172.21.15.32:0/1373547097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480116d0 con 0x7f155010e9a0 2024-04-16T05:44:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.075+0000 7f1557aeb640 1 -- 172.21.15.32:0/1373547097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15501a75e0 con 0x7f155010e9a0 2024-04-16T05:44:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.077+0000 7f1546ffd640 1 -- 172.21.15.32:0/1373547097 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f15480051d0 con 0x7f155010e9a0 2024-04-16T05:44:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.078+0000 7f155a7b4740 1 Processor -- start 2024-04-16T05:44:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.078+0000 7f155a7b4740 1 -- start start 2024-04-16T05:44:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.078+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 0x55d738e77240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.078+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 0x55d738e7fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.078+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e82750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.078+0000 7f155a7b4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d738ddc0e0 con 0x55d738e80360 2024-04-16T05:44:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.078+0000 7f155a7b4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d738d79520 con 0x55d738e76e60 2024-04-16T05:44:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.078+0000 7f155a7b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d738dd80a0 con 0x55d738e77780 2024-04-16T05:44:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1546ffd640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f153c0a3d20 0x7f153c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e82750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e82750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34940/0 (socket says 172.21.15.32:34940) 2024-04-16T05:44:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 -- 172.21.15.32:0/4073932106 learned_addr learned my addr 172.21.15.32:0/4073932106 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555887640 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 0x55d738e7fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1556088640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f153c0a3d20 0x7f153c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 0x55d738e77240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1546ffd640 1 -- 172.21.15.32:0/1373547097 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f15480d11f0 con 0x7f155010e9a0 2024-04-16T05:44:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 -- 172.21.15.32:0/4073932106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 msgr2=0x55d738e7fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 0x55d738e7fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 -- 172.21.15.32:0/4073932106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 msgr2=0x55d738e77240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 0x55d738e77240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 -- 172.21.15.32:0/4073932106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d738da18d0 con 0x55d738e80360 2024-04-16T05:44:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1555086640 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e82750 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1540035000 tx=0x7f154002cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba5bc200d309583 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f1556088640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f153c0a3d20 0x7f153c0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1548099770 tx=0x7f1548005b40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f153bfff640 1 -- 172.21.15.32:0/4073932106 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540047070 con 0x55d738e80360 2024-04-16T05:44:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f153bfff640 1 -- 172.21.15.32:0/4073932106 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1540004030 con 0x55d738e80360 2024-04-16T05:44:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.079+0000 7f153bfff640 1 -- 172.21.15.32:0/4073932106 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540003c50 con 0x55d738e80360 2024-04-16T05:44:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 -- 172.21.15.32:0/4073932106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 msgr2=0x55d738e82750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e82750 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1540035000 tx=0x7f154002cf90 comp rx=0 tx=0).stop 2024-04-16T05:44:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 -- 172.21.15.32:0/4073932106 shutdown_connections 2024-04-16T05:44:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e82750 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 0x55d738e7fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 --2- 172.21.15.32:0/4073932106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 0x55d738e77240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 -- 172.21.15.32:0/4073932106 >> 172.21.15.32:0/4073932106 conn(0x55d738e71590 msgr2=0x55d738e71970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 -- 172.21.15.32:0/4073932106 shutdown_connections 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.080+0000 7f155a7b4740 1 -- 172.21.15.32:0/4073932106 wait complete. 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 1 -- start start 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 1 Processor -- start 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 0x55d738e7d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 0x55d738e7d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e7def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d738e82fa0 con 0x55d738e80360 2024-04-16T05:44:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d738d954d0 con 0x55d738e76e60 2024-04-16T05:44:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d738e83120 con 0x55d738e77780 2024-04-16T05:44:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f155a7b4740 10 client.0 _ll_register_callbacks cb 0x55d738d7a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f1555086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e7def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f1555086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e7def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34944/0 (socket says 172.21.15.32:34944) 2024-04-16T05:44:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f1555086640 1 -- 172.21.15.32:0/3242652379 learned_addr learned my addr 172.21.15.32:0/3242652379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f1555887640 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 0x55d738e7d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.081+0000 7f1555086640 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 0x55d738e7d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f1555086640 1 -- 172.21.15.32:0/3242652379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 msgr2=0x55d738e7d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f1555086640 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 0x55d738e7d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f1555086640 1 -- 172.21.15.32:0/3242652379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 msgr2=0x55d738e7d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f1555086640 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 0x55d738e7d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f1555086640 1 -- 172.21.15.32:0/3242652379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d738aee3f0 con 0x55d738e80360 2024-04-16T05:44:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f1555086640 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e7def0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1540003730 tx=0x7f1540003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb58d8c0be9c8a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f15397fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f15397fa640 1 -- 172.21.15.32:0/3242652379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540047040 con 0x55d738e80360 2024-04-16T05:44:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f15397fa640 1 -- 172.21.15.32:0/3242652379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f1540003ce0 con 0x55d738e80360 2024-04-16T05:44:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f155a7b4740 1 -- 172.21.15.32:0/3242652379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1540004030 con 0x55d738e80360 2024-04-16T05:44:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f15397fa640 1 -- 172.21.15.32:0/3242652379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1540040460 con 0x55d738e80360 2024-04-16T05:44:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f155a7b4740 10 client.17901 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f155a7b4740 1 -- 172.21.15.32:0/3242652379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d738bb87d0 con 0x55d738e80360 2024-04-16T05:44:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.082+0000 7f15397fa640 1 -- 172.21.15.32:0/3242652379 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1540004030 con 0x55d738e80360 2024-04-16T05:44:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f15397fa640 1 -- 172.21.15.32:0/3242652379 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f154004d030 con 0x55d738e80360 2024-04-16T05:44:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f155a7b4740 10 client.17901 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f155a7b4740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f155a7b4740 20 client.17901 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d738e901f0 0x55d738e925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f155a7b4740 4 client.17901 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f155a7b4740 1 -- 172.21.15.32:0/3242652379 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d738de9b00 con 0x55d738e901f0 2024-04-16T05:44:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f1555887640 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d738e901f0 0x55d738e925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f15397fa640 10 client.17901 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.083+0000 7f1555887640 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d738e901f0 0x55d738e925e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185a4bd53ee98d8 server_cookie=5d83c23d747d04cc in_seq=0 out_seq=0 2024-04-16T05:44:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f15397fa640 1 -- 172.21.15.32:0/3242652379 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d738de9b00 con 0x55d738e901f0 2024-04-16T05:44:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f15397fa640 10 client.17901 handle_command_reply: tid=0 2024-04-16T05:44:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 client.17901 shutdown 2024-04-16T05:44:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 -- 172.21.15.32:0/3242652379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 msgr2=0x55d738e7def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e7def0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1540003730 tx=0x7f1540003830 comp rx=0 tx=0).stop 2024-04-16T05:44:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 -- 172.21.15.32:0/3242652379 shutdown_connections 2024-04-16T05:44:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e7def0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d738e901f0 0x55d738e925e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e77780 0x55d738e7d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3242652379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e76e60 0x55d738e7d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 -- 172.21.15.32:0/3242652379 >> 172.21.15.32:0/3242652379 conn(0x55d738e71590 msgr2=0x55d738e732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f15397fa640 0 client.17901 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 -- 172.21.15.32:0/3242652379 shutdown_connections 2024-04-16T05:44:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 1 -- 172.21.15.32:0/3242652379 wait complete. 2024-04-16T05:44:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.084+0000 7f155a7b4740 20 client.17901 trim_cache size 0 max 16384 2024-04-16T05:44:45.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f155a7b4740 1 Processor -- start 2024-04-16T05:44:45.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f155a7b4740 1 -- start start 2024-04-16T05:44:45.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e76e60 0x55d738e7fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 0x55d738e6c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e80360 0x55d738e97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f155a7b4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15501b3f00 con 0x55d738e76e60 2024-04-16T05:44:45.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f155a7b4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f15501b17d0 con 0x55d738e77780 2024-04-16T05:44:45.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f155a7b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15501af640 con 0x55d738e80360 2024-04-16T05:44:45.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e76e60 0x55d738e7fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e76e60 0x55d738e7fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34948/0 (socket says 172.21.15.32:34948) 2024-04-16T05:44:45.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 -- 172.21.15.32:0/717195187 learned_addr learned my addr 172.21.15.32:0/717195187 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:45.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 -- 172.21.15.32:0/717195187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e80360 msgr2=0x55d738e97be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:45.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 --2- 172.21.15.32:0/717195187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e80360 0x55d738e97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 -- 172.21.15.32:0/717195187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 msgr2=0x55d738e6c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:45.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 --2- 172.21.15.32:0/717195187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 0x55d738e6c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 -- 172.21.15.32:0/717195187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d738e92950 con 0x55d738e76e60 2024-04-16T05:44:45.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f1555086640 1 --2- 172.21.15.32:0/717195187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e76e60 0x55d738e7fba0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f154002bc70 tx=0x7f154002cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7d13ed275abb77 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:45.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f153b7fe640 1 -- 172.21.15.32:0/717195187 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540047070 con 0x55d738e76e60 2024-04-16T05:44:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f153b7fe640 1 -- 172.21.15.32:0/717195187 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154005e070 con 0x55d738e76e60 2024-04-16T05:44:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.098+0000 7f153b7fe640 1 -- 172.21.15.32:0/717195187 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540009520 con 0x55d738e76e60 2024-04-16T05:44:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 -- 172.21.15.32:0/717195187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e76e60 msgr2=0x55d738e7fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 --2- 172.21.15.32:0/717195187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e76e60 0x55d738e7fba0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f154002bc70 tx=0x7f154002cf90 comp rx=0 tx=0).stop 2024-04-16T05:44:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 -- 172.21.15.32:0/717195187 shutdown_connections 2024-04-16T05:44:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 --2- 172.21.15.32:0/717195187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e80360 0x55d738e97be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 --2- 172.21.15.32:0/717195187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 0x55d738e6c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 --2- 172.21.15.32:0/717195187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e76e60 0x55d738e7fba0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 -- 172.21.15.32:0/717195187 >> 172.21.15.32:0/717195187 conn(0x55d738e71590 msgr2=0x55d738e71970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:45.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 -- 172.21.15.32:0/717195187 shutdown_connections 2024-04-16T05:44:45.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 -- 172.21.15.32:0/717195187 wait complete. 2024-04-16T05:44:45.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 -- start start 2024-04-16T05:44:45.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.099+0000 7f155a7b4740 1 Processor -- start 2024-04-16T05:44:45.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e76e60 0x55d738e88780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 0x55d738e83af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f155a7b4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e84030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f155a7b4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15501afc90 con 0x55d738e80360 2024-04-16T05:44:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f155a7b4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f15501b1100 con 0x55d738e77780 2024-04-16T05:44:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f155a7b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15501b0360 con 0x55d738e76e60 2024-04-16T05:44:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f155a7b4740 10 client.0 _ll_register_callbacks cb 0x7f1540045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e84030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e84030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34950/0 (socket says 172.21.15.32:34950) 2024-04-16T05:44:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 -- 172.21.15.32:0/3852175252 learned_addr learned my addr 172.21.15.32:0/3852175252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:45.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555887640 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 0x55d738e83af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e76e60 0x55d738e88780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 -- 172.21.15.32:0/3852175252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e76e60 msgr2=0x55d738e88780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e76e60 0x55d738e88780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 -- 172.21.15.32:0/3852175252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 msgr2=0x55d738e83af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 0x55d738e83af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 -- 172.21.15.32:0/3852175252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d738e84720 con 0x55d738e80360 2024-04-16T05:44:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f1555086640 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e84030 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1540009900 tx=0x7f1540052040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3eb48e0e850f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:45.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.100+0000 7f15397fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f15397fa640 1 -- 172.21.15.32:0/3852175252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540047070 con 0x55d738e80360 2024-04-16T05:44:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f15397fa640 1 -- 172.21.15.32:0/3852175252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f1540005260 con 0x55d738e80360 2024-04-16T05:44:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 1 -- 172.21.15.32:0/3852175252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154005e070 con 0x55d738e80360 2024-04-16T05:44:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f15397fa640 1 -- 172.21.15.32:0/3852175252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1540009520 con 0x55d738e80360 2024-04-16T05:44:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 10 client.17907 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 1 -- 172.21.15.32:0/3852175252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d738ef6350 con 0x55d738e80360 2024-04-16T05:44:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f15397fa640 1 -- 172.21.15.32:0/3852175252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154005e070 con 0x55d738e80360 2024-04-16T05:44:45.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f15397fa640 1 -- 172.21.15.32:0/3852175252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f154004d040 con 0x55d738e80360 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 10 client.17907 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 20 client.17907 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d738ef6860 0x55d738ef8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 4 client.17907 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f155a7b4740 1 -- 172.21.15.32:0/3852175252 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d738de3970 con 0x55d738ef6860 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.101+0000 7f1555887640 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d738ef6860 0x55d738ef8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f1555887640 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d738ef6860 0x55d738ef8c80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d57863ec4ff25352 server_cookie=3d3fd4295e05529f in_seq=0 out_seq=0 2024-04-16T05:44:45.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f15397fa640 10 client.17907 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f15397fa640 1 -- 172.21.15.32:0/3852175252 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d738de3970 con 0x55d738ef6860 2024-04-16T05:44:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f15397fa640 10 client.17907 handle_command_reply: tid=0 2024-04-16T05:44:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 client.17907 shutdown 2024-04-16T05:44:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 -- 172.21.15.32:0/3852175252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 msgr2=0x55d738e84030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e84030 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1540009900 tx=0x7f1540052040 comp rx=0 tx=0).stop 2024-04-16T05:44:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 -- 172.21.15.32:0/3852175252 shutdown_connections 2024-04-16T05:44:45.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d738e80360 0x55d738e84030 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d738e77780 0x55d738e83af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55d738ef6860 0x55d738ef8c80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 --2- 172.21.15.32:0/3852175252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d738e76e60 0x55d738e88780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 -- 172.21.15.32:0/3852175252 >> 172.21.15.32:0/3852175252 conn(0x55d738e71590 msgr2=0x55d738e6c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f15397fa640 0 client.17907 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 -- 172.21.15.32:0/3852175252 shutdown_connections 2024-04-16T05:44:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 1 -- 172.21.15.32:0/3852175252 wait complete. 2024-04-16T05:44:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.102+0000 7f155a7b4740 20 client.17907 trim_cache size 0 max 16384 2024-04-16T05:44:45.112 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:44:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 -- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f153c0a3d20 msgr2=0x7f153c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f153c0a3d20 0x7f153c0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1548099770 tx=0x7f1548005b40 comp rx=0 tx=0).stop 2024-04-16T05:44:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 -- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155010e9a0 msgr2=0x7f155019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155010e9a0 0x7f155019a8d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f154800e9d0 tx=0x7f154800ee90 comp rx=0 tx=0).stop 2024-04-16T05:44:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 -- 172.21.15.32:0/1373547097 shutdown_connections 2024-04-16T05:44:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f153c0a3d20 0x7f153c0a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155010e9a0 0x7f155019a8d0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1550101710 0x7f15501a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 --2- 172.21.15.32:0/1373547097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1550100df0 0x7f15501a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 -- 172.21.15.32:0/1373547097 >> 172.21.15.32:0/1373547097 conn(0x7f15500780b0 msgr2=0x7f155010ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 -- 172.21.15.32:0/1373547097 shutdown_connections 2024-04-16T05:44:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:45.103+0000 7f1557aeb640 1 -- 172.21.15.32:0/1373547097 wait complete. 2024-04-16T05:44:45.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:44 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3059180940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:45.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:44 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3059180940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:44:46.116 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T05:44:46.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:44:46.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T05:44:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:45 smithi134 ceph-mon[36153]: pgmap v303: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:46.146 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:45 smithi032 ceph-mon[32351]: pgmap v303: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:46.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:45 smithi082 ceph-mon[36195]: pgmap v303: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 -- 172.21.15.32:0/2525504664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 msgr2=0x7f951810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 --2- 172.21.15.32:0/2525504664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 0x7f951810dd30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9510009cd0 tx=0x7f951002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 -- 172.21.15.32:0/2525504664 shutdown_connections 2024-04-16T05:44:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 --2- 172.21.15.32:0/2525504664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 0x7f951810dd30 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 --2- 172.21.15.32:0/2525504664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9518101250 0x7f95181016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 --2- 172.21.15.32:0/2525504664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95181070e0 0x7f95181074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 -- 172.21.15.32:0/2525504664 >> 172.21.15.32:0/2525504664 conn(0x7f9518076d00 msgr2=0x7f9518077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 -- 172.21.15.32:0/2525504664 shutdown_connections 2024-04-16T05:44:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.241+0000 7f951e332640 1 -- 172.21.15.32:0/2525504664 wait complete. 2024-04-16T05:44:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f951e332640 1 Processor -- start 2024-04-16T05:44:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f951e332640 1 -- start start 2024-04-16T05:44:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f951e332640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9518101250 0x7f951819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f951e332640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 0x7f951819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f951e332640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181070e0 0x7f9518198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f951e332640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9518112460 con 0x7f9518101bf0 2024-04-16T05:44:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f951e332640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95181122e0 con 0x7f9518101250 2024-04-16T05:44:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f951e332640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95181125e0 con 0x7f95181070e0 2024-04-16T05:44:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.242+0000 7f95177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 0x7f951819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 0x7f951819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34964/0 (socket says 172.21.15.32:34964) 2024-04-16T05:44:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95177fe640 1 -- 172.21.15.32:0/3941853328 learned_addr learned my addr 172.21.15.32:0/3941853328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f951c8cf640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181070e0 0x7f9518198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f9517fff640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9518101250 0x7f951819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95177fe640 1 -- 172.21.15.32:0/3941853328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181070e0 msgr2=0x7f9518198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95177fe640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181070e0 0x7f9518198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95177fe640 1 -- 172.21.15.32:0/3941853328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9518101250 msgr2=0x7f951819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95177fe640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9518101250 0x7f951819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95177fe640 1 -- 172.21.15.32:0/3941853328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9518198fc0 con 0x7f9518101bf0 2024-04-16T05:44:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f9517fff640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9518101250 0x7f951819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95177fe640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 0x7f951819eb30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f950c00d770 tx=0x7f950c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95157fa640 1 -- 172.21.15.32:0/3941853328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c017070 con 0x7f9518101bf0 2024-04-16T05:44:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f951e332640 1 -- 172.21.15.32:0/3941853328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95181992b0 con 0x7f9518101bf0 2024-04-16T05:44:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95157fa640 1 -- 172.21.15.32:0/3941853328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950c005500 con 0x7f9518101bf0 2024-04-16T05:44:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f95157fa640 1 -- 172.21.15.32:0/3941853328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c004e00 con 0x7f9518101bf0 2024-04-16T05:44:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.243+0000 7f951e332640 1 -- 172.21.15.32:0/3941853328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95181a5570 con 0x7f9518101bf0 2024-04-16T05:44:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.245+0000 7f95157fa640 1 -- 172.21.15.32:0/3941853328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f950c004fa0 con 0x7f9518101bf0 2024-04-16T05:44:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9520ff9740 1 Processor -- start 2024-04-16T05:44:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9520ff9740 1 -- start start 2024-04-16T05:44:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f95157fa640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95000a3d20 0x7f95000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x564967907870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 0x564967914e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649679153c0 0x5649679177b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9520ff9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56496786ce90 con 0x564967907490 2024-04-16T05:44:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9520ff9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56496780a4c0 con 0x564967907e40 2024-04-16T05:44:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9520ff9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564967868e50 con 0x5649679153c0 2024-04-16T05:44:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f951c8cf640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95000a3d20 0x7f95000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f95157fa640 1 -- 172.21.15.32:0/3941853328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f950c0d06c0 con 0x7f9518101bf0 2024-04-16T05:44:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9517fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x564967907870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9517fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x564967907870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34976/0 (socket says 172.21.15.32:34976) 2024-04-16T05:44:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.246+0000 7f9517fff640 1 -- 172.21.15.32:0/1517625305 learned_addr learned my addr 172.21.15.32:0/1517625305 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f95177fe640 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 0x564967914e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f951c8cf640 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649679153c0 0x5649679177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f9517fff640 1 -- 172.21.15.32:0/1517625305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649679153c0 msgr2=0x5649679177b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f9517fff640 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649679153c0 0x5649679177b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f9517fff640 1 -- 172.21.15.32:0/1517625305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 msgr2=0x564967914e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f9517fff640 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 0x564967914e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f9517fff640 1 -- 172.21.15.32:0/1517625305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649676477e0 con 0x564967907490 2024-04-16T05:44:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f951c8cf640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95000a3d20 0x7f95000a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9510006010 tx=0x7f951002ce00 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f9517fff640 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x564967907870 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9508009c30 tx=0x7f9508010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5861bef594d437 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f94fe7fc640 1 -- 172.21.15.32:0/1517625305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508004030 con 0x564967907490 2024-04-16T05:44:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f94fe7fc640 1 -- 172.21.15.32:0/1517625305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95080041d0 con 0x564967907490 2024-04-16T05:44:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.247+0000 7f94fe7fc640 1 -- 172.21.15.32:0/1517625305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95080126b0 con 0x564967907490 2024-04-16T05:44:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 -- 172.21.15.32:0/1517625305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 msgr2=0x564967907870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x564967907870 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9508009c30 tx=0x7f9508010740 comp rx=0 tx=0).stop 2024-04-16T05:44:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 -- 172.21.15.32:0/1517625305 shutdown_connections 2024-04-16T05:44:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649679153c0 0x5649679177b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 0x564967914e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 --2- 172.21.15.32:0/1517625305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x564967907870 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 -- 172.21.15.32:0/1517625305 >> 172.21.15.32:0/1517625305 conn(0x5649679021e0 msgr2=0x5649679025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 -- 172.21.15.32:0/1517625305 shutdown_connections 2024-04-16T05:44:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 -- 172.21.15.32:0/1517625305 wait complete. 2024-04-16T05:44:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.248+0000 7f9520ff9740 1 -- start start 2024-04-16T05:44:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9520ff9740 1 Processor -- start 2024-04-16T05:44:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 0x564967912430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 0x564967912970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5649679153c0 0x564967912eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9520ff9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564967918180 con 0x5649679153c0 2024-04-16T05:44:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9520ff9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5649677f5410 con 0x564967907e40 2024-04-16T05:44:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9520ff9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564967918300 con 0x564967907490 2024-04-16T05:44:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9520ff9740 10 client.0 _ll_register_callbacks cb 0x5649677f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5649679153c0 0x564967912eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5649679153c0 0x564967912eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34982/0 (socket says 172.21.15.32:34982) 2024-04-16T05:44:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 -- 172.21.15.32:0/2720052864 learned_addr learned my addr 172.21.15.32:0/2720052864 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f9517fff640 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 0x564967912430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f95177fe640 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 0x564967912970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 -- 172.21.15.32:0/2720052864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 msgr2=0x564967912430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 0x564967912430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 -- 172.21.15.32:0/2720052864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 msgr2=0x564967912970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 0x564967912970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 -- 172.21.15.32:0/2720052864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649676a7f70 con 0x5649679153c0 2024-04-16T05:44:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f951c8cf640 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5649679153c0 0x564967912eb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9510009cd0 tx=0x7f951002e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17f8bc6736872ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.249+0000 7f94f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f94f77fe640 1 -- 172.21.15.32:0/2720052864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510052060 con 0x5649679153c0 2024-04-16T05:44:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f94f77fe640 1 -- 172.21.15.32:0/2720052864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f9510003b70 con 0x5649679153c0 2024-04-16T05:44:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 1 -- 172.21.15.32:0/2720052864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95080041d0 con 0x5649679153c0 2024-04-16T05:44:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f94f77fe640 1 -- 172.21.15.32:0/2720052864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9510059050 con 0x5649679153c0 2024-04-16T05:44:46.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 10 client.17919 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 1 -- 172.21.15.32:0/2720052864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56496753f170 con 0x5649679153c0 2024-04-16T05:44:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f94f77fe640 1 -- 172.21.15.32:0/2720052864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95080041d0 con 0x5649679153c0 2024-04-16T05:44:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f94f77fe640 1 -- 172.21.15.32:0/2720052864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9510004350 con 0x5649679153c0 2024-04-16T05:44:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 10 client.17919 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 10 client.17919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 20 client.17919 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5649679257d0 0x564967927c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 4 client.17919 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.250+0000 7f9520ff9740 1 -- 172.21.15.32:0/2720052864 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56496787a8b0 con 0x5649679257d0 2024-04-16T05:44:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.251+0000 7f9517fff640 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5649679257d0 0x564967927c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.251+0000 7f94f77fe640 10 client.17919 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.251+0000 7f9517fff640 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5649679257d0 0x564967927c00 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30619d10ef97ced7 server_cookie=d650a916b9db11f9 in_seq=0 out_seq=0 2024-04-16T05:44:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f94f77fe640 1 -- 172.21.15.32:0/2720052864 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56496787a8b0 con 0x5649679257d0 2024-04-16T05:44:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f94f77fe640 10 client.17919 handle_command_reply: tid=0 2024-04-16T05:44:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 client.17919 shutdown 2024-04-16T05:44:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 -- 172.21.15.32:0/2720052864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5649679153c0 msgr2=0x564967912eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5649679153c0 0x564967912eb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9510009cd0 tx=0x7f951002e050 comp rx=0 tx=0).stop 2024-04-16T05:44:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 -- 172.21.15.32:0/2720052864 shutdown_connections 2024-04-16T05:44:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5649679153c0 0x564967912eb0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564967907e40 0x564967912970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5649679257d0 0x564967927c00 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2720052864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 0x564967912430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f94f77fe640 0 client.17919 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 -- 172.21.15.32:0/2720052864 >> 172.21.15.32:0/2720052864 conn(0x5649679021e0 msgr2=0x564967903f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 -- 172.21.15.32:0/2720052864 shutdown_connections 2024-04-16T05:44:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 1 -- 172.21.15.32:0/2720052864 wait complete. 2024-04-16T05:44:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.252+0000 7f9520ff9740 20 client.17919 trim_cache size 0 max 16384 2024-04-16T05:44:46.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9520ff9740 1 Processor -- start 2024-04-16T05:44:46.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9520ff9740 1 -- start start 2024-04-16T05:44:46.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 0x5649679210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907e40 0x56496792d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496792da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9520ff9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95181afb80 con 0x564967907e40 2024-04-16T05:44:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9520ff9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95181ad450 con 0x5649679153c0 2024-04-16T05:44:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9520ff9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95181ab2c0 con 0x564967907490 2024-04-16T05:44:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f95177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907e40 0x56496792d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f95177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907e40 0x56496792d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34990/0 (socket says 172.21.15.32:34990) 2024-04-16T05:44:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f95177fe640 1 -- 172.21.15.32:0/861153063 learned_addr learned my addr 172.21.15.32:0/861153063 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f9517fff640 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 0x5649679210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.270+0000 7f951c8cf640 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496792da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f95177fe640 1 -- 172.21.15.32:0/861153063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 msgr2=0x5649679210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f95177fe640 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 0x5649679210b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f95177fe640 1 -- 172.21.15.32:0/861153063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 msgr2=0x56496792da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f95177fe640 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496792da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f95177fe640 1 -- 172.21.15.32:0/861153063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649679113e0 con 0x564967907e40 2024-04-16T05:44:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f951c8cf640 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496792da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f95177fe640 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907e40 0x56496792d4d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f950c09c260 tx=0x7f950c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e93c50174881e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f94fdffb640 1 -- 172.21.15.32:0/861153063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c020040 con 0x564967907e40 2024-04-16T05:44:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f94fdffb640 1 -- 172.21.15.32:0/861153063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950c09c2d0 con 0x564967907e40 2024-04-16T05:44:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.271+0000 7f94fdffb640 1 -- 172.21.15.32:0/861153063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c0cdd00 con 0x564967907e40 2024-04-16T05:44:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 -- 172.21.15.32:0/861153063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907e40 msgr2=0x56496792d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907e40 0x56496792d4d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f950c09c260 tx=0x7f950c00ce60 comp rx=0 tx=0).stop 2024-04-16T05:44:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 -- 172.21.15.32:0/861153063 shutdown_connections 2024-04-16T05:44:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496792da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907e40 0x56496792d4d0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 --2- 172.21.15.32:0/861153063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907490 0x5649679210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 -- 172.21.15.32:0/861153063 >> 172.21.15.32:0/861153063 conn(0x5649679021e0 msgr2=0x564967901250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 -- 172.21.15.32:0/861153063 shutdown_connections 2024-04-16T05:44:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 -- 172.21.15.32:0/861153063 wait complete. 2024-04-16T05:44:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 -- start start 2024-04-16T05:44:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.272+0000 7f9520ff9740 1 Processor -- start 2024-04-16T05:44:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x56496791d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907e40 0x564967919b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9520ff9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496791a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9520ff9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95181ab910 con 0x564967907490 2024-04-16T05:44:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9520ff9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95181acd80 con 0x5649679153c0 2024-04-16T05:44:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9520ff9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95181abfe0 con 0x564967907e40 2024-04-16T05:44:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9520ff9740 10 client.0 _ll_register_callbacks cb 0x7f9510047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x56496791d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x56496791d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34994/0 (socket says 172.21.15.32:34994) 2024-04-16T05:44:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 -- 172.21.15.32:0/2873585433 learned_addr learned my addr 172.21.15.32:0/2873585433 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f95177fe640 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907e40 0x564967919b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f951c8cf640 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496791a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 -- 172.21.15.32:0/2873585433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907e40 msgr2=0x564967919b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907e40 0x564967919b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 -- 172.21.15.32:0/2873585433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 msgr2=0x56496791a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496791a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 -- 172.21.15.32:0/2873585433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56496791a630 con 0x564967907490 2024-04-16T05:44:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9517fff640 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x56496791d760 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9508003cb0 tx=0x7f9508009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a55ea6713cae0a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f94f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f94f77fe640 1 -- 172.21.15.32:0/2873585433 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950802d030 con 0x564967907490 2024-04-16T05:44:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f94f77fe640 1 -- 172.21.15.32:0/2873585433 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f95080053f0 con 0x564967907490 2024-04-16T05:44:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f9520ff9740 1 -- 172.21.15.32:0/2873585433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f950c09c2d0 con 0x564967907490 2024-04-16T05:44:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.273+0000 7f94f77fe640 1 -- 172.21.15.32:0/2873585433 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9508037050 con 0x564967907490 2024-04-16T05:44:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f9520ff9740 10 client.17925 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f9520ff9740 1 -- 172.21.15.32:0/2873585433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56496798cb60 con 0x564967907490 2024-04-16T05:44:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f94f77fe640 1 -- 172.21.15.32:0/2873585433 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950c09c2d0 con 0x564967907490 2024-04-16T05:44:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f94f77fe640 1 -- 172.21.15.32:0/2873585433 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f950800f040 con 0x564967907490 2024-04-16T05:44:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f9520ff9740 10 client.17925 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f9520ff9740 10 client.17925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f9520ff9740 20 client.17925 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56496798d000 0x56496798f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f9520ff9740 4 client.17925 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T05:44:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f9520ff9740 1 -- 172.21.15.32:0/2873585433 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564967874720 con 0x56496798d000 2024-04-16T05:44:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f95177fe640 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56496798d000 0x56496798f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f95177fe640 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56496798d000 0x56496798f420 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767ab47e09d66832 server_cookie=20f2304af4b8ba20 in_seq=0 out_seq=0 2024-04-16T05:44:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.274+0000 7f94f77fe640 10 client.17925 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.275+0000 7f94f77fe640 1 -- 172.21.15.32:0/2873585433 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564967874720 con 0x56496798d000 2024-04-16T05:44:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.275+0000 7f94f77fe640 10 client.17925 handle_command_reply: tid=0 2024-04-16T05:44:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.275+0000 7f9520ff9740 1 client.17925 shutdown 2024-04-16T05:44:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 -- 172.21.15.32:0/2873585433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 msgr2=0x56496791d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x56496791d760 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9508003cb0 tx=0x7f9508009570 comp rx=0 tx=0).stop 2024-04-16T05:44:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 -- 172.21.15.32:0/2873585433 shutdown_connections 2024-04-16T05:44:46.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649679153c0 0x56496791a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56496798d000 0x56496798f420 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564967907e40 0x564967919b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f94f77fe640 0 client.17925 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 --2- 172.21.15.32:0/2873585433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564967907490 0x56496791d760 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 -- 172.21.15.32:0/2873585433 >> 172.21.15.32:0/2873585433 conn(0x5649679021e0 msgr2=0x564967903630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 -- 172.21.15.32:0/2873585433 shutdown_connections 2024-04-16T05:44:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 1 -- 172.21.15.32:0/2873585433 wait complete. 2024-04-16T05:44:46.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f9520ff9740 20 client.17925 trim_cache size 0 max 16384 2024-04-16T05:44:46.285 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:44:46.285 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-16T05:44:46.285 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "c8e4b734-12f3-4505-8240-fd6bbde81b9b", 2024-04-16T05:44:46.285 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-16T05:44:46.285 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:44:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f951e332640 1 -- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95000a3d20 msgr2=0x7f95000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f951e332640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95000a3d20 0x7f95000a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9510006010 tx=0x7f951002ce00 comp rx=0 tx=0).stop 2024-04-16T05:44:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f951e332640 1 -- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 msgr2=0x7f951819eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.276+0000 7f951e332640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 0x7f951819eb30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f950c00d770 tx=0x7f950c00dc30 comp rx=0 tx=0).stop 2024-04-16T05:44:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.277+0000 7f951e332640 1 -- 172.21.15.32:0/3941853328 shutdown_connections 2024-04-16T05:44:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.277+0000 7f951e332640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181070e0 0x7f9518198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.277+0000 7f951e332640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f95000a3d20 0x7f95000a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.277+0000 7f951e332640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9518101bf0 0x7f951819eb30 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.277+0000 7f951e332640 1 --2- 172.21.15.32:0/3941853328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9518101250 0x7f951819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.277+0000 7f951e332640 1 -- 172.21.15.32:0/3941853328 >> 172.21.15.32:0/3941853328 conn(0x7f9518076d00 msgr2=0x7f95180ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.277+0000 7f951e332640 1 -- 172.21.15.32:0/3941853328 shutdown_connections 2024-04-16T05:44:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.277+0000 7f951e332640 1 -- 172.21.15.32:0/3941853328 wait complete. 2024-04-16T05:44:46.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:44:46.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:44:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.436+0000 7fcf4c29a640 1 -- 172.21.15.32:0/4285509972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 msgr2=0x7fcf44101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.436+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/4285509972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 0x7fcf44101530 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009900 tx=0x7fcf4002bc30 comp rx=0 tx=0).stop 2024-04-16T05:44:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 -- 172.21.15.32:0/4285509972 shutdown_connections 2024-04-16T05:44:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/4285509972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf44106220 0x7fcf4410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/4285509972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf44101b00 0x7fcf44105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/4285509972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 0x7fcf44101530 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 -- 172.21.15.32:0/4285509972 >> 172.21.15.32:0/4285509972 conn(0x7fcf44077f60 msgr2=0x7fcf44078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 -- 172.21.15.32:0/4285509972 shutdown_connections 2024-04-16T05:44:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 -- 172.21.15.32:0/4285509972 wait complete. 2024-04-16T05:44:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 Processor -- start 2024-04-16T05:44:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 -- start start 2024-04-16T05:44:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 0x7fcf4419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf44106220 0x7fcf4419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf44197590 0x7fcf44197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441112d0 con 0x7fcf44101150 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44111150 con 0x7fcf44197590 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.437+0000 7fcf4c29a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44111450 con 0x7fcf44106220 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 0x7fcf4419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 0x7fcf4419d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35016/0 (socket says 172.21.15.32:35016) 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 -- 172.21.15.32:0/302079090 learned_addr learned my addr 172.21.15.32:0/302079090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf49835640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf44106220 0x7fcf4419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a837640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf44197590 0x7fcf44197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 -- 172.21.15.32:0/302079090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf44106220 msgr2=0x7fcf4419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf44106220 0x7fcf4419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 -- 172.21.15.32:0/302079090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf44197590 msgr2=0x7fcf44197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf44197590 0x7fcf44197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 -- 172.21.15.32:0/302079090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf441980b0 con 0x7fcf44101150 2024-04-16T05:44:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a837640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf44197590 0x7fcf44197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:44:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4a036640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 0x7fcf4419d3e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcf40006010 tx=0x7fcf4002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf3b7fe640 1 -- 172.21.15.32:0/302079090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40041070 con 0x7fcf44101150 2024-04-16T05:44:46.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf3b7fe640 1 -- 172.21.15.32:0/302079090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf40004450 con 0x7fcf44101150 2024-04-16T05:44:46.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4c29a640 1 -- 172.21.15.32:0/302079090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf44198340 con 0x7fcf44101150 2024-04-16T05:44:46.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf3b7fe640 1 -- 172.21.15.32:0/302079090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40034440 con 0x7fcf44101150 2024-04-16T05:44:46.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.438+0000 7fcf4c29a640 1 -- 172.21.15.32:0/302079090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441a4530 con 0x7fcf44101150 2024-04-16T05:44:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.441+0000 7fcf3b7fe640 1 -- 172.21.15.32:0/302079090 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf40044070 con 0x7fcf44101150 2024-04-16T05:44:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.441+0000 7fcf4ef65740 1 Processor -- start 2024-04-16T05:44:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4ef65740 1 -- start start 2024-04-16T05:44:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a2db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55895a235180 con 0x55895a2d9350 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55895a1d2360 con 0x55895a2d0770 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55895a231140 con 0x55895a2cfe50 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf49835640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a2db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf49835640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a2db740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35022/0 (socket says 172.21.15.32:35022) 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf49835640 1 -- 172.21.15.32:0/2622173677 learned_addr learned my addr 172.21.15.32:0/2622173677 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4a837640 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4a036640 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf3b7fe640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcf300a3d20 0x7fcf300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4a837640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcf300a3d20 0x7fcf300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf49835640 1 -- 172.21.15.32:0/2622173677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 msgr2=0x55895a2d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf49835640 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf49835640 1 -- 172.21.15.32:0/2622173677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 msgr2=0x55895a2d8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf49835640 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2d8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf49835640 1 -- 172.21.15.32:0/2622173677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895a0117d0 con 0x55895a2d9350 2024-04-16T05:44:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf4a036640 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.442+0000 7fcf3b7fe640 1 -- 172.21.15.32:0/302079090 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fcf400f2740 con 0x7fcf44101150 2024-04-16T05:44:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.443+0000 7fcf49835640 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a2db740 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400bc20 tx=0x7fcf3400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66975ae11b6c62f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.443+0000 7fcf4a837640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcf300a3d20 0x7fcf300a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c005cc0 tx=0x7fcf3c004ab0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.443+0000 7fcf38bfe640 1 -- 172.21.15.32:0/2622173677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34015070 con 0x55895a2d9350 2024-04-16T05:44:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.443+0000 7fcf38bfe640 1 -- 172.21.15.32:0/2622173677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3401a940 con 0x55895a2d9350 2024-04-16T05:44:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.443+0000 7fcf38bfe640 1 -- 172.21.15.32:0/2622173677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011940 con 0x55895a2d9350 2024-04-16T05:44:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.443+0000 7fcf4ef65740 1 -- 172.21.15.32:0/2622173677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 msgr2=0x55895a2db740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a2db740 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400bc20 tx=0x7fcf3400f730 comp rx=0 tx=0).stop 2024-04-16T05:44:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 -- 172.21.15.32:0/2622173677 shutdown_connections 2024-04-16T05:44:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a2db740 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/2622173677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 -- 172.21.15.32:0/2622173677 >> 172.21.15.32:0/2622173677 conn(0x55895a2ca580 msgr2=0x55895a2ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 -- 172.21.15.32:0/2622173677 shutdown_connections 2024-04-16T05:44:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 -- 172.21.15.32:0/2622173677 wait complete. 2024-04-16T05:44:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 -- start start 2024-04-16T05:44:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 Processor -- start 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2d68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2d6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55895a2dc140 con 0x55895a2d0770 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55895a1ee2c0 con 0x55895a2d9350 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55895a2dc2c0 con 0x55895a2cfe50 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf4ef65740 10 client.0 _ll_register_callbacks cb 0x55895a1d5090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf49835640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2d68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf49835640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2d68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35036/0 (socket says 172.21.15.32:35036) 2024-04-16T05:44:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.444+0000 7fcf49835640 1 -- 172.21.15.32:0/1705325411 learned_addr learned my addr 172.21.15.32:0/1705325411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf4a837640 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2d6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf4a036640 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf49835640 1 -- 172.21.15.32:0/1705325411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 msgr2=0x55895a2d63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf49835640 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf49835640 1 -- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 msgr2=0x55895a2d6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf49835640 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2d6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf49835640 1 -- 172.21.15.32:0/1705325411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558959f473f0 con 0x55895a2d0770 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf4a837640 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf49835640 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2d68f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fcf34010010 tx=0x7fcf3400b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f39822f143e6253 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1705325411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34015030 con 0x55895a2d0770 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1705325411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fcf3401aaf0 con 0x55895a2d0770 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1705325411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3401a940 con 0x55895a2d0770 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1705325411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf3402d050 con 0x55895a2d0770 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf4ef65740 10 client.17937 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1705325411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558959f070f0 con 0x55895a2d0770 2024-04-16T05:44:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.445+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1705325411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3401a940 con 0x55895a2d0770 2024-04-16T05:44:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1705325411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf34011ac0 con 0x55895a2d0770 2024-04-16T05:44:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf4ef65740 10 client.17937 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf4ef65740 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf4ef65740 20 client.17937 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55895a2e96a0 0x55895a2eba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf4ef65740 4 client.17937 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:44:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1705325411 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895a242ba0 con 0x55895a2e96a0 2024-04-16T05:44:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf4a036640 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55895a2e96a0 0x55895a2eba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf2dffb640 10 client.17937 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.446+0000 7fcf4a036640 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55895a2e96a0 0x55895a2eba90 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98546aeb3b47504 server_cookie=7983746de495b6af in_seq=0 out_seq=0 2024-04-16T05:44:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1705325411 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55895a242ba0 con 0x55895a2e96a0 2024-04-16T05:44:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf2dffb640 10 client.17937 handle_command_reply: tid=0 2024-04-16T05:44:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 client.17937 shutdown 2024-04-16T05:44:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1705325411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 msgr2=0x55895a2d68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2d68f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fcf34010010 tx=0x7fcf3400b970 comp rx=0 tx=0).stop 2024-04-16T05:44:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1705325411 shutdown_connections 2024-04-16T05:44:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2d68f0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55895a2e96a0 0x55895a2eba90 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1705325411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf2dffb640 0 client.17937 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1705325411 >> 172.21.15.32:0/1705325411 conn(0x55895a2ca580 msgr2=0x55895a2cc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1705325411 shutdown_connections 2024-04-16T05:44:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1705325411 wait complete. 2024-04-16T05:44:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.447+0000 7fcf4ef65740 20 client.17937 trim_cache size 0 max 16384 2024-04-16T05:44:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4ef65740 1 Processor -- start 2024-04-16T05:44:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4ef65740 1 -- start start 2024-04-16T05:44:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2e5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441b0b50 con 0x55895a2d0770 2024-04-16T05:44:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441ae420 con 0x55895a2d9350 2024-04-16T05:44:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441ac290 con 0x55895a2cfe50 2024-04-16T05:44:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf49835640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf49835640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35046/0 (socket says 172.21.15.32:35046) 2024-04-16T05:44:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf49835640 1 -- 172.21.15.32:0/3821596794 learned_addr learned my addr 172.21.15.32:0/3821596794 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.465+0000 7fcf4a837640 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf4a036640 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf49835640 1 -- 172.21.15.32:0/3821596794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 msgr2=0x55895a2d8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf49835640 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf49835640 1 -- 172.21.15.32:0/3821596794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 msgr2=0x55895a2e5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf49835640 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2e5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf49835640 1 -- 172.21.15.32:0/3821596794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895a2d5360 con 0x55895a2d0770 2024-04-16T05:44:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf49835640 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2c55f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400f770 tx=0x7fcf34011510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b541b15f02c3a993 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf2ffff640 1 -- 172.21.15.32:0/3821596794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34036070 con 0x55895a2d0770 2024-04-16T05:44:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf2ffff640 1 -- 172.21.15.32:0/3821596794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3400b770 con 0x55895a2d0770 2024-04-16T05:44:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.466+0000 7fcf2ffff640 1 -- 172.21.15.32:0/3821596794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34015040 con 0x55895a2d0770 2024-04-16T05:44:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- 172.21.15.32:0/3821596794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 msgr2=0x55895a2c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2c55f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400f770 tx=0x7fcf34011510 comp rx=0 tx=0).stop 2024-04-16T05:44:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- 172.21.15.32:0/3821596794 shutdown_connections 2024-04-16T05:44:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d9350 0x55895a2e5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d0770 0x55895a2c55f0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/3821596794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- 172.21.15.32:0/3821596794 >> 172.21.15.32:0/3821596794 conn(0x55895a2ca580 msgr2=0x55895a2c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- 172.21.15.32:0/3821596794 shutdown_connections 2024-04-16T05:44:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- 172.21.15.32:0/3821596794 wait complete. 2024-04-16T05:44:46.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- start start 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 Processor -- start 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2df3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a352cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441ac8e0 con 0x55895a2d9350 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441add50 con 0x55895a2d0770 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441acfb0 con 0x55895a2cfe50 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4ef65740 10 client.0 _ll_register_callbacks cb 0x7fcf3400ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:44:46.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4a837640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a352cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4a837640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a352cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35054/0 (socket says 172.21.15.32:35054) 2024-04-16T05:44:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.467+0000 7fcf4a837640 1 -- 172.21.15.32:0/1112863577 learned_addr learned my addr 172.21.15.32:0/1112863577 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:44:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf49835640 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4a036640 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2df3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4a837640 1 -- 172.21.15.32:0/1112863577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 msgr2=0x55895a2df3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4a837640 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2df3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4a837640 1 -- 172.21.15.32:0/1112863577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 msgr2=0x55895a2df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4a837640 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4a837640 1 -- 172.21.15.32:0/1112863577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895a353230 con 0x55895a2d9350 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4a837640 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a352cf0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00dc50 tx=0x7fcf3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a56590dacb5567 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1112863577 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c028640 con 0x55895a2d9350 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1112863577 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fcf3c0273e0 con 0x55895a2d9350 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1112863577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3400b770 con 0x55895a2d9350 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1112863577 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf3c0287e0 con 0x55895a2d9350 2024-04-16T05:44:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4ef65740 10 client.17943 fetch_fsmap learned FSMap version 25 2024-04-16T05:44:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1112863577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55895a354440 con 0x55895a2d9350 2024-04-16T05:44:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.468+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1112863577 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3400b770 con 0x55895a2d9350 2024-04-16T05:44:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1112863577 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf3c026070 con 0x55895a2d9350 2024-04-16T05:44:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 10 client.17943 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:44:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 10 client.17943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:44:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 20 client.17943 populate_metadata read hostname 'smithi032' 2024-04-16T05:44:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55895a354c40 0x55895a357060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:44:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 4 client.17943 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1112863577 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55895a23ca10 con 0x55895a354c40 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4a036640 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55895a354c40 0x55895a357060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4a036640 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55895a354c40 0x55895a357060 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72991c5379efca9 server_cookie=9ae5beb3d701a676 in_seq=0 out_seq=0 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf2dffb640 10 client.17943 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf2dffb640 1 -- 172.21.15.32:0/1112863577 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55895a23ca10 con 0x55895a354c40 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf2dffb640 10 client.17943 handle_command_reply: tid=0 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 1 client.17943 shutdown 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1112863577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 msgr2=0x55895a352cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.469+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a352cf0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00dc50 tx=0x7fcf3c004a40 comp rx=0 tx=0).stop 2024-04-16T05:44:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1112863577 shutdown_connections 2024-04-16T05:44:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55895a2d9350 0x55895a352cf0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55895a2d0770 0x55895a2df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55895a354c40 0x55895a357060 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 1 --2- 172.21.15.32:0/1112863577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895a2cfe50 0x55895a2df3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf2dffb640 0 client.17943 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:44:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1112863577 >> 172.21.15.32:0/1112863577 conn(0x55895a2ca580 msgr2=0x55895a2d6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1112863577 shutdown_connections 2024-04-16T05:44:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 1 -- 172.21.15.32:0/1112863577 wait complete. 2024-04-16T05:44:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4ef65740 20 client.17943 trim_cache size 0 max 16384 2024-04-16T05:44:46.479 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:44:46.479 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T05:44:46.479 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-16T05:44:46.479 INFO:teuthology.orchestra.run.smithi032.stdout: "c8e4b734-12f3-4505-8240-fd6bbde81b9b": { 2024-04-16T05:44:46.480 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-16T05:44:46.480 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "c8e4b734-12f3-4505-8240-fd6bbde81b9b", 2024-04-16T05:44:46.480 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-04-16T05:44:46.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:44:46.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:44:46.480 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:44:46.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 -- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcf300a3d20 msgr2=0x7fcf300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcf300a3d20 0x7fcf300a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c005cc0 tx=0x7fcf3c004ab0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 -- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 msgr2=0x7fcf4419d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:44:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 0x7fcf4419d3e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcf40006010 tx=0x7fcf4002cc70 comp rx=0 tx=0).stop 2024-04-16T05:44:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 -- 172.21.15.32:0/302079090 shutdown_connections 2024-04-16T05:44:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf44197590 0x7fcf44197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf44106220 0x7fcf4419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fcf300a3d20 0x7fcf300a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 --2- 172.21.15.32:0/302079090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf44101150 0x7fcf4419d3e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:44:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 -- 172.21.15.32:0/302079090 >> 172.21.15.32:0/302079090 conn(0x7fcf44077f60 msgr2=0x7fcf440feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:44:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 -- 172.21.15.32:0/302079090 shutdown_connections 2024-04-16T05:44:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:44:46.470+0000 7fcf4c29a640 1 -- 172.21.15.32:0/302079090 wait complete. 2024-04-16T05:44:46.483 INFO:tasks.cephfs.filesystem:scrub status for tag:c8e4b734-12f3-4505-8240-fd6bbde81b9b - {'path': '/', 'tag': 'c8e4b734-12f3-4505-8240-fd6bbde81b9b', 'options': 'recursive,force'} 2024-04-16T05:44:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:46 smithi134 ceph-mon[36153]: scrub queued for path: / 2024-04-16T05:44:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:46 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:46 smithi134 ceph-mon[36153]: scrub summary: active paths [/] 2024-04-16T05:44:47.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:46 smithi134 ceph-mon[36153]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:47.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:46 smithi082 ceph-mon[36195]: scrub queued for path: / 2024-04-16T05:44:47.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:46 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:47.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:46 smithi082 ceph-mon[36195]: scrub summary: active paths [/] 2024-04-16T05:44:47.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:46 smithi082 ceph-mon[36195]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:47.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:46 smithi032 ceph-mon[32351]: scrub queued for path: / 2024-04-16T05:44:47.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:46 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:47.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:46 smithi032 ceph-mon[32351]: scrub summary: active paths [/] 2024-04-16T05:44:47.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:46 smithi032 ceph-mon[32351]: scrub summary: idle+waiting paths [/] 2024-04-16T05:44:48.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:47 smithi134 ceph-mon[36153]: pgmap v304: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:48.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:47 smithi082 ceph-mon[36195]: pgmap v304: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:48.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:47 smithi032 ceph-mon[32351]: pgmap v304: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:44:50.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:49 smithi134 ceph-mon[36153]: pgmap v305: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:50.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:49 smithi082 ceph-mon[36195]: pgmap v305: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:50.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:49 smithi032 ceph-mon[32351]: pgmap v305: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:51.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:50 smithi134 ceph-mon[36153]: scrub summary: idle 2024-04-16T05:44:51.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:50 smithi082 ceph-mon[36195]: scrub summary: idle 2024-04-16T05:44:51.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:50 smithi032 ceph-mon[32351]: scrub summary: idle 2024-04-16T05:44:52.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:51 smithi134 ceph-mon[36153]: pgmap v306: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:52.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:51 smithi082 ceph-mon[36195]: pgmap v306: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:52.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:51 smithi032 ceph-mon[32351]: pgmap v306: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:54.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:53 smithi134 ceph-mon[36153]: pgmap v307: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:54.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:53 smithi082 ceph-mon[36195]: pgmap v307: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:54.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:53 smithi032 ceph-mon[32351]: pgmap v307: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:55 smithi134 ceph-mon[36153]: pgmap v308: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:56.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:55 smithi082 ceph-mon[36195]: pgmap v308: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:56.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:55 smithi032 ceph-mon[32351]: pgmap v308: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:57.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:56 smithi032 ceph-mon[32351]: pgmap v309: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:57.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:56 smithi134 ceph-mon[36153]: pgmap v309: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:57.399 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:56 smithi082 ceph-mon[36195]: pgmap v309: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T05:44:59.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:44:59 smithi134 ceph-mon[36153]: pgmap v310: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:59.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:44:59 smithi082 ceph-mon[36195]: pgmap v310: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:44:59.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:44:59 smithi032 ceph-mon[32351]: pgmap v310: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-16T05:45:01.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:01 smithi134 ceph-mon[36153]: pgmap v311: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:01.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:01 smithi082 ceph-mon[36195]: pgmap v311: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:01.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:01 smithi032 ceph-mon[32351]: pgmap v311: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:03.308 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-16T05:45:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 -- 172.21.15.32:0/3150277892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac101190 msgr2=0x7f9eac101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 --2- 172.21.15.32:0/3150277892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac101190 0x7f9eac101570 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009900 tx=0x7f9e9402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 -- 172.21.15.32:0/3150277892 shutdown_connections 2024-04-16T05:45:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 --2- 172.21.15.32:0/3150277892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac106260 0x7f9eac10de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 --2- 172.21.15.32:0/3150277892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac101b40 0x7f9eac105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 --2- 172.21.15.32:0/3150277892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac101190 0x7f9eac101570 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 -- 172.21.15.32:0/3150277892 >> 172.21.15.32:0/3150277892 conn(0x7f9eac077280 msgr2=0x7f9eac077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 -- 172.21.15.32:0/3150277892 shutdown_connections 2024-04-16T05:45:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.432+0000 7f9eb3182640 1 -- 172.21.15.32:0/3150277892 wait complete. 2024-04-16T05:45:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb3182640 1 Processor -- start 2024-04-16T05:45:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb3182640 1 -- start start 2024-04-16T05:45:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb3182640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac101190 0x7f9eac19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb3182640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac106260 0x7f9eac19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb3182640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac1986a0 0x7f9eac198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb3182640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac111320 con 0x7f9eac106260 2024-04-16T05:45:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb3182640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1111a0 con 0x7f9eac101190 2024-04-16T05:45:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb3182640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1114a0 con 0x7f9eac1986a0 2024-04-16T05:45:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac106260 0x7f9eac19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac106260 0x7f9eac19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42190/0 (socket says 172.21.15.32:42190) 2024-04-16T05:45:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eabfff640 1 -- 172.21.15.32:0/2790253062 learned_addr learned my addr 172.21.15.32:0/2790253062 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb0f1e640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac101190 0x7f9eac19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.433+0000 7f9eb171f640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac1986a0 0x7f9eac198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9eabfff640 1 -- 172.21.15.32:0/2790253062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac1986a0 msgr2=0x7f9eac198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9eabfff640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac1986a0 0x7f9eac198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9eabfff640 1 -- 172.21.15.32:0/2790253062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac101190 msgr2=0x7f9eac19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9eabfff640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac101190 0x7f9eac19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9eabfff640 1 -- 172.21.15.32:0/2790253062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac1992a0 con 0x7f9eac106260 2024-04-16T05:45:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9eabfff640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac106260 0x7f9eac19eb20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d770 tx=0x7f9e9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9ea9ffb640 1 -- 172.21.15.32:0/2790253062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c017070 con 0x7f9eac106260 2024-04-16T05:45:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9ea9ffb640 1 -- 172.21.15.32:0/2790253062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c009c30 con 0x7f9eac106260 2024-04-16T05:45:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac1a5380 con 0x7f9eac106260 2024-04-16T05:45:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9ea9ffb640 1 -- 172.21.15.32:0/2790253062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00f620 con 0x7f9eac106260 2024-04-16T05:45:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.434+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1a5780 con 0x7f9eac106260 2024-04-16T05:45:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.435+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac102660 con 0x7f9eac106260 2024-04-16T05:45:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.436+0000 7f9ea9ffb640 1 -- 172.21.15.32:0/2790253062 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e9c01a070 con 0x7f9eac106260 2024-04-16T05:45:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.437+0000 7f9ea9ffb640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9e980a3dc0 0x7f9e980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.437+0000 7f9eb0f1e640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9e980a3dc0 0x7f9e980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.437+0000 7f9ea9ffb640 1 -- 172.21.15.32:0/2790253062 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f9e9c0d0610 con 0x7f9eac106260 2024-04-16T05:45:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.438+0000 7f9eb0f1e640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9e980a3dc0 0x7f9e980a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e94006010 tx=0x7f9e94005dd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.441+0000 7f9ea9ffb640 1 -- 172.21.15.32:0/2790253062 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e9c009de0 con 0x7f9eac106260 2024-04-16T05:45:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:03 smithi134 ceph-mon[36153]: pgmap v312: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:45:03.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:03 smithi082 ceph-mon[36195]: pgmap v312: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:45:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.653+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9eac101b40 con 0x7f9e980a3dc0 2024-04-16T05:45:03.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.660+0000 7f9ea9ffb640 1 -- 172.21.15.32:0/2790253062 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9eac101b40 con 0x7f9e980a3dc0 2024-04-16T05:45:03.659 INFO:teuthology.orchestra.run.smithi032.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-16T00:00:00", "created": "2024-04-16T05:35:02", "first": "2024-04-16T05:36:00", "last": "2024-04-16T05:45:00", "last_pruned": "2024-04-16T05:45:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-16T05:45:03.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9e980a3dc0 msgr2=0x7f9e980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:03.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9e980a3dc0 0x7f9e980a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e94006010 tx=0x7f9e94005dd0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac106260 msgr2=0x7f9eac19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac106260 0x7f9eac19eb20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d770 tx=0x7f9e9c00dc30 comp rx=0 tx=0).stop 2024-04-16T05:45:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 shutdown_connections 2024-04-16T05:45:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac1986a0 0x7f9eac198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9e980a3dc0 0x7f9e980a6280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac106260 0x7f9eac19eb20 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 --2- 172.21.15.32:0/2790253062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac101190 0x7f9eac19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 >> 172.21.15.32:0/2790253062 conn(0x7f9eac077280 msgr2=0x7f9eac06aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 shutdown_connections 2024-04-16T05:45:03.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.663+0000 7f9eb3182640 1 -- 172.21.15.32:0/2790253062 wait complete. 2024-04-16T05:45:03.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:03 smithi032 ceph-mon[32351]: pgmap v312: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-16T05:45:03.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-16T05:45:03.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 -- 172.21.15.32:0/822458620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65406b1d0 msgr2=0x7ff65410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 --2- 172.21.15.32:0/822458620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65406b1d0 0x7ff65410eea0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009900 tx=0x7ff64c02bcd0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 -- 172.21.15.32:0/822458620 shutdown_connections 2024-04-16T05:45:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 --2- 172.21.15.32:0/822458620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff65406b1d0 0x7ff65410eea0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 --2- 172.21.15.32:0/822458620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65406a830 0x7ff65406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 --2- 172.21.15.32:0/822458620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff654108190 0x7ff654108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 -- 172.21.15.32:0/822458620 >> 172.21.15.32:0/822458620 conn(0x7ff654076d00 msgr2=0x7ff654077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 -- 172.21.15.32:0/822458620 shutdown_connections 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.780+0000 7ff658bbd640 1 -- 172.21.15.32:0/822458620 wait complete. 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff658bbd640 1 Processor -- start 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff658bbd640 1 -- start start 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff658bbd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65406a830 0x7ff654198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff658bbd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65406b1d0 0x7ff654198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff658bbd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff654108190 0x7ff65419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff658bbd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541123b0 con 0x7ff654108190 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff658bbd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff654112230 con 0x7ff65406a830 2024-04-16T05:45:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff658bbd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff654112530 con 0x7ff65406b1d0 2024-04-16T05:45:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff652d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff654108190 0x7ff65419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff652d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff654108190 0x7ff65419cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42204/0 (socket says 172.21.15.32:42204) 2024-04-16T05:45:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff652d9d640 1 -- 172.21.15.32:0/2424898 learned_addr learned my addr 172.21.15.32:0/2424898 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff651d9b640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65406b1d0 0x7ff654198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff65259c640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65406a830 0x7ff654198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff652d9d640 1 -- 172.21.15.32:0/2424898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65406b1d0 msgr2=0x7ff654198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff652d9d640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65406b1d0 0x7ff654198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff652d9d640 1 -- 172.21.15.32:0/2424898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65406a830 msgr2=0x7ff654198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff652d9d640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65406a830 0x7ff654198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.781+0000 7ff652d9d640 1 -- 172.21.15.32:0/2424898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65419d710 con 0x7ff654108190 2024-04-16T05:45:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.782+0000 7ff652d9d640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff654108190 0x7ff65419cfe0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff64c005940 tx=0x7ff64c02cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.782+0000 7ff6437fe640 1 -- 172.21.15.32:0/2424898 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c041070 con 0x7ff654108190 2024-04-16T05:45:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.782+0000 7ff6437fe640 1 -- 172.21.15.32:0/2424898 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64c0054e0 con 0x7ff654108190 2024-04-16T05:45:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.782+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65419d9a0 con 0x7ff654108190 2024-04-16T05:45:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.782+0000 7ff6437fe640 1 -- 172.21.15.32:0/2424898 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c003e50 con 0x7ff654108190 2024-04-16T05:45:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.782+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6541a5290 con 0x7ff654108190 2024-04-16T05:45:03.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.783+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff654103710 con 0x7ff654108190 2024-04-16T05:45:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.784+0000 7ff6437fe640 1 -- 172.21.15.32:0/2424898 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff64c044070 con 0x7ff654108190 2024-04-16T05:45:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.784+0000 7ff6437fe640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff63c0a3fe0 0x7ff63c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.785+0000 7ff6437fe640 1 -- 172.21.15.32:0/2424898 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7ff64c0f2b70 con 0x7ff654108190 2024-04-16T05:45:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.785+0000 7ff65259c640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff63c0a3fe0 0x7ff63c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.785+0000 7ff65259c640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff63c0a3fe0 0x7ff63c0a64a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff644005fd0 tx=0x7ff644009680 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.788+0000 7ff6437fe640 1 -- 172.21.15.32:0/2424898 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff64c0be940 con 0x7ff654108190 2024-04-16T05:45:03.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:03.998+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 --> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff6541004a0 con 0x7ff63c0a3fe0 2024-04-16T05:45:04.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.005+0000 7ff6437fe640 1 -- 172.21.15.32:0/2424898 <== mgr.15429 v2:172.21.15.32:6800/3496546430 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff6541004a0 con 0x7ff63c0a3fe0 2024-04-16T05:45:04.004 INFO:teuthology.orchestra.run.smithi032.stdout:/ 1m 6m3h 2024-04-16T05:45:04.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff63c0a3fe0 msgr2=0x7ff63c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:04.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff63c0a3fe0 0x7ff63c0a64a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff644005fd0 tx=0x7ff644009680 comp rx=0 tx=0).stop 2024-04-16T05:45:04.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff654108190 msgr2=0x7ff65419cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:04.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff654108190 0x7ff65419cfe0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff64c005940 tx=0x7ff64c02cd10 comp rx=0 tx=0).stop 2024-04-16T05:45:04.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 shutdown_connections 2024-04-16T05:45:04.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7ff63c0a3fe0 0x7ff63c0a64a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:04.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff654108190 0x7ff65419cfe0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:04.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65406b1d0 0x7ff654198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:04.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 --2- 172.21.15.32:0/2424898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff65406a830 0x7ff654198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:04.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 >> 172.21.15.32:0/2424898 conn(0x7ff654076d00 msgr2=0x7ff6540ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:04.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 shutdown_connections 2024-04-16T05:45:04.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:04.007+0000 7ff658bbd640 1 -- 172.21.15.32:0/2424898 wait complete. 2024-04-16T05:45:04.017 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-16T05:45:04.028 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-16T05:45:04.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:04 smithi032 ceph-mon[32351]: from='client.17949 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:04.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:04 smithi134 ceph-mon[36153]: from='client.17949 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:04.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:04 smithi082 ceph-mon[36195]: from='client.17949 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:05.536 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:05 smithi134 ceph-mon[36153]: from='client.17955 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:05.536 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:05 smithi134 ceph-mon[36153]: pgmap v313: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:45:05.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:05 smithi082 ceph-mon[36195]: from='client.17955 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:05.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:05 smithi082 ceph-mon[36195]: pgmap v313: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:45:05.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:05 smithi032 ceph-mon[32351]: from='client.17955 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T05:45:05.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:05 smithi032 ceph-mon[32351]: pgmap v313: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:45:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:07 smithi134 ceph-mon[36153]: pgmap v314: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:45:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:07 smithi134 ceph-mon[36153]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T05:45:07.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:07 smithi082 ceph-mon[36195]: pgmap v314: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:45:07.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:07 smithi082 ceph-mon[36195]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T05:45:07.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:07 smithi032 ceph-mon[32351]: pgmap v314: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-16T05:45:07.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:07 smithi032 ceph-mon[32351]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T05:45:09.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:09 smithi134 ceph-mon[36153]: pgmap v316: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-16T05:45:09.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:09 smithi082 ceph-mon[36195]: pgmap v316: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-16T05:45:09.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:09 smithi032 ceph-mon[32351]: pgmap v316: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-16T05:45:10.535 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:10 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:45:10.596 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:10 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:45:10.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:10 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T05:45:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:11 smithi134 ceph-mon[36153]: pgmap v317: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-16T05:45:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:11 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:11 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:11 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:11 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:11 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:11 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:11 smithi082 ceph-mon[36195]: pgmap v317: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-16T05:45:11.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:11 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:11 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:11 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:11 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:11 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:11 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:11 smithi032 ceph-mon[32351]: pgmap v317: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-16T05:45:11.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:11 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:11 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:11 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:11 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:11 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:11.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:11 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:12.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi134 to 174.1M 2024-04-16T05:45:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: Unable to set osd_memory_target on smithi134 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:45:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.635 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi082 to 174.1M 2024-04-16T05:45:12.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: Unable to set osd_memory_target on smithi082 to 182610534: error parsing value: Value '182610534' is below minimum 939524096 2024-04-16T05:45:12.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.636 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:45:12.637 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:45:12.637 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:12 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:12.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi134 to 174.1M 2024-04-16T05:45:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: Unable to set osd_memory_target on smithi134 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:45:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi082 to 174.1M 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: Unable to set osd_memory_target on smithi082 to 182610534: error parsing value: Value '182610534' is below minimum 939524096 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:45:12.651 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:45:12.652 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:12 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:12.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi134 to 174.1M 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: Unable to set osd_memory_target on smithi134 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi082 to 174.1M 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: Unable to set osd_memory_target on smithi082 to 182610534: error parsing value: Value '182610534' is below minimum 939524096 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T05:45:12.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:12 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' 2024-04-16T05:45:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:13 smithi134 ceph-mon[36153]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-04-16T05:45:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:13 smithi134 ceph-mon[36153]: Unable to set osd_memory_target on smithi032 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-16T05:45:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:13 smithi134 ceph-mon[36153]: pgmap v318: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-16T05:45:14.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:13 smithi134 ceph-mon[36153]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T05:45:14.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:13 smithi082 ceph-mon[36195]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-04-16T05:45:14.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:13 smithi082 ceph-mon[36195]: Unable to set osd_memory_target on smithi032 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-16T05:45:14.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:13 smithi082 ceph-mon[36195]: pgmap v318: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-16T05:45:14.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:13 smithi082 ceph-mon[36195]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T05:45:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:13 smithi032 ceph-mon[32351]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-04-16T05:45:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:13 smithi032 ceph-mon[32351]: Unable to set osd_memory_target on smithi032 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-16T05:45:14.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:13 smithi032 ceph-mon[32351]: pgmap v318: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-16T05:45:14.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:13 smithi032 ceph-mon[32351]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T05:45:16.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:15 smithi134 ceph-mon[36153]: pgmap v320: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-16T05:45:16.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:15 smithi082 ceph-mon[36195]: pgmap v320: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-16T05:45:16.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:15 smithi032 ceph-mon[32351]: pgmap v320: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-16T05:45:16.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T05:45:16.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T05:45:16.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 -- 172.21.15.32:0/3671193025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020101b00 msgr2=0x7f2020105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 --2- 172.21.15.32:0/3671193025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020101b00 0x7f2020105b40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2014009900 tx=0x7f201402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 -- 172.21.15.32:0/3671193025 shutdown_connections 2024-04-16T05:45:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 --2- 172.21.15.32:0/3671193025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2020106220 0x7f202010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 --2- 172.21.15.32:0/3671193025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020101b00 0x7f2020105b40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 --2- 172.21.15.32:0/3671193025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2020101150 0x7f2020101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 -- 172.21.15.32:0/3671193025 >> 172.21.15.32:0/3671193025 conn(0x7f2020077f60 msgr2=0x7f2020078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 -- 172.21.15.32:0/3671193025 shutdown_connections 2024-04-16T05:45:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.597+0000 7f202591c640 1 -- 172.21.15.32:0/3671193025 wait complete. 2024-04-16T05:45:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f202591c640 1 Processor -- start 2024-04-16T05:45:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f202591c640 1 -- start start 2024-04-16T05:45:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f202591c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2020101150 0x7f202019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f202591c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020106220 0x7f202019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f202591c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2020198770 0x7f2020198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f202591c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20201123f0 con 0x7f2020106220 2024-04-16T05:45:16.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f202591c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2020112270 con 0x7f2020198770 2024-04-16T05:45:16.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f202591c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2020112570 con 0x7f2020101150 2024-04-16T05:45:16.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020106220 0x7f202019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020106220 0x7f202019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40360/0 (socket says 172.21.15.32:40360) 2024-04-16T05:45:16.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201e7fc640 1 -- 172.21.15.32:0/2493666273 learned_addr learned my addr 172.21.15.32:0/2493666273 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:16.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201effd640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2020101150 0x7f202019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201f7fe640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2020198770 0x7f2020198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201e7fc640 1 -- 172.21.15.32:0/2493666273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2020101150 msgr2=0x7f202019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201e7fc640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2020101150 0x7f202019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201e7fc640 1 -- 172.21.15.32:0/2493666273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2020198770 msgr2=0x7f2020198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201e7fc640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2020198770 0x7f2020198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.598+0000 7f201e7fc640 1 -- 172.21.15.32:0/2493666273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2020199370 con 0x7f2020106220 2024-04-16T05:45:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.599+0000 7f201e7fc640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020106220 0x7f202019eba0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2014036040 tx=0x7f201402ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.599+0000 7f202491a640 1 -- 172.21.15.32:0/2493666273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014041070 con 0x7f2020106220 2024-04-16T05:45:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.599+0000 7f202491a640 1 -- 172.21.15.32:0/2493666273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20140054e0 con 0x7f2020106220 2024-04-16T05:45:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.599+0000 7f202591c640 1 -- 172.21.15.32:0/2493666273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2020199600 con 0x7f2020106220 2024-04-16T05:45:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.599+0000 7f202491a640 1 -- 172.21.15.32:0/2493666273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014034440 con 0x7f2020106220 2024-04-16T05:45:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.599+0000 7f202591c640 1 -- 172.21.15.32:0/2493666273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20201a56f0 con 0x7f2020106220 2024-04-16T05:45:16.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.601+0000 7f202491a640 1 -- 172.21.15.32:0/2493666273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2014044070 con 0x7f2020106220 2024-04-16T05:45:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.601+0000 7f20285e2740 1 Processor -- start 2024-04-16T05:45:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.601+0000 7f20285e2740 1 -- start start 2024-04-16T05:45:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f202491a640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f20080a3d20 0x7f20080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f20285e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff3e50 0x561362ff4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f20285e2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f20285e2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362fff740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f20285e2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561362f59180 con 0x561362ffd350 2024-04-16T05:45:16.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f20285e2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561362ef6360 con 0x561362ff4770 2024-04-16T05:45:16.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f20285e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561362f55140 con 0x561362ff3e50 2024-04-16T05:45:16.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f202491a640 1 -- 172.21.15.32:0/2493666273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f20140f2750 con 0x7f2020106220 2024-04-16T05:45:16.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f20080a3d20 0x7f20080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362fff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362fff740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40366/0 (socket says 172.21.15.32:40366) 2024-04-16T05:45:16.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 -- 172.21.15.32:0/923472042 learned_addr learned my addr 172.21.15.32:0/923472042 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201e7fc640 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201effd640 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff3e50 0x561362ff4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 -- 172.21.15.32:0/923472042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff3e50 msgr2=0x561362ff4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff3e50 0x561362ff4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 -- 172.21.15.32:0/923472042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 msgr2=0x561362ffce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 -- 172.21.15.32:0/923472042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561362d357d0 con 0x561362ffd350 2024-04-16T05:45:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f20080a3d20 0x7f20080a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f20180137e0 tx=0x7f201800e9f0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f201f7fe640 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362fff740 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2018010740 tx=0x7f201801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d807bb286705414a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f2005ffb640 1 -- 172.21.15.32:0/923472042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018010e80 con 0x561362ffd350 2024-04-16T05:45:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f2005ffb640 1 -- 172.21.15.32:0/923472042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2018005020 con 0x561362ffd350 2024-04-16T05:45:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.602+0000 7f2005ffb640 1 -- 172.21.15.32:0/923472042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180049a0 con 0x561362ffd350 2024-04-16T05:45:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 -- 172.21.15.32:0/923472042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 msgr2=0x561362fff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362fff740 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2018010740 tx=0x7f201801a000 comp rx=0 tx=0).stop 2024-04-16T05:45:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 -- 172.21.15.32:0/923472042 shutdown_connections 2024-04-16T05:45:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362fff740 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 --2- 172.21.15.32:0/923472042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff3e50 0x561362ff4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 -- 172.21.15.32:0/923472042 >> 172.21.15.32:0/923472042 conn(0x561362fee580 msgr2=0x561362fee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 -- 172.21.15.32:0/923472042 shutdown_connections 2024-04-16T05:45:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 -- 172.21.15.32:0/923472042 wait complete. 2024-04-16T05:45:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.603+0000 7f20285e2740 1 -- start start 2024-04-16T05:45:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f20285e2740 1 Processor -- start 2024-04-16T05:45:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f20285e2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff3e50 0x561362ffa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f20285e2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f20285e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561362ffae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f20285e2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5613630000e0 con 0x561362ff3e50 2024-04-16T05:45:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f20285e2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561362f122c0 con 0x561362ff4770 2024-04-16T05:45:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f20285e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561363000260 con 0x561362ffd350 2024-04-16T05:45:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f20285e2740 10 client.0 _ll_register_callbacks cb 0x561362ef9090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff3e50 0x561362ffa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff3e50 0x561362ffa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40382/0 (socket says 172.21.15.32:40382) 2024-04-16T05:45:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201effd640 1 -- 172.21.15.32:0/3364461116 learned_addr learned my addr 172.21.15.32:0/3364461116 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201f7fe640 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561362ffae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201e7fc640 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201effd640 1 -- 172.21.15.32:0/3364461116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 msgr2=0x561362ffae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201effd640 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561362ffae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201effd640 1 -- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 msgr2=0x561362ffa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201effd640 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201effd640 1 -- 172.21.15.32:0/3364461116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561362c6b3f0 con 0x561362ff3e50 2024-04-16T05:45:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.604+0000 7f201e7fc640 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f201effd640 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff3e50 0x561362ffa380 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2010009fc0 tx=0x7f2010003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cfd9498246c643 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f1ffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:45:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3364461116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010012aa0 con 0x561362ff3e50 2024-04-16T05:45:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3364461116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f2010012c40 con 0x561362ff3e50 2024-04-16T05:45:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 1 -- 172.21.15.32:0/3364461116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2018005020 con 0x561362ff3e50 2024-04-16T05:45:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3364461116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2010012e40 con 0x561362ff3e50 2024-04-16T05:45:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 10 client.17967 fetch_fsmap learned FSMap version 25 2024-04-16T05:45:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 1 -- 172.21.15.32:0/3364461116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561362c2b0f0 con 0x561362ff3e50 2024-04-16T05:45:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3364461116 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2018005020 con 0x561362ff3e50 2024-04-16T05:45:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3364461116 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2010019070 con 0x561362ff3e50 2024-04-16T05:45:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 10 client.17967 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:45:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 20 client.17967 populate_metadata read hostname 'smithi032' 2024-04-16T05:45:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56136300d710 0x56136300fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 4 client.17967 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.605+0000 7f20285e2740 1 -- 172.21.15.32:0/3364461116 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561362f66ba0 con 0x56136300d710 2024-04-16T05:45:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.606+0000 7f201e7fc640 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56136300d710 0x56136300fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.606+0000 7f201e7fc640 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56136300d710 0x56136300fb80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b899279f3b8edd26 server_cookie=43190835b4710b63 in_seq=0 out_seq=0 2024-04-16T05:45:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.606+0000 7f1ffeffd640 10 client.17967 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:45:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3364461116 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561362f66ba0 con 0x56136300d710 2024-04-16T05:45:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f1ffeffd640 10 client.17967 handle_command_reply: tid=0 2024-04-16T05:45:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 client.17967 shutdown 2024-04-16T05:45:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 -- 172.21.15.32:0/3364461116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff3e50 msgr2=0x561362ffa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff3e50 0x561362ffa380 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2010009fc0 tx=0x7f2010003f90 comp rx=0 tx=0).stop 2024-04-16T05:45:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 -- 172.21.15.32:0/3364461116 shutdown_connections 2024-04-16T05:45:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x56136300d710 0x56136300fb80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561362ffae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff4770 0x561362ffa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f1ffeffd640 0 client.17967 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:45:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 --2- 172.21.15.32:0/3364461116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff3e50 0x561362ffa380 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 -- 172.21.15.32:0/3364461116 >> 172.21.15.32:0/3364461116 conn(0x561362fee580 msgr2=0x561362ff02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 -- 172.21.15.32:0/3364461116 shutdown_connections 2024-04-16T05:45:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 1 -- 172.21.15.32:0/3364461116 wait complete. 2024-04-16T05:45:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.607+0000 7f20285e2740 20 client.17967 trim_cache size 0 max 16384 2024-04-16T05:45:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f20285e2740 1 Processor -- start 2024-04-16T05:45:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f20285e2740 1 -- start start 2024-04-16T05:45:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f20285e2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363008ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f20285e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff4770 0x561362ff4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f20285e2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362ffac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f20285e2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20201b1df0 con 0x561362ffd350 2024-04-16T05:45:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f20285e2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f20201af6c0 con 0x561362ff3e50 2024-04-16T05:45:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f20285e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20201ad530 con 0x561362ff4770 2024-04-16T05:45:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362ffac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362ffac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40386/0 (socket says 172.21.15.32:40386) 2024-04-16T05:45:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201f7fe640 1 -- 172.21.15.32:0/626086407 learned_addr learned my addr 172.21.15.32:0/626086407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201effd640 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363008ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201e7fc640 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff4770 0x561362ff4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201f7fe640 1 -- 172.21.15.32:0/626086407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff4770 msgr2=0x561362ff4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201f7fe640 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff4770 0x561362ff4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201f7fe640 1 -- 172.21.15.32:0/626086407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 msgr2=0x561363008ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201f7fe640 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363008ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.625+0000 7f201f7fe640 1 -- 172.21.15.32:0/626086407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561362ff9330 con 0x561362ffd350 2024-04-16T05:45:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f201f7fe640 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362ffac20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2018004600 tx=0x7f201800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd537795f0562958 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20057fa640 1 -- 172.21.15.32:0/626086407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018013d80 con 0x561362ffd350 2024-04-16T05:45:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20057fa640 1 -- 172.21.15.32:0/626086407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201802d070 con 0x561362ffd350 2024-04-16T05:45:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20057fa640 1 -- 172.21.15.32:0/626086407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800b6a0 con 0x561362ffd350 2024-04-16T05:45:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 -- 172.21.15.32:0/626086407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 msgr2=0x561362ffac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362ffac20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2018004600 tx=0x7f201800e350 comp rx=0 tx=0).stop 2024-04-16T05:45:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 -- 172.21.15.32:0/626086407 shutdown_connections 2024-04-16T05:45:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ffd350 0x561362ffac20 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ff4770 0x561362ff4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 --2- 172.21.15.32:0/626086407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363008ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 -- 172.21.15.32:0/626086407 >> 172.21.15.32:0/626086407 conn(0x561362fee580 msgr2=0x561362ff0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 -- 172.21.15.32:0/626086407 shutdown_connections 2024-04-16T05:45:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.626+0000 7f20285e2740 1 -- 172.21.15.32:0/626086407 wait complete. 2024-04-16T05:45:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 1 -- start start 2024-04-16T05:45:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 1 Processor -- start 2024-04-16T05:45:16.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363005a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff4770 0x561363002e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561363003570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20201adb80 con 0x561362ff4770 2024-04-16T05:45:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f20201aeff0 con 0x561362ff3e50 2024-04-16T05:45:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20201ae250 con 0x561362ffd350 2024-04-16T05:45:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f20285e2740 10 client.0 _ll_register_callbacks cb 0x7f2010009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff4770 0x561363002e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff4770 0x561363002e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40402/0 (socket says 172.21.15.32:40402) 2024-04-16T05:45:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201e7fc640 1 -- 172.21.15.32:0/3135962898 learned_addr learned my addr 172.21.15.32:0/3135962898 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:16.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201f7fe640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561363003570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201effd640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363005a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201e7fc640 1 -- 172.21.15.32:0/3135962898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 msgr2=0x561363003570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:16.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201e7fc640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561363003570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201e7fc640 1 -- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 msgr2=0x561363005a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:16.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201e7fc640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363005a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201e7fc640 1 -- 172.21.15.32:0/3135962898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561363072cd0 con 0x561362ff4770 2024-04-16T05:45:16.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201f7fe640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561363003570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:16.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.627+0000 7f201effd640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363005a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:16.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f201e7fc640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff4770 0x561363002e30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f20140ba960 tx=0x7f201402c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f481d0151ec7a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:16.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f1ffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:45:16.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3135962898 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201404b040 con 0x561362ff4770 2024-04-16T05:45:16.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3135962898 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f20140efd90 con 0x561362ff4770 2024-04-16T05:45:16.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 1 -- 172.21.15.32:0/3135962898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201802d070 con 0x561362ff4770 2024-04-16T05:45:16.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3135962898 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2014004c50 con 0x561362ff4770 2024-04-16T05:45:16.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 10 client.17973 fetch_fsmap learned FSMap version 25 2024-04-16T05:45:16.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 1 -- 172.21.15.32:0/3135962898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561363073e70 con 0x561362ff4770 2024-04-16T05:45:16.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3135962898 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201802d070 con 0x561362ff4770 2024-04-16T05:45:16.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3135962898 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f20140dd890 con 0x561362ff4770 2024-04-16T05:45:16.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 10 client.17973 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:45:16.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 20 client.17973 populate_metadata read hostname 'smithi032' 2024-04-16T05:45:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5613630745a0 0x5613630769c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 4 client.17973 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T05:45:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.628+0000 7f20285e2740 1 -- 172.21.15.32:0/3135962898 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561362f60a10 con 0x5613630745a0 2024-04-16T05:45:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f201effd640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5613630745a0 0x5613630769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f1ffeffd640 10 client.17973 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:45:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f201effd640 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5613630745a0 0x5613630769c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d51ae711de086c49 server_cookie=ebf4b4cc6d44acc8 in_seq=0 out_seq=0 2024-04-16T05:45:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f1ffeffd640 1 -- 172.21.15.32:0/3135962898 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561362f60a10 con 0x5613630745a0 2024-04-16T05:45:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f1ffeffd640 10 client.17973 handle_command_reply: tid=0 2024-04-16T05:45:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 client.17973 shutdown 2024-04-16T05:45:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 -- 172.21.15.32:0/3135962898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff4770 msgr2=0x561363002e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff4770 0x561363002e30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f20140ba960 tx=0x7f201402c1f0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 -- 172.21.15.32:0/3135962898 shutdown_connections 2024-04-16T05:45:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x5613630745a0 0x5613630769c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561362ffd350 0x561363003570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561362ff4770 0x561363002e30 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f1ffeffd640 0 client.17973 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:45:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 --2- 172.21.15.32:0/3135962898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561362ff3e50 0x561363005a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 -- 172.21.15.32:0/3135962898 >> 172.21.15.32:0/3135962898 conn(0x561362fee580 msgr2=0x561362feced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 -- 172.21.15.32:0/3135962898 shutdown_connections 2024-04-16T05:45:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.629+0000 7f20285e2740 1 -- 172.21.15.32:0/3135962898 wait complete. 2024-04-16T05:45:16.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f20285e2740 20 client.17973 trim_cache size 0 max 16384 2024-04-16T05:45:16.641 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:45:16.641 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-16T05:45:16.641 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-16T05:45:16.641 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:45:16.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 -- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f20080a3d20 msgr2=0x7f20080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f20080a3d20 0x7f20080a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f20180137e0 tx=0x7f201800e9f0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 -- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020106220 msgr2=0x7f202019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020106220 0x7f202019eba0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2014036040 tx=0x7f201402ce20 comp rx=0 tx=0).stop 2024-04-16T05:45:16.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 -- 172.21.15.32:0/2493666273 shutdown_connections 2024-04-16T05:45:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2020198770 0x7f2020198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f20080a3d20 0x7f20080a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2020106220 0x7f202019eba0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 --2- 172.21.15.32:0/2493666273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2020101150 0x7f202019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 -- 172.21.15.32:0/2493666273 >> 172.21.15.32:0/2493666273 conn(0x7f2020077f60 msgr2=0x7f20200fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 -- 172.21.15.32:0/2493666273 shutdown_connections 2024-04-16T05:45:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.630+0000 7f202591c640 1 -- 172.21.15.32:0/2493666273 wait complete. 2024-04-16T05:45:16.645 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T05:45:16.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:45:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.752+0000 7fc544fdb640 1 -- 172.21.15.32:0/4128366638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc540101b00 msgr2=0x7fc540105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.752+0000 7fc544fdb640 1 --2- 172.21.15.32:0/4128366638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc540101b00 0x7fc540105b40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc528009900 tx=0x7fc52802bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 -- 172.21.15.32:0/4128366638 shutdown_connections 2024-04-16T05:45:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 --2- 172.21.15.32:0/4128366638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc540106220 0x7fc54010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 --2- 172.21.15.32:0/4128366638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc540101b00 0x7fc540105b40 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 --2- 172.21.15.32:0/4128366638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc540101150 0x7fc540101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 -- 172.21.15.32:0/4128366638 >> 172.21.15.32:0/4128366638 conn(0x7fc540078000 msgr2=0x7fc540078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 -- 172.21.15.32:0/4128366638 shutdown_connections 2024-04-16T05:45:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 -- 172.21.15.32:0/4128366638 wait complete. 2024-04-16T05:45:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 Processor -- start 2024-04-16T05:45:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.753+0000 7fc544fdb640 1 -- start start 2024-04-16T05:45:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc544fdb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc540101150 0x7fc54019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc544fdb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc540106220 0x7fc54019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc544fdb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5401986b0 0x7fc540198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc544fdb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401112b0 con 0x7fc5401986b0 2024-04-16T05:45:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc544fdb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc540111130 con 0x7fc540101150 2024-04-16T05:45:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc544fdb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc540111430 con 0x7fc540106220 2024-04-16T05:45:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5401986b0 0x7fc540198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5401986b0 0x7fc540198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40426/0 (socket says 172.21.15.32:40426) 2024-04-16T05:45:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 -- 172.21.15.32:0/1192993635 learned_addr learned my addr 172.21.15.32:0/1192993635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53e59c640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc540101150 0x7fc54019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc535d9b640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc540106220 0x7fc54019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 -- 172.21.15.32:0/1192993635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc540106220 msgr2=0x7fc54019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc540106220 0x7fc54019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 -- 172.21.15.32:0/1192993635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc540101150 msgr2=0x7fc54019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc540101150 0x7fc54019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 -- 172.21.15.32:0/1192993635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5401992b0 con 0x7fc5401986b0 2024-04-16T05:45:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc53ed9d640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5401986b0 0x7fc540198b30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fc53800eb10 tx=0x7fc53800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc537fff640 1 -- 172.21.15.32:0/1192993635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538004b20 con 0x7fc5401986b0 2024-04-16T05:45:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5401a5390 con 0x7fc5401986b0 2024-04-16T05:45:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc537fff640 1 -- 172.21.15.32:0/1192993635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc538004cc0 con 0x7fc5401986b0 2024-04-16T05:45:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.754+0000 7fc537fff640 1 -- 172.21.15.32:0/1192993635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538010760 con 0x7fc5401986b0 2024-04-16T05:45:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.755+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5401a5740 con 0x7fc5401986b0 2024-04-16T05:45:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.755+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc540102620 con 0x7fc5401986b0 2024-04-16T05:45:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.757+0000 7fc537fff640 1 -- 172.21.15.32:0/1192993635 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc538005020 con 0x7fc5401986b0 2024-04-16T05:45:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.757+0000 7fc537fff640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.757+0000 7fc537fff640 1 -- 172.21.15.32:0/1192993635 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7fc5380d1200 con 0x7fc5401986b0 2024-04-16T05:45:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.757+0000 7fc53e59c640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.758+0000 7fc53e59c640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc530005fd0 tx=0x7fc530005850 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.762+0000 7fc537fff640 1 -- 172.21.15.32:0/1192993635 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc53809d510 con 0x7fc5401986b0 2024-04-16T05:45:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.987+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc540068e20 con 0x7fc5401986b0 2024-04-16T05:45:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.988+0000 7fc537fff640 1 -- 172.21.15.32:0/1192993635 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7fc53809ce10 con 0x7fc5401986b0 2024-04-16T05:45:16.986 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:45:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:45:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:45:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc52c0a3d20 msgr2=0x7fc52c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc530005fd0 tx=0x7fc530005850 comp rx=0 tx=0).stop 2024-04-16T05:45:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5401986b0 msgr2=0x7fc540198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5401986b0 0x7fc540198b30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fc53800eb10 tx=0x7fc53800c2d0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 shutdown_connections 2024-04-16T05:45:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5401986b0 0x7fc540198b30 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc540106220 0x7fc54019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 --2- 172.21.15.32:0/1192993635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc540101150 0x7fc54019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 >> 172.21.15.32:0/1192993635 conn(0x7fc540078000 msgr2=0x7fc5400694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 shutdown_connections 2024-04-16T05:45:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:16.990+0000 7fc544fdb640 1 -- 172.21.15.32:0/1192993635 wait complete. 2024-04-16T05:45:16.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T05:45:17.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/62317183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101610 msgr2=0x7f3d68101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/62317183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101610 0x7f3d68101a90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009900 tx=0x7f3d5002bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/62317183 shutdown_connections 2024-04-16T05:45:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/62317183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d68101fd0 0x7f3d6810ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/62317183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101610 0x7f3d68101a90 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/62317183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d68105f30 0x7f3d68106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/62317183 >> 172.21.15.32:0/62317183 conn(0x7f3d68077450 msgr2=0x7f3d680fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/62317183 shutdown_connections 2024-04-16T05:45:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/62317183 wait complete. 2024-04-16T05:45:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 Processor -- start 2024-04-16T05:45:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.111+0000 7f3d6d3c8640 1 -- start start 2024-04-16T05:45:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d6d3c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d68101610 0x7f3d681996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d6d3c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101fd0 0x7f3d68199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d6d3c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d68105f30 0x7f3d681a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d6d3c8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d68111290 con 0x7f3d68101fd0 2024-04-16T05:45:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d6d3c8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d68111110 con 0x7f3d68101610 2024-04-16T05:45:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d6d3c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d68111410 con 0x7f3d68105f30 2024-04-16T05:45:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101fd0 0x7f3d68199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101fd0 0x7f3d68199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40444/0 (socket says 172.21.15.32:40444) 2024-04-16T05:45:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 -- 172.21.15.32:0/1776706228 learned_addr learned my addr 172.21.15.32:0/1776706228 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d68101610 0x7f3d681996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d677fe640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d68105f30 0x7f3d681a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 -- 172.21.15.32:0/1776706228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d68105f30 msgr2=0x7f3d681a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d68105f30 0x7f3d681a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 -- 172.21.15.32:0/1776706228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d68101610 msgr2=0x7f3d681996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d68101610 0x7f3d681996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 -- 172.21.15.32:0/1776706228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d681a4c70 con 0x7f3d68101fd0 2024-04-16T05:45:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.112+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101fd0 0x7f3d68199bf0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3d50005940 tx=0x7f3d500043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.113+0000 7f3d64ff9640 1 -- 172.21.15.32:0/1776706228 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50041070 con 0x7f3d68101fd0 2024-04-16T05:45:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.113+0000 7f3d64ff9640 1 -- 172.21.15.32:0/1776706228 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d5002ccc0 con 0x7f3d68101fd0 2024-04-16T05:45:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.113+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/1776706228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d681a4e40 con 0x7f3d68101fd0 2024-04-16T05:45:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.113+0000 7f3d64ff9640 1 -- 172.21.15.32:0/1776706228 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50003e20 con 0x7f3d68101fd0 2024-04-16T05:45:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.113+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/1776706228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d681a5240 con 0x7f3d68101fd0 2024-04-16T05:45:17.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d64ff9640 1 -- 172.21.15.32:0/1776706228 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3d50044070 con 0x7f3d68101fd0 2024-04-16T05:45:17.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d6ffb2740 1 Processor -- start 2024-04-16T05:45:17.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d6ffb2740 1 -- start start 2024-04-16T05:45:17.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 0x5581b9544f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b9552230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9554b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5581b94aa0e0 con 0x5581b9552770 2024-04-16T05:45:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5581b9447520 con 0x5581b9544b90 2024-04-16T05:45:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.115+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5581b94a60a0 con 0x5581b95454b0 2024-04-16T05:45:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d64ff9640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3d540a3d20 0x7f3d540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9554b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d677fe640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3d540a3d20 0x7f3d540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9554b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40448/0 (socket says 172.21.15.32:40448) 2024-04-16T05:45:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 -- 172.21.15.32:0/1131719008 learned_addr learned my addr 172.21.15.32:0/1131719008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b9552230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d64ff9640 1 -- 172.21.15.32:0/1776706228 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f3d500f26f0 con 0x7f3d68101fd0 2024-04-16T05:45:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d677fe640 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 0x5581b9544f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 -- 172.21.15.32:0/1131719008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 msgr2=0x5581b9552230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b9552230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 -- 172.21.15.32:0/1131719008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 msgr2=0x5581b9544f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 0x5581b9544f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 -- 172.21.15.32:0/1131719008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b946f8d0 con 0x5581b9552770 2024-04-16T05:45:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d677fe640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3d540a3d20 0x7f3d540a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005fd0 tx=0x7f3d60009a30 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9554b60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3d500889e0 tx=0x7f3d50090f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140390991c066d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5d7fa640 1 -- 172.21.15.32:0/1131719008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5004b040 con 0x5581b9552770 2024-04-16T05:45:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5d7fa640 1 -- 172.21.15.32:0/1131719008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d50088f20 con 0x5581b9552770 2024-04-16T05:45:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.116+0000 7f3d5d7fa640 1 -- 172.21.15.32:0/1131719008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d500c27c0 con 0x5581b9552770 2024-04-16T05:45:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/1131719008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 msgr2=0x5581b9554b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9554b60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3d500889e0 tx=0x7f3d50090f00 comp rx=0 tx=0).stop 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/1131719008 shutdown_connections 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9554b60 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b9552230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/1131719008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 0x5581b9544f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/1131719008 >> 172.21.15.32:0/1131719008 conn(0x5581b953f590 msgr2=0x5581b953f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/1131719008 shutdown_connections 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/1131719008 wait complete. 2024-04-16T05:45:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.117+0000 7f3d6ffb2740 1 -- start start 2024-04-16T05:45:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d6ffb2740 1 Processor -- start 2024-04-16T05:45:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9544b90 0x5581b954f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b954fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9552770 0x5581b95502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5581b95552f0 con 0x5581b9544b90 2024-04-16T05:45:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5581b94634d0 con 0x5581b95454b0 2024-04-16T05:45:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5581b9555470 con 0x5581b9552770 2024-04-16T05:45:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d6ffb2740 10 client.0 _ll_register_callbacks cb 0x5581b94482c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9544b90 0x5581b954f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9544b90 0x5581b954f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40458/0 (socket says 172.21.15.32:40458) 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 -- 172.21.15.32:0/2877635733 learned_addr learned my addr 172.21.15.32:0/2877635733 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d677fe640 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9552770 0x5581b95502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b954fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 -- 172.21.15.32:0/2877635733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9552770 msgr2=0x5581b95502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9552770 0x5581b95502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 -- 172.21.15.32:0/2877635733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 msgr2=0x5581b954fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b954fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 -- 172.21.15.32:0/2877635733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b91bc3f0 con 0x5581b9544b90 2024-04-16T05:45:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b954fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9544b90 0x5581b954f820 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800be70 tx=0x7f3d58012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4690fd2c97b06c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:45:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/2877635733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5801f060 con 0x5581b9544b90 2024-04-16T05:45:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.118+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/2877635733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f3d580108a0 con 0x5581b9544b90 2024-04-16T05:45:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/2877635733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d50088f20 con 0x5581b9544b90 2024-04-16T05:45:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/2877635733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d58026050 con 0x5581b9544b90 2024-04-16T05:45:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 10 client.17991 fetch_fsmap learned FSMap version 25 2024-04-16T05:45:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/2877635733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5581b92867d0 con 0x5581b9544b90 2024-04-16T05:45:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/2877635733 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d50088f20 con 0x5581b9544b90 2024-04-16T05:45:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/2877635733 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3d58019030 con 0x5581b9544b90 2024-04-16T05:45:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 10 client.17991 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:45:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 10 client.17991 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:45:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 20 client.17991 populate_metadata read hostname 'smithi032' 2024-04-16T05:45:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5581b9562510 0x5581b95649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 4 client.17991 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/2877635733 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581b94b7b00 con 0x5581b9562510 2024-04-16T05:45:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.119+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5581b9562510 0x5581b95649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.120+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5581b9562510 0x5581b95649a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=990698ab1c0a4c96 server_cookie=25ea76bc7089722e in_seq=0 out_seq=0 2024-04-16T05:45:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.120+0000 7f3d4e7fc640 10 client.17991 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:45:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/2877635733 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581b94b7b00 con 0x5581b9562510 2024-04-16T05:45:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d4e7fc640 10 client.17991 handle_command_reply: tid=0 2024-04-16T05:45:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 client.17991 shutdown 2024-04-16T05:45:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/2877635733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9544b90 msgr2=0x5581b954f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9544b90 0x5581b954f820 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800be70 tx=0x7f3d58012820 comp rx=0 tx=0).stop 2024-04-16T05:45:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/2877635733 shutdown_connections 2024-04-16T05:45:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5581b9562510 0x5581b95649a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9552770 0x5581b95502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b954fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d4e7fc640 0 client.17991 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:45:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/2877635733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9544b90 0x5581b954f820 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/2877635733 >> 172.21.15.32:0/2877635733 conn(0x5581b953f590 msgr2=0x5581b95412e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/2877635733 shutdown_connections 2024-04-16T05:45:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/2877635733 wait complete. 2024-04-16T05:45:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.121+0000 7f3d6ffb2740 20 client.17991 trim_cache size 0 max 16384 2024-04-16T05:45:17.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.134+0000 7f3d6ffb2740 1 Processor -- start 2024-04-16T05:45:17.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.134+0000 7f3d6ffb2740 1 -- start start 2024-04-16T05:45:17.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9544b90 0x5581b9551f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b955e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9541060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681afbc0 con 0x5581b9552770 2024-04-16T05:45:17.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681ad490 con 0x5581b95454b0 2024-04-16T05:45:17.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681ab300 con 0x5581b9544b90 2024-04-16T05:45:17.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9541060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9541060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40468/0 (socket says 172.21.15.32:40468) 2024-04-16T05:45:17.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 -- 172.21.15.32:0/3992871703 learned_addr learned my addr 172.21.15.32:0/3992871703 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9544b90 0x5581b9551f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b955e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 -- 172.21.15.32:0/3992871703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9544b90 msgr2=0x5581b9551f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9544b90 0x5581b9551f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 -- 172.21.15.32:0/3992871703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 msgr2=0x5581b955e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b955e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 -- 172.21.15.32:0/3992871703 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b9545bb0 con 0x5581b9552770 2024-04-16T05:45:17.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d677fe640 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9541060 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f3d60010a40 tx=0x7f3d6001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584a1a3a70d8e592 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.135+0000 7f3d5cff9640 1 -- 172.21.15.32:0/3992871703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60004d20 con 0x5581b9552770 2024-04-16T05:45:17.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d5cff9640 1 -- 172.21.15.32:0/3992871703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d60005020 con 0x5581b9552770 2024-04-16T05:45:17.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d5cff9640 1 -- 172.21.15.32:0/3992871703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d600295e0 con 0x5581b9552770 2024-04-16T05:45:17.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/3992871703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 msgr2=0x5581b9541060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9541060 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f3d60010a40 tx=0x7f3d6001d000 comp rx=0 tx=0).stop 2024-04-16T05:45:17.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/3992871703 shutdown_connections 2024-04-16T05:45:17.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b9541060 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b95454b0 0x5581b955e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/3992871703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b9544b90 0x5581b9551f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/3992871703 >> 172.21.15.32:0/3992871703 conn(0x5581b953f590 msgr2=0x5581b9540950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/3992871703 shutdown_connections 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.136+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/3992871703 wait complete. 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 1 -- start start 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 1 Processor -- start 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 0x5581b955ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b954c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b954cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681ab950 con 0x5581b9552770 2024-04-16T05:45:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681acdc0 con 0x5581b9544b90 2024-04-16T05:45:17.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681ac020 con 0x5581b95454b0 2024-04-16T05:45:17.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d6ffb2740 10 client.0 _ll_register_callbacks cb 0x7f3d5800b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:17.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b954cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b954cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40480/0 (socket says 172.21.15.32:40480) 2024-04-16T05:45:17.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d677fe640 1 -- 172.21.15.32:0/777238884 learned_addr learned my addr 172.21.15.32:0/777238884 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 0x5581b955ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b954c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d677fe640 1 -- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 msgr2=0x5581b954c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d677fe640 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b954c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d677fe640 1 -- 172.21.15.32:0/777238884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 msgr2=0x5581b955ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d677fe640 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 0x5581b955ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d677fe640 1 -- 172.21.15.32:0/777238884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b95c7ae0 con 0x5581b9552770 2024-04-16T05:45:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.137+0000 7f3d5ffff640 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b954c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d677fe640 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b954cb20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3d600207d0 tx=0x7f3d60004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fbf45f8ae32fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/777238884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60036020 con 0x5581b9552770 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/777238884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f3d60020c50 con 0x5581b9552770 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/777238884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d60005020 con 0x5581b9552770 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/777238884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d600051d0 con 0x5581b9552770 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 10 client.17997 fetch_fsmap learned FSMap version 25 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/777238884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5581b95c8ca0 con 0x5581b9552770 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/777238884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d60005020 con 0x5581b9552770 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/777238884 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3d60032a60 con 0x5581b9552770 2024-04-16T05:45:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 10 client.17997 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:45:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 10 client.17997 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T05:45:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 20 client.17997 populate_metadata read hostname 'smithi032' 2024-04-16T05:45:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5581b95c93b0 0x5581b95cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 4 client.17997 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.138+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/777238884 --> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581b94b1970 con 0x5581b95c93b0 2024-04-16T05:45:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5581b95c93b0 0x5581b95cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d4e7fc640 10 client.17997 ms_handle_connect on v2:172.21.15.134:6838/3919403886 2024-04-16T05:45:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d66ffd640 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5581b95c93b0 0x5581b95cb7a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9139cb9e8e23cfb server_cookie=f9a2ba116e448fc2 in_seq=0 out_seq=0 2024-04-16T05:45:17.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d4e7fc640 1 -- 172.21.15.32:0/777238884 <== mds.2 v2:172.21.15.134:6838/3919403886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581b94b1970 con 0x5581b95c93b0 2024-04-16T05:45:17.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d4e7fc640 10 client.17997 handle_command_reply: tid=0 2024-04-16T05:45:17.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 client.17997 shutdown 2024-04-16T05:45:17.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/777238884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 msgr2=0x5581b954cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b954cb20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3d600207d0 tx=0x7f3d60004dd0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/777238884 shutdown_connections 2024-04-16T05:45:17.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:6838/3919403886,v1:172.21.15.134:6839/3919403886] conn(0x5581b95c93b0 0x5581b95cb7a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5581b9552770 0x5581b954cb20 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581b95454b0 0x5581b954c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d4e7fc640 0 client.17997 ms_handle_reset on v2:172.21.15.134:6838/3919403886 2024-04-16T05:45:17.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 --2- 172.21.15.32:0/777238884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5581b9544b90 0x5581b955ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/777238884 >> 172.21.15.32:0/777238884 conn(0x5581b953f590 msgr2=0x5581b95411e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/777238884 shutdown_connections 2024-04-16T05:45:17.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 1 -- 172.21.15.32:0/777238884 wait complete. 2024-04-16T05:45:17.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.139+0000 7f3d6ffb2740 20 client.17997 trim_cache size 0 max 16384 2024-04-16T05:45:17.152 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:45:17.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3d540a3d20 msgr2=0x7f3d540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3d540a3d20 0x7f3d540a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005fd0 tx=0x7f3d60009a30 comp rx=0 tx=0).stop 2024-04-16T05:45:17.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101fd0 msgr2=0x7f3d68199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101fd0 0x7f3d68199bf0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3d50005940 tx=0x7f3d500043b0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/1776706228 shutdown_connections 2024-04-16T05:45:17.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d68105f30 0x7f3d681a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3d540a3d20 0x7f3d540a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68101fd0 0x7f3d68199bf0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 --2- 172.21.15.32:0/1776706228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d68101610 0x7f3d681996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/1776706228 >> 172.21.15.32:0/1776706228 conn(0x7f3d68077450 msgr2=0x7f3d68103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/1776706228 shutdown_connections 2024-04-16T05:45:17.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.140+0000 7f3d6d3c8640 1 -- 172.21.15.32:0/1776706228 wait complete. 2024-04-16T05:45:17.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T05:45:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 -- 172.21.15.32:0/2300914244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0106200 msgr2=0x7f8cc010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 --2- 172.21.15.32:0/2300914244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0106200 0x7f8cc010ddd0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009900 tx=0x7f8cb002bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 -- 172.21.15.32:0/2300914244 shutdown_connections 2024-04-16T05:45:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 --2- 172.21.15.32:0/2300914244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0106200 0x7f8cc010ddd0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 --2- 172.21.15.32:0/2300914244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc0101ae0 0x7f8cc0105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 --2- 172.21.15.32:0/2300914244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc0101130 0x7f8cc0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 -- 172.21.15.32:0/2300914244 >> 172.21.15.32:0/2300914244 conn(0x7f8cc00780b0 msgr2=0x7f8cc00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 -- 172.21.15.32:0/2300914244 shutdown_connections 2024-04-16T05:45:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.264+0000 7f8cc5422640 1 -- 172.21.15.32:0/2300914244 wait complete. 2024-04-16T05:45:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.265+0000 7f8cc5422640 1 Processor -- start 2024-04-16T05:45:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.265+0000 7f8cc5422640 1 -- start start 2024-04-16T05:45:17.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.265+0000 7f8cc5422640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc0101130 0x7f8cc019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.265+0000 7f8cc5422640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc0106200 0x7f8cc019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.265+0000 7f8cc5422640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0198690 0x7f8cc0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.265+0000 7f8cc5422640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01123d0 con 0x7f8cc0198690 2024-04-16T05:45:17.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.265+0000 7f8cc5422640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0112250 con 0x7f8cc0101130 2024-04-16T05:45:17.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cc5422640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0112550 con 0x7f8cc0106200 2024-04-16T05:45:17.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0198690 0x7f8cc0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0198690 0x7f8cc0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40512/0 (socket says 172.21.15.32:40512) 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 -- 172.21.15.32:0/888940832 learned_addr learned my addr 172.21.15.32:0/888940832 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc0106200 0x7f8cc019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc0101130 0x7f8cc019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 -- 172.21.15.32:0/888940832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc0106200 msgr2=0x7f8cc019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc0106200 0x7f8cc019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 -- 172.21.15.32:0/888940832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc0101130 msgr2=0x7f8cc019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc0101130 0x7f8cc019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 -- 172.21.15.32:0/888940832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc0199290 con 0x7f8cc0198690 2024-04-16T05:45:17.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc0101130 0x7f8cc019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:17.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cbf7fe640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0198690 0x7f8cc0198b10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009230 tx=0x7f8cb002cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cb7fff640 1 -- 172.21.15.32:0/888940832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0041070 con 0x7f8cc0198690 2024-04-16T05:45:17.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cb7fff640 1 -- 172.21.15.32:0/888940832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb0004450 con 0x7f8cc0198690 2024-04-16T05:45:17.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cc5422640 1 -- 172.21.15.32:0/888940832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc0199520 con 0x7f8cc0198690 2024-04-16T05:45:17.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cb7fff640 1 -- 172.21.15.32:0/888940832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0003d90 con 0x7f8cc0198690 2024-04-16T05:45:17.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.266+0000 7f8cc5422640 1 -- 172.21.15.32:0/888940832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc01a5660 con 0x7f8cc0198690 2024-04-16T05:45:17.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.269+0000 7f8cb7fff640 1 -- 172.21.15.32:0/888940832 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8cb0044070 con 0x7f8cc0198690 2024-04-16T05:45:17.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.269+0000 7f8cc800d740 1 Processor -- start 2024-04-16T05:45:17.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.269+0000 7f8cc800d740 1 -- start start 2024-04-16T05:45:17.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.269+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55655556fe60 0x556555570240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x556555578e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 0x55655557b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cc800d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5565554d50e0 con 0x556555570780 2024-04-16T05:45:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cc800d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556555472520 con 0x55655556fe60 2024-04-16T05:45:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cc800d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565554d10a0 con 0x556555579360 2024-04-16T05:45:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x556555578e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cb7fff640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8cac0a3d20 0x7f8cac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x556555578e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40522/0 (socket says 172.21.15.32:40522) 2024-04-16T05:45:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 -- 172.21.15.32:0/3359648084 learned_addr learned my addr 172.21.15.32:0/3359648084 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55655556fe60 0x556555570240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 0x55655557b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cb7fff640 1 -- 172.21.15.32:0/888940832 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f8cb00f26c0 con 0x7f8cc0198690 2024-04-16T05:45:17.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbf7fe640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8cac0a3d20 0x7f8cac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 -- 172.21.15.32:0/3359648084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 msgr2=0x55655557b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 0x55655557b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 -- 172.21.15.32:0/3359648084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55655556fe60 msgr2=0x556555570240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55655556fe60 0x556555570240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 -- 172.21.15.32:0/3359648084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55655549a8d0 con 0x556555570780 2024-04-16T05:45:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55655556fe60 0x556555570240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x556555578e20 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80104a0 tx=0x7f8cb8010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409f247d1dadcc04 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cbf7fe640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8cac0a3d20 0x7f8cac0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00efb90 tx=0x7f8cb009d040 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cb57fa640 1 -- 172.21.15.32:0/3359648084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8017070 con 0x556555570780 2024-04-16T05:45:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cb57fa640 1 -- 172.21.15.32:0/3359648084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb8011be0 con 0x556555570780 2024-04-16T05:45:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.270+0000 7f8cb57fa640 1 -- 172.21.15.32:0/3359648084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800edb0 con 0x556555570780 2024-04-16T05:45:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 -- 172.21.15.32:0/3359648084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 msgr2=0x556555578e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x556555578e20 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80104a0 tx=0x7f8cb8010960 comp rx=0 tx=0).stop 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 -- 172.21.15.32:0/3359648084 shutdown_connections 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 0x55655557b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x556555578e20 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3359648084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55655556fe60 0x556555570240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 -- 172.21.15.32:0/3359648084 >> 172.21.15.32:0/3359648084 conn(0x55655556a590 msgr2=0x55655556a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 -- 172.21.15.32:0/3359648084 shutdown_connections 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.271+0000 7f8cc800d740 1 -- 172.21.15.32:0/3359648084 wait complete. 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 1 -- start start 2024-04-16T05:45:17.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 1 Processor -- start 2024-04-16T05:45:17.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 0x556555576390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x5565555768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555579360 0x556555576e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55655557c000 con 0x556555579360 2024-04-16T05:45:17.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55655548e4d0 con 0x556555570780 2024-04-16T05:45:17.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55655557c180 con 0x55655556fe60 2024-04-16T05:45:17.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cc800d740 10 client.0 _ll_register_callbacks cb 0x5565554732c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:17.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555579360 0x556555576e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555579360 0x556555576e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40534/0 (socket says 172.21.15.32:40534) 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 -- 172.21.15.32:0/1419674631 learned_addr learned my addr 172.21.15.32:0/1419674631 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 0x556555576390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x5565555768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 -- 172.21.15.32:0/1419674631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 msgr2=0x556555576390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 0x556555576390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 -- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 msgr2=0x5565555768d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x5565555768d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 -- 172.21.15.32:0/1419674631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565551e73f0 con 0x556555579360 2024-04-16T05:45:17.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x5565555768d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:17.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555579360 0x556555576e10 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80129a0 tx=0x7f8ca8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e46880437feb40b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.272+0000 7f8ca67fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:45:17.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8ca67fc640 1 -- 172.21.15.32:0/1419674631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8025060 con 0x556555579360 2024-04-16T05:45:17.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8ca67fc640 1 -- 172.21.15.32:0/1419674631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f8ca8010980 con 0x556555579360 2024-04-16T05:45:17.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 1 -- 172.21.15.32:0/1419674631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb8011be0 con 0x556555579360 2024-04-16T05:45:17.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8ca67fc640 1 -- 172.21.15.32:0/1419674631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8ca8010b80 con 0x556555579360 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 10 client.18009 fetch_fsmap learned FSMap version 25 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 1 -- 172.21.15.32:0/1419674631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565552b17d0 con 0x556555579360 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8ca67fc640 1 -- 172.21.15.32:0/1419674631 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb8011be0 con 0x556555579360 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8ca67fc640 1 -- 172.21.15.32:0/1419674631 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8ca8003aa0 con 0x556555579360 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 10 client.18009 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 10 client.18009 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 20 client.18009 populate_metadata read hostname 'smithi032' 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5565555893f0 0x55655558b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 4 client.18009 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:17.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.273+0000 7f8cc800d740 1 -- 172.21.15.32:0/1419674631 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565554e2b00 con 0x5565555893f0 2024-04-16T05:45:17.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.274+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5565555893f0 0x55655558b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.274+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5565555893f0 0x55655558b850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b0859cb165f81e5 server_cookie=125628ce63c9cba7 in_seq=0 out_seq=0 2024-04-16T05:45:17.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.274+0000 7f8ca67fc640 10 client.18009 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:45:17.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8ca67fc640 1 -- 172.21.15.32:0/1419674631 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565554e2b00 con 0x5565555893f0 2024-04-16T05:45:17.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8ca67fc640 10 client.18009 handle_command_reply: tid=0 2024-04-16T05:45:17.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 client.18009 shutdown 2024-04-16T05:45:17.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 -- 172.21.15.32:0/1419674631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555579360 msgr2=0x556555576e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555579360 0x556555576e10 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80129a0 tx=0x7f8ca8014040 comp rx=0 tx=0).stop 2024-04-16T05:45:17.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 -- 172.21.15.32:0/1419674631 shutdown_connections 2024-04-16T05:45:17.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5565555893f0 0x55655558b850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555579360 0x556555576e10 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x5565555768d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1419674631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 0x556555576390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8ca67fc640 0 client.18009 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:45:17.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 -- 172.21.15.32:0/1419674631 >> 172.21.15.32:0/1419674631 conn(0x55655556a590 msgr2=0x55655556c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 -- 172.21.15.32:0/1419674631 shutdown_connections 2024-04-16T05:45:17.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 1 -- 172.21.15.32:0/1419674631 wait complete. 2024-04-16T05:45:17.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.275+0000 7f8cc800d740 20 client.18009 trim_cache size 0 max 16384 2024-04-16T05:45:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.288+0000 7f8cc800d740 1 Processor -- start 2024-04-16T05:45:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.288+0000 7f8cc800d740 1 -- start start 2024-04-16T05:45:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 0x556555585090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x5565555695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555579360 0x556555591120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cc800d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01b1d50 con 0x556555570780 2024-04-16T05:45:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cc800d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01af620 con 0x556555579360 2024-04-16T05:45:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cc800d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ad490 con 0x55655556fe60 2024-04-16T05:45:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x5565555695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x5565555695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40546/0 (socket says 172.21.15.32:40546) 2024-04-16T05:45:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 -- 172.21.15.32:0/1491193428 learned_addr learned my addr 172.21.15.32:0/1491193428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/1491193428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 0x556555585090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 -- 172.21.15.32:0/1491193428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 msgr2=0x556555585090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/1491193428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 0x556555585090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 -- 172.21.15.32:0/1491193428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555579360 msgr2=0x556555591120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/1491193428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555579360 0x556555591120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 -- 172.21.15.32:0/1491193428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55655556a0c0 con 0x556555570780 2024-04-16T05:45:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/1491193428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x5565555695b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006010 tx=0x7f8cb8009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcc90b737332d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.289+0000 7f8cb4ff9640 1 -- 172.21.15.32:0/1491193428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8017040 con 0x556555570780 2024-04-16T05:45:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cb4ff9640 1 -- 172.21.15.32:0/1491193428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb802d070 con 0x556555570780 2024-04-16T05:45:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cb4ff9640 1 -- 172.21.15.32:0/1491193428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8020040 con 0x556555570780 2024-04-16T05:45:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 -- 172.21.15.32:0/1491193428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 msgr2=0x5565555695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1491193428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x5565555695b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006010 tx=0x7f8cb8009590 comp rx=0 tx=0).stop 2024-04-16T05:45:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 -- 172.21.15.32:0/1491193428 shutdown_connections 2024-04-16T05:45:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1491193428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555579360 0x556555591120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1491193428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556555570780 0x5565555695b0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 --2- 172.21.15.32:0/1491193428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55655556fe60 0x556555585090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 -- 172.21.15.32:0/1491193428 >> 172.21.15.32:0/1491193428 conn(0x55655556a590 msgr2=0x556555568ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 -- 172.21.15.32:0/1491193428 shutdown_connections 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.290+0000 7f8cc800d740 1 -- 172.21.15.32:0/1491193428 wait complete. 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 1 -- start start 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 1 Processor -- start 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55655556fe60 0x556555581970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x55655557cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 0x55655557d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01adae0 con 0x55655556fe60 2024-04-16T05:45:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01aef50 con 0x556555570780 2024-04-16T05:45:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ae1b0 con 0x556555579360 2024-04-16T05:45:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cc800d740 10 client.0 _ll_register_callbacks cb 0x7f8ca800bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55655556fe60 0x556555581970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55655556fe60 0x556555581970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40550/0 (socket says 172.21.15.32:40550) 2024-04-16T05:45:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 -- 172.21.15.32:0/3857281989 learned_addr learned my addr 172.21.15.32:0/3857281989 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 0x55655557d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 -- 172.21.15.32:0/3857281989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 msgr2=0x55655557d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 0x55655557d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x55655557cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 -- 172.21.15.32:0/3857281989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 msgr2=0x55655557cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x55655557cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 -- 172.21.15.32:0/3857281989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655557da00 con 0x55655556fe60 2024-04-16T05:45:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8cbe7fc640 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55655556fe60 0x556555581970 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80057c0 tx=0x7f8ca800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c58e142acf6c519 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.291+0000 7f8ca67fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:45:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8ca67fc640 1 -- 172.21.15.32:0/3857281989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8010980 con 0x55655556fe60 2024-04-16T05:45:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 1 -- 172.21.15.32:0/3857281989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb802d070 con 0x55655556fe60 2024-04-16T05:45:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8ca67fc640 1 -- 172.21.15.32:0/3857281989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f8ca80204e0 con 0x55655556fe60 2024-04-16T05:45:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8ca67fc640 1 -- 172.21.15.32:0/3857281989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8ca803f050 con 0x55655556fe60 2024-04-16T05:45:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 10 client.18015 fetch_fsmap learned FSMap version 25 2024-04-16T05:45:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 1 -- 172.21.15.32:0/3857281989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565555f3db0 con 0x55655556fe60 2024-04-16T05:45:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8ca67fc640 1 -- 172.21.15.32:0/3857281989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb802d070 con 0x55655556fe60 2024-04-16T05:45:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8ca67fc640 1 -- 172.21.15.32:0/3857281989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8ca80035f0 con 0x55655556fe60 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 10 client.18015 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 10 client.18015 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 20 client.18015 populate_metadata read hostname 'smithi032' 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5565555f4440 0x5565555f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 4 client.18015 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.292+0000 7f8cc800d740 1 -- 172.21.15.32:0/3857281989 --> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565554dc970 con 0x5565555f4440 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5565555f4440 0x5565555f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8ca67fc640 10 client.18015 ms_handle_connect on v2:172.21.15.82:6836/1413733447 2024-04-16T05:45:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cbeffd640 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5565555f4440 0x5565555f6860 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce79a996f0f6aeee server_cookie=48479a0108825807 in_seq=0 out_seq=0 2024-04-16T05:45:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8ca67fc640 1 -- 172.21.15.32:0/3857281989 <== mds.1 v2:172.21.15.82:6836/1413733447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565554dc970 con 0x5565555f4440 2024-04-16T05:45:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8ca67fc640 10 client.18015 handle_command_reply: tid=0 2024-04-16T05:45:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 client.18015 shutdown 2024-04-16T05:45:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 -- 172.21.15.32:0/3857281989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55655556fe60 msgr2=0x556555581970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55655556fe60 0x556555581970 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80057c0 tx=0x7f8ca800b390 comp rx=0 tx=0).stop 2024-04-16T05:45:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 -- 172.21.15.32:0/3857281989 shutdown_connections 2024-04-16T05:45:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.82:6836/1413733447,v1:172.21.15.82:6837/1413733447] conn(0x5565555f4440 0x5565555f6860 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556555579360 0x55655557d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8ca67fc640 0 client.18015 ms_handle_reset on v2:172.21.15.82:6836/1413733447 2024-04-16T05:45:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556555570780 0x55655557cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 --2- 172.21.15.32:0/3857281989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55655556fe60 0x556555581970 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 -- 172.21.15.32:0/3857281989 >> 172.21.15.32:0/3857281989 conn(0x55655556a590 msgr2=0x556555576be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 -- 172.21.15.32:0/3857281989 shutdown_connections 2024-04-16T05:45:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 1 -- 172.21.15.32:0/3857281989 wait complete. 2024-04-16T05:45:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.293+0000 7f8cc800d740 20 client.18015 trim_cache size 0 max 16384 2024-04-16T05:45:17.306 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:45:17.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 -- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8cac0a3d20 msgr2=0x7f8cac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8cac0a3d20 0x7f8cac0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00efb90 tx=0x7f8cb009d040 comp rx=0 tx=0).stop 2024-04-16T05:45:17.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 -- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0198690 msgr2=0x7f8cc0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0198690 0x7f8cc0198b10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009230 tx=0x7f8cb002cba0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 -- 172.21.15.32:0/888940832 shutdown_connections 2024-04-16T05:45:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8cac0a3d20 0x7f8cac0a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc0198690 0x7f8cc0198b10 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc0106200 0x7f8cc019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 --2- 172.21.15.32:0/888940832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc0101130 0x7f8cc019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 -- 172.21.15.32:0/888940832 >> 172.21.15.32:0/888940832 conn(0x7f8cc00780b0 msgr2=0x7f8cc00fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 -- 172.21.15.32:0/888940832 shutdown_connections 2024-04-16T05:45:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.294+0000 7f8cc5422640 1 -- 172.21.15.32:0/888940832 wait complete. 2024-04-16T05:45:17.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T05:45:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 -- 172.21.15.32:0/3008373805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 msgr2=0x7f648010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 --2- 172.21.15.32:0/3008373805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 0x7f648010dd30 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6478009900 tx=0x7f647802bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 -- 172.21.15.32:0/3008373805 shutdown_connections 2024-04-16T05:45:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 --2- 172.21.15.32:0/3008373805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 0x7f648010dd30 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 --2- 172.21.15.32:0/3008373805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6480101150 0x7f64801015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 --2- 172.21.15.32:0/3008373805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6480105e80 0x7f6480106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 -- 172.21.15.32:0/3008373805 >> 172.21.15.32:0/3008373805 conn(0x7f6480077cf0 msgr2=0x7f6480078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 -- 172.21.15.32:0/3008373805 shutdown_connections 2024-04-16T05:45:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.418+0000 7f648591c640 1 -- 172.21.15.32:0/3008373805 wait complete. 2024-04-16T05:45:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.419+0000 7f648591c640 1 Processor -- start 2024-04-16T05:45:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.419+0000 7f648591c640 1 -- start start 2024-04-16T05:45:17.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.419+0000 7f648591c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6480101150 0x7f6480197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.419+0000 7f648591c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 0x7f6480197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.419+0000 7f648591c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6480105e80 0x7f648019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.419+0000 7f648591c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6480111350 con 0x7f6480101af0 2024-04-16T05:45:17.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.419+0000 7f648591c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f64801111d0 con 0x7f6480101150 2024-04-16T05:45:17.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.419+0000 7f648591c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64801114d0 con 0x7f6480105e80 2024-04-16T05:45:17.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 0x7f6480197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 0x7f6480197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40560/0 (socket says 172.21.15.32:40560) 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 -- 172.21.15.32:0/4125070860 learned_addr learned my addr 172.21.15.32:0/4125070860 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647effd640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6480101150 0x7f6480197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647f7fe640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6480105e80 0x7f648019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 -- 172.21.15.32:0/4125070860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6480105e80 msgr2=0x7f648019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6480105e80 0x7f648019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 -- 172.21.15.32:0/4125070860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6480101150 msgr2=0x7f6480197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6480101150 0x7f6480197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 -- 172.21.15.32:0/4125070860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648019c740 con 0x7f6480101af0 2024-04-16T05:45:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f647e7fc640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 0x7f6480197c90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f646800d950 tx=0x7f646800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f648491a640 1 -- 172.21.15.32:0/4125070860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468017070 con 0x7f6480101af0 2024-04-16T05:45:17.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f648491a640 1 -- 172.21.15.32:0/4125070860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6468009c30 con 0x7f6480101af0 2024-04-16T05:45:17.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f648591c640 1 -- 172.21.15.32:0/4125070860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648019ca30 con 0x7f6480101af0 2024-04-16T05:45:17.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f648491a640 1 -- 172.21.15.32:0/4125070860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646800f6b0 con 0x7f6480101af0 2024-04-16T05:45:17.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.420+0000 7f648591c640 1 -- 172.21.15.32:0/4125070860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64801a42c0 con 0x7f6480101af0 2024-04-16T05:45:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.423+0000 7f648491a640 1 -- 172.21.15.32:0/4125070860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f64680109d0 con 0x7f6480101af0 2024-04-16T05:45:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.423+0000 7f64885e7740 1 Processor -- start 2024-04-16T05:45:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.423+0000 7f64885e7740 1 -- start start 2024-04-16T05:45:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f64885e7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc67b50 0x55937fc67f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f64885e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc73030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f64885e7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc73570 0x55937fc75960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f64885e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55937fbcd0e0 con 0x55937fc68470 2024-04-16T05:45:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f64885e7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55937fb6a520 con 0x55937fc73570 2024-04-16T05:45:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f64885e7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55937fbc90a0 con 0x55937fc67b50 2024-04-16T05:45:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc73030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc73030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40574/0 (socket says 172.21.15.32:40574) 2024-04-16T05:45:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 -- 172.21.15.32:0/1744641791 learned_addr learned my addr 172.21.15.32:0/1744641791 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647e7fc640 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc73570 0x55937fc75960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647f7fe640 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc67b50 0x55937fc67f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 -- 172.21.15.32:0/1744641791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc67b50 msgr2=0x55937fc67f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc67b50 0x55937fc67f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 -- 172.21.15.32:0/1744641791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc73570 msgr2=0x55937fc75960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc73570 0x55937fc75960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 -- 172.21.15.32:0/1744641791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937fb928d0 con 0x55937fc68470 2024-04-16T05:45:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f648491a640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f646c0a3da0 0x7f646c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647effd640 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc73030 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f64700114f0 tx=0x7f64700119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7f7df57a4f6859 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f647f7fe640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f646c0a3da0 0x7f646c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f648491a640 1 -- 172.21.15.32:0/4125070860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f64680d06e0 con 0x7f6480101af0 2024-04-16T05:45:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f6475ffb640 1 -- 172.21.15.32:0/1744641791 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470004d20 con 0x55937fc68470 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f6475ffb640 1 -- 172.21.15.32:0/1744641791 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6470004050 con 0x55937fc68470 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.424+0000 7f6475ffb640 1 -- 172.21.15.32:0/1744641791 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647000f7c0 con 0x55937fc68470 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 -- 172.21.15.32:0/1744641791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 msgr2=0x55937fc73030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f647f7fe640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f646c0a3da0 0x7f646c0a6260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6478005ab0 tx=0x7f64780058d0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc73030 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f64700114f0 tx=0x7f64700119b0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 -- 172.21.15.32:0/1744641791 shutdown_connections 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc73570 0x55937fc75960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc73030 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 --2- 172.21.15.32:0/1744641791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc67b50 0x55937fc67f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 -- 172.21.15.32:0/1744641791 >> 172.21.15.32:0/1744641791 conn(0x55937fc62590 msgr2=0x55937fc62970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 -- 172.21.15.32:0/1744641791 shutdown_connections 2024-04-16T05:45:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 -- 172.21.15.32:0/1744641791 wait complete. 2024-04-16T05:45:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.425+0000 7f64885e7740 1 -- start start 2024-04-16T05:45:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f64885e7740 1 Processor -- start 2024-04-16T05:45:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f64885e7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f64885e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc70a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f64885e7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 0x55937fc70f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f64885e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55937fc76360 con 0x55937fc68470 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f64885e7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55937fb864d0 con 0x55937fc67b50 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f64885e7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55937fc764e0 con 0x55937fc73570 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f64885e7740 10 client.0 _ll_register_callbacks cb 0x55937fb6b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc70a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc70a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40578/0 (socket says 172.21.15.32:40578) 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 -- 172.21.15.32:0/722969390 learned_addr learned my addr 172.21.15.32:0/722969390 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647f7fe640 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 0x55937fc70f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647effd640 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 -- 172.21.15.32:0/722969390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 msgr2=0x55937fc70f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 0x55937fc70f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 -- 172.21.15.32:0/722969390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 msgr2=0x55937fc704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc704f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 -- 172.21.15.32:0/722969390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937f8df3f0 con 0x55937fc68470 2024-04-16T05:45:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f647e7fc640 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc70a30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f64680a49d0 tx=0x7f6468005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c0b0cea230259a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.426+0000 7f645effd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:45:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f645effd640 1 -- 172.21.15.32:0/722969390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468020070 con 0x55937fc68470 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f645effd640 1 -- 172.21.15.32:0/722969390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f64680d0280 con 0x55937fc68470 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 1 -- 172.21.15.32:0/722969390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6470004050 con 0x55937fc68470 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f645effd640 1 -- 172.21.15.32:0/722969390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f64680cdec0 con 0x55937fc68470 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 10 client.18027 fetch_fsmap learned FSMap version 25 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 1 -- 172.21.15.32:0/722969390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55937f9a97d0 con 0x55937fc68470 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f645effd640 1 -- 172.21.15.32:0/722969390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6470004050 con 0x55937fc68470 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f645effd640 1 -- 172.21.15.32:0/722969390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f646809b900 con 0x55937fc68470 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 10 client.18027 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:45:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 20 client.18027 populate_metadata read hostname 'smithi032' 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55937fc83520 0x55937fc859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 4 client.18027 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f64885e7740 1 -- 172.21.15.32:0/722969390 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55937fbdab00 con 0x55937fc83520 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.427+0000 7f647effd640 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55937fc83520 0x55937fc859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.428+0000 7f647effd640 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55937fc83520 0x55937fc859b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b11590bb394e49e3 server_cookie=216d96a83dc62a26 in_seq=0 out_seq=0 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.428+0000 7f645effd640 10 client.18027 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.428+0000 7f645effd640 1 -- 172.21.15.32:0/722969390 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55937fbdab00 con 0x55937fc83520 2024-04-16T05:45:17.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.428+0000 7f645effd640 10 client.18027 handle_command_reply: tid=0 2024-04-16T05:45:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 client.18027 shutdown 2024-04-16T05:45:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 -- 172.21.15.32:0/722969390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 msgr2=0x55937fc70a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc70a30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f64680a49d0 tx=0x7f6468005880 comp rx=0 tx=0).stop 2024-04-16T05:45:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 -- 172.21.15.32:0/722969390 shutdown_connections 2024-04-16T05:45:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55937fc83520 0x55937fc859b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 0x55937fc70f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc70a30 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f645effd640 0 client.18027 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:45:17.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 --2- 172.21.15.32:0/722969390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 -- 172.21.15.32:0/722969390 >> 172.21.15.32:0/722969390 conn(0x55937fc62590 msgr2=0x55937fc642e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 -- 172.21.15.32:0/722969390 shutdown_connections 2024-04-16T05:45:17.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 1 -- 172.21.15.32:0/722969390 wait complete. 2024-04-16T05:45:17.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.429+0000 7f64885e7740 20 client.18027 trim_cache size 0 max 16384 2024-04-16T05:45:17.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.442+0000 7f64885e7740 1 Processor -- start 2024-04-16T05:45:17.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.442+0000 7f64885e7740 1 -- start start 2024-04-16T05:45:17.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.442+0000 7f64885e7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc7f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.442+0000 7f64885e7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc68470 0x55937fc7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.442+0000 7f64885e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc73570 0x55937fc63990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.442+0000 7f64885e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f64801af920 con 0x55937fc73570 2024-04-16T05:45:17.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.442+0000 7f64885e7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f64801ad1f0 con 0x55937fc67b50 2024-04-16T05:45:17.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.442+0000 7f64885e7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64801ab060 con 0x55937fc68470 2024-04-16T05:45:17.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc73570 0x55937fc63990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc73570 0x55937fc63990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40588/0 (socket says 172.21.15.32:40588) 2024-04-16T05:45:17.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 -- 172.21.15.32:0/718257484 learned_addr learned my addr 172.21.15.32:0/718257484 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647e7fc640 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc68470 0x55937fc7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647effd640 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc7f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 -- 172.21.15.32:0/718257484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc68470 msgr2=0x55937fc7bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc68470 0x55937fc7bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 -- 172.21.15.32:0/718257484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 msgr2=0x55937fc7f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc7f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 -- 172.21.15.32:0/718257484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937fc5de60 con 0x55937fc73570 2024-04-16T05:45:17.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f647f7fe640 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc73570 0x55937fc63990 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6478009900 tx=0x7f647802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41459e3f7e1b8668 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f64757fa640 1 -- 172.21.15.32:0/718257484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64780048c0 con 0x55937fc73570 2024-04-16T05:45:17.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f64757fa640 1 -- 172.21.15.32:0/718257484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64780053f0 con 0x55937fc73570 2024-04-16T05:45:17.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.443+0000 7f64757fa640 1 -- 172.21.15.32:0/718257484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478003970 con 0x55937fc73570 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 -- 172.21.15.32:0/718257484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc73570 msgr2=0x55937fc63990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc73570 0x55937fc63990 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6478009900 tx=0x7f647802e050 comp rx=0 tx=0).stop 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 -- 172.21.15.32:0/718257484 shutdown_connections 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc73570 0x55937fc63990 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc68470 0x55937fc7bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 --2- 172.21.15.32:0/718257484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc7f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 -- 172.21.15.32:0/718257484 >> 172.21.15.32:0/718257484 conn(0x55937fc62590 msgr2=0x55937fc6fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 -- 172.21.15.32:0/718257484 shutdown_connections 2024-04-16T05:45:17.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 -- 172.21.15.32:0/718257484 wait complete. 2024-04-16T05:45:17.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 -- start start 2024-04-16T05:45:17.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.444+0000 7f64885e7740 1 Processor -- start 2024-04-16T05:45:17.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f64885e7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc8aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f64885e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc8b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f64885e7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 0x55937fc8b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f64885e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f64801ab6b0 con 0x55937fc68470 2024-04-16T05:45:17.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f64885e7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f64801acb20 con 0x55937fc67b50 2024-04-16T05:45:17.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f64885e7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64801abd80 con 0x55937fc73570 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f64885e7740 10 client.0 _ll_register_callbacks cb 0x7f646800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc8b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc8b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40602/0 (socket says 172.21.15.32:40602) 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 -- 172.21.15.32:0/2277966262 learned_addr learned my addr 172.21.15.32:0/2277966262 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647f7fe640 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 0x55937fc8b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647effd640 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc8aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 -- 172.21.15.32:0/2277966262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 msgr2=0x55937fc8b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 0x55937fc8b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 -- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 msgr2=0x55937fc8aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T05:45:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc8aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 -- 172.21.15.32:0/2277966262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937fceab10 con 0x55937fc68470 2024-04-16T05:45:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647effd640 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc8aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f647e7fc640 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc8b1e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f646809c040 tx=0x7f64680d04b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc769e2b4d3d1ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f645effd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-16T05:45:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f645effd640 1 -- 172.21.15.32:0/2277966262 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468014070 con 0x55937fc68470 2024-04-16T05:45:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f645effd640 1 -- 172.21.15.32:0/2277966262 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f646809b790 con 0x55937fc68470 2024-04-16T05:45:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.445+0000 7f64885e7740 1 -- 172.21.15.32:0/2277966262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64780053f0 con 0x55937fc68470 2024-04-16T05:45:17.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f645effd640 1 -- 172.21.15.32:0/2277966262 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f646809c110 con 0x55937fc68470 2024-04-16T05:45:17.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f64885e7740 10 client.18033 fetch_fsmap learned FSMap version 25 2024-04-16T05:45:17.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f64885e7740 1 -- 172.21.15.32:0/2277966262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55937fcebdf0 con 0x55937fc68470 2024-04-16T05:45:17.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f645effd640 1 -- 172.21.15.32:0/2277966262 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64780053f0 con 0x55937fc68470 2024-04-16T05:45:17.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f645effd640 1 -- 172.21.15.32:0/2277966262 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6468010460 con 0x55937fc68470 2024-04-16T05:45:17.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f64885e7740 10 client.18033 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T05:45:17.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f64885e7740 10 client.18033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f64885e7740 20 client.18033 populate_metadata read hostname 'smithi032' 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f64885e7740 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55937fcec460 0x55937fcee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f64885e7740 4 client.18033 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f64885e7740 1 -- 172.21.15.32:0/2277966262 --> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55937fbd4970 con 0x55937fcec460 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.446+0000 7f647effd640 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55937fcec460 0x55937fcee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f645effd640 10 client.18033 ms_handle_connect on v2:172.21.15.82:6838/2453097092 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f647effd640 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55937fcec460 0x55937fcee880 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55bee8d57a7d290 server_cookie=632f6a0e6efb8007 in_seq=0 out_seq=0 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f645effd640 1 -- 172.21.15.32:0/2277966262 <== mds.0 v2:172.21.15.82:6838/2453097092 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55937fbd4970 con 0x55937fcec460 2024-04-16T05:45:17.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f645effd640 10 client.18033 handle_command_reply: tid=0 2024-04-16T05:45:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 client.18033 shutdown 2024-04-16T05:45:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 -- 172.21.15.32:0/2277966262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 msgr2=0x55937fc8b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc8b1e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f646809c040 tx=0x7f64680d04b0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 -- 172.21.15.32:0/2277966262 shutdown_connections 2024-04-16T05:45:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:6838/2453097092,v1:172.21.15.82:6839/2453097092] conn(0x55937fcec460 0x55937fcee880 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55937fc73570 0x55937fc8b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fc68470 0x55937fc8b1e0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 --2- 172.21.15.32:0/2277966262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55937fc67b50 0x55937fc8aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f645effd640 0 client.18033 ms_handle_reset on v2:172.21.15.82:6838/2453097092 2024-04-16T05:45:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 -- 172.21.15.32:0/2277966262 >> 172.21.15.32:0/2277966262 conn(0x55937fc62590 msgr2=0x55937fc71b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 -- 172.21.15.32:0/2277966262 shutdown_connections 2024-04-16T05:45:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 1 -- 172.21.15.32:0/2277966262 wait complete. 2024-04-16T05:45:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.447+0000 7f64885e7740 20 client.18033 trim_cache size 0 max 16384 2024-04-16T05:45:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-16T05:45:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 -- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f646c0a3da0 msgr2=0x7f646c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f646c0a3da0 0x7f646c0a6260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6478005ab0 tx=0x7f64780058d0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 -- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 msgr2=0x7f6480197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 0x7f6480197c90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f646800d950 tx=0x7f646800de10 comp rx=0 tx=0).stop 2024-04-16T05:45:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 -- 172.21.15.32:0/4125070860 shutdown_connections 2024-04-16T05:45:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6480105e80 0x7f648019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f646c0a3da0 0x7f646c0a6260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6480101af0 0x7f6480197c90 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 --2- 172.21.15.32:0/4125070860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6480101150 0x7f6480197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 -- 172.21.15.32:0/4125070860 >> 172.21.15.32:0/4125070860 conn(0x7f6480077cf0 msgr2=0x7f64800fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 -- 172.21.15.32:0/4125070860 shutdown_connections 2024-04-16T05:45:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:17.448+0000 7f648591c640 1 -- 172.21.15.32:0/4125070860 wait complete. 2024-04-16T05:45:18.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:17 smithi134 ceph-mon[36153]: pgmap v321: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 9 op/s 2024-04-16T05:45:18.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:17 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1192993635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:18.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:17 smithi082 ceph-mon[36195]: pgmap v321: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 9 op/s 2024-04-16T05:45:18.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:17 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1192993635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:18.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:17 smithi032 ceph-mon[32351]: pgmap v321: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 9 op/s 2024-04-16T05:45:18.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:17 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1192993635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:45:18.461 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-16T05:45:18.461 INFO:tasks.fwd_scrub:done joining 2024-04-16T05:45:18.462 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-16T05:45:18.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:45:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 -- 172.21.15.32:0/707647615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 msgr2=0x7f2b2410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 --2- 172.21.15.32:0/707647615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 0x7f2b2410de10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009ca0 tx=0x7f2b1c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 -- 172.21.15.32:0/707647615 shutdown_connections 2024-04-16T05:45:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 --2- 172.21.15.32:0/707647615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 0x7f2b2410de10 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 --2- 172.21.15.32:0/707647615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b24101b70 0x7f2b24105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 --2- 172.21.15.32:0/707647615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241011c0 0x7f2b241015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 -- 172.21.15.32:0/707647615 >> 172.21.15.32:0/707647615 conn(0x7f2b24078060 msgr2=0x7f2b24078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 -- 172.21.15.32:0/707647615 shutdown_connections 2024-04-16T05:45:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 -- 172.21.15.32:0/707647615 wait complete. 2024-04-16T05:45:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 Processor -- start 2024-04-16T05:45:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.593+0000 7f2b2c293640 1 -- start start 2024-04-16T05:45:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2c293640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241011c0 0x7f2b241a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2c293640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 0x7f2b241a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2c293640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b2419a830 0x7f2b2419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2c293640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b24112410 con 0x7f2b24106240 2024-04-16T05:45:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2c293640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b24112290 con 0x7f2b2419a830 2024-04-16T05:45:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2c293640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b24112590 con 0x7f2b241011c0 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 0x7f2b241a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 0x7f2b241a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57146/0 (socket says 172.21.15.32:57146) 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 -- 172.21.15.32:0/2957120430 learned_addr learned my addr 172.21.15.32:0/2957120430 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2a830640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b2419a830 0x7f2b2419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 -- 172.21.15.32:0/2957120430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241011c0 msgr2=0x7f2b241a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241011c0 0x7f2b241a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 -- 172.21.15.32:0/2957120430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b2419a830 msgr2=0x7f2b2419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b2419a830 0x7f2b2419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 -- 172.21.15.32:0/2957120430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2419b3c0 con 0x7f2b24106240 2024-04-16T05:45:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2a830640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b2419a830 0x7f2b2419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2982e640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 0x7f2b241a0c00 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d770 tx=0x7f2b1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b1b7fe640 1 -- 172.21.15.32:0/2957120430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14017070 con 0x7f2b24106240 2024-04-16T05:45:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b1b7fe640 1 -- 172.21.15.32:0/2957120430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b14009c30 con 0x7f2b24106240 2024-04-16T05:45:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2419b6b0 con 0x7f2b24106240 2024-04-16T05:45:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b1b7fe640 1 -- 172.21.15.32:0/2957120430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400f620 con 0x7f2b24106240 2024-04-16T05:45:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.594+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b241a7890 con 0x7f2b24106240 2024-04-16T05:45:18.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.596+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b24102640 con 0x7f2b24106240 2024-04-16T05:45:18.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.597+0000 7f2b1b7fe640 1 -- 172.21.15.32:0/2957120430 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b1401a070 con 0x7f2b24106240 2024-04-16T05:45:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.597+0000 7f2b1b7fe640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2b100a3d20 0x7f2b100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.598+0000 7f2b2a02f640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2b100a3d20 0x7f2b100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.598+0000 7f2b1b7fe640 1 -- 172.21.15.32:0/2957120430 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f2b140d06c0 con 0x7f2b24106240 2024-04-16T05:45:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.598+0000 7f2b2a02f640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2b100a3d20 0x7f2b100a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000a990 tx=0x7f2b20005c70 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.601+0000 7f2b1b7fe640 1 -- 172.21.15.32:0/2957120430 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b2419b6b0 con 0x7f2b24106240 2024-04-16T05:45:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.788+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b24101b70 con 0x7f2b24106240 2024-04-16T05:45:18.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.790+0000 7f2b1b7fe640 1 -- 172.21.15.32:0/2957120430 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25177 (secure 0 0 0) 0x7f2b14009de0 con 0x7f2b24106240 2024-04-16T05:45:18.789 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:45:18.789 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":100,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:45:12.798785+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1356969979":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6800/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/558122612":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2362160984":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1729232468":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:45:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2b100a3d20 msgr2=0x7f2b100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2b100a3d20 0x7f2b100a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000a990 tx=0x7f2b20005c70 comp rx=0 tx=0).stop 2024-04-16T05:45:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 msgr2=0x7f2b241a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 0x7f2b241a0c00 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d770 tx=0x7f2b1400dc30 comp rx=0 tx=0).stop 2024-04-16T05:45:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 shutdown_connections 2024-04-16T05:45:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b2419a830 0x7f2b2419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f2b100a3d20 0x7f2b100a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b24106240 0x7f2b241a0c00 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 --2- 172.21.15.32:0/2957120430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241011c0 0x7f2b241a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.792+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 >> 172.21.15.32:0/2957120430 conn(0x7f2b24078060 msgr2=0x7f2b2410c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.793+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 shutdown_connections 2024-04-16T05:45:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.793+0000 7f2b2c293640 1 -- 172.21.15.32:0/2957120430 wait complete. 2024-04-16T05:45:18.803 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:45:18.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:45:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 -- 172.21.15.32:0/2243634915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980101af0 msgr2=0x7f4980105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 --2- 172.21.15.32:0/2243634915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980101af0 0x7f4980105b30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4970009900 tx=0x7f497002bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 -- 172.21.15.32:0/2243634915 shutdown_connections 2024-04-16T05:45:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 --2- 172.21.15.32:0/2243634915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4980106210 0x7f498010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 --2- 172.21.15.32:0/2243634915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980101af0 0x7f4980105b30 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 --2- 172.21.15.32:0/2243634915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4980101140 0x7f4980101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 -- 172.21.15.32:0/2243634915 >> 172.21.15.32:0/2243634915 conn(0x7f4980078000 msgr2=0x7f4980078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 -- 172.21.15.32:0/2243634915 shutdown_connections 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 -- 172.21.15.32:0/2243634915 wait complete. 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 Processor -- start 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 -- start start 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.914+0000 7f49876eb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4980101140 0x7f4980197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49876eb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4980106210 0x7f4980197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49876eb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f498019c000 0x7f498019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49876eb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49801112a0 con 0x7f498019c000 2024-04-16T05:45:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49876eb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4980111120 con 0x7f4980101140 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49876eb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4980111420 con 0x7f4980106210 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f498019c000 0x7f498019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f498019c000 0x7f498019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57160/0 (socket says 172.21.15.32:57160) 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 -- 172.21.15.32:0/3274788965 learned_addr learned my addr 172.21.15.32:0/3274788965 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985487640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4980101140 0x7f4980197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4984c86640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4980106210 0x7f4980197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 -- 172.21.15.32:0/3274788965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4980106210 msgr2=0x7f4980197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4980106210 0x7f4980197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 -- 172.21.15.32:0/3274788965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4980101140 msgr2=0x7f4980197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:18.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4980101140 0x7f4980197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:18.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 -- 172.21.15.32:0/3274788965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498019caf0 con 0x7f498019c000 2024-04-16T05:45:18.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f4985c88640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f498019c000 0x7f498019c4b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f497800eaa0 tx=0x7f497800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:18.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49767fc640 1 -- 172.21.15.32:0/3274788965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978004d20 con 0x7f498019c000 2024-04-16T05:45:18.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49767fc640 1 -- 172.21.15.32:0/3274788965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4978005020 con 0x7f498019c000 2024-04-16T05:45:18.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498019cde0 con 0x7f498019c000 2024-04-16T05:45:18.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49767fc640 1 -- 172.21.15.32:0/3274788965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49780116d0 con 0x7f498019c000 2024-04-16T05:45:18.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.915+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49801a45a0 con 0x7f498019c000 2024-04-16T05:45:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.916+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4980068e20 con 0x7f498019c000 2024-04-16T05:45:18.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.917+0000 7f49767fc640 1 -- 172.21.15.32:0/3274788965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49780051d0 con 0x7f498019c000 2024-04-16T05:45:18.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.918+0000 7f49767fc640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f496c0a3d20 0x7f496c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:18.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.918+0000 7f4985487640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f496c0a3d20 0x7f496c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:18.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.918+0000 7f49767fc640 1 -- 172.21.15.32:0/3274788965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f49780d1100 con 0x7f498019c000 2024-04-16T05:45:18.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.919+0000 7f4985487640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f496c0a3d20 0x7f496c0a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f497c00abe0 tx=0x7f497c00a2d0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:18.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:18.922+0000 7f49767fc640 1 -- 172.21.15.32:0/3274788965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4978016030 con 0x7f498019c000 2024-04-16T05:45:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.108+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4980101af0 con 0x7f498019c000 2024-04-16T05:45:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.109+0000 7f49767fc640 1 -- 172.21.15.32:0/3274788965 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f497809d410 con 0x7f498019c000 2024-04-16T05:45:19.108 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:45:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f496c0a3d20 msgr2=0x7f496c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f496c0a3d20 0x7f496c0a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f497c00abe0 tx=0x7f497c00a2d0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f498019c000 msgr2=0x7f498019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f498019c000 0x7f498019c4b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f497800eaa0 tx=0x7f497800eea0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 shutdown_connections 2024-04-16T05:45:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f496c0a3d20 0x7f496c0a61e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f498019c000 0x7f498019c4b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4980106210 0x7f4980197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 --2- 172.21.15.32:0/3274788965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4980101140 0x7f4980197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 >> 172.21.15.32:0/3274788965 conn(0x7f4980078000 msgr2=0x7f49800fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 shutdown_connections 2024-04-16T05:45:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.111+0000 7f49876eb640 1 -- 172.21.15.32:0/3274788965 wait complete. 2024-04-16T05:45:19.119 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:18 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2957120430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:19.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:45:19.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:18 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2957120430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:19.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:18 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2957120430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:19.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 -- 172.21.15.32:0/646560005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478103280 msgr2=0x7f5478103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 --2- 172.21.15.32:0/646560005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478103280 0x7f5478103660 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5468009ca0 tx=0x7f546802c230 comp rx=0 tx=0).stop 2024-04-16T05:45:19.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 -- 172.21.15.32:0/646560005 shutdown_connections 2024-04-16T05:45:19.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 --2- 172.21.15.32:0/646560005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f547810c5c0 0x7f547810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 --2- 172.21.15.32:0/646560005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5478103ba0 0x7f547810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 --2- 172.21.15.32:0/646560005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5478103280 0x7f5478103660 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 -- 172.21.15.32:0/646560005 >> 172.21.15.32:0/646560005 conn(0x7f5478078000 msgr2=0x7f5478078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:19.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 -- 172.21.15.32:0/646560005 shutdown_connections 2024-04-16T05:45:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.232+0000 7f547ce35640 1 -- 172.21.15.32:0/646560005 wait complete. 2024-04-16T05:45:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547ce35640 1 Processor -- start 2024-04-16T05:45:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547ce35640 1 -- start start 2024-04-16T05:45:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547ce35640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5478103280 0x7f5478100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547ce35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478103ba0 0x7f5478101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547ce35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f547810c5c0 0x7f5478101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547ce35640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54781123a0 con 0x7f547810c5c0 2024-04-16T05:45:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547ce35640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5478112220 con 0x7f5478103ba0 2024-04-16T05:45:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547ce35640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5478112520 con 0x7f5478103280 2024-04-16T05:45:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5476d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f547810c5c0 0x7f5478101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5476d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f547810c5c0 0x7f5478101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57168/0 (socket says 172.21.15.32:57168) 2024-04-16T05:45:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5476d9d640 1 -- 172.21.15.32:0/254306483 learned_addr learned my addr 172.21.15.32:0/254306483 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f547659c640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5478103280 0x7f5478100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5475d9b640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478103ba0 0x7f5478101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5476d9d640 1 -- 172.21.15.32:0/254306483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5478103280 msgr2=0x7f5478100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5476d9d640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5478103280 0x7f5478100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5476d9d640 1 -- 172.21.15.32:0/254306483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478103ba0 msgr2=0x7f5478101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5476d9d640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478103ba0 0x7f5478101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5476d9d640 1 -- 172.21.15.32:0/254306483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54781a6e80 con 0x7f547810c5c0 2024-04-16T05:45:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.233+0000 7f5475d9b640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478103ba0 0x7f5478101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.234+0000 7f5476d9d640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f547810c5c0 0x7f5478101950 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f547000c970 tx=0x7f547000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.234+0000 7f54677fe640 1 -- 172.21.15.32:0/254306483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470007bb0 con 0x7f547810c5c0 2024-04-16T05:45:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.234+0000 7f54677fe640 1 -- 172.21.15.32:0/254306483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5470007d50 con 0x7f547810c5c0 2024-04-16T05:45:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.234+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54781a7170 con 0x7f547810c5c0 2024-04-16T05:45:19.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.234+0000 7f54677fe640 1 -- 172.21.15.32:0/254306483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547000f450 con 0x7f547810c5c0 2024-04-16T05:45:19.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.234+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54781a7670 con 0x7f547810c5c0 2024-04-16T05:45:19.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.235+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5478068e20 con 0x7f547810c5c0 2024-04-16T05:45:19.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.236+0000 7f54677fe640 1 -- 172.21.15.32:0/254306483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5470004f60 con 0x7f547810c5c0 2024-04-16T05:45:19.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.236+0000 7f54677fe640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54600a3d20 0x7f54600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.237+0000 7f547659c640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54600a3d20 0x7f54600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.237+0000 7f54677fe640 1 -- 172.21.15.32:0/254306483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f54700d08a0 con 0x7f547810c5c0 2024-04-16T05:45:19.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.237+0000 7f547659c640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54600a3d20 0x7f54600a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5468004af0 tx=0x7f5468005c50 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:19.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.241+0000 7f54677fe640 1 -- 172.21.15.32:0/254306483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5470014030 con 0x7f547810c5c0 2024-04-16T05:45:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.425+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5478107740 con 0x7f547810c5c0 2024-04-16T05:45:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.426+0000 7f54677fe640 1 -- 172.21.15.32:0/254306483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f547009cb50 con 0x7f547810c5c0 2024-04-16T05:45:19.424 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:45:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54600a3d20 msgr2=0x7f54600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54600a3d20 0x7f54600a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5468004af0 tx=0x7f5468005c50 comp rx=0 tx=0).stop 2024-04-16T05:45:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f547810c5c0 msgr2=0x7f5478101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f547810c5c0 0x7f5478101950 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f547000c970 tx=0x7f547000ce30 comp rx=0 tx=0).stop 2024-04-16T05:45:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 shutdown_connections 2024-04-16T05:45:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f54600a3d20 0x7f54600a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f547810c5c0 0x7f5478101950 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478103ba0 0x7f5478101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 --2- 172.21.15.32:0/254306483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5478103280 0x7f5478100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 >> 172.21.15.32:0/254306483 conn(0x7f5478078000 msgr2=0x7f54780ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 shutdown_connections 2024-04-16T05:45:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.428+0000 7f547ce35640 1 -- 172.21.15.32:0/254306483 wait complete. 2024-04-16T05:45:19.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:45:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.547+0000 7fefb99f4640 1 -- 172.21.15.32:0/1304073138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb4106210 msgr2=0x7fefb410dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.547+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1304073138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb4106210 0x7fefb410dde0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009900 tx=0x7fefa402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.547+0000 7fefb99f4640 1 -- 172.21.15.32:0/1304073138 shutdown_connections 2024-04-16T05:45:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.547+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1304073138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb4106210 0x7fefb410dde0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.547+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1304073138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefb4101af0 0x7fefb4105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.547+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1304073138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefb4101140 0x7fefb4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.547+0000 7fefb99f4640 1 -- 172.21.15.32:0/1304073138 >> 172.21.15.32:0/1304073138 conn(0x7fefb4078000 msgr2=0x7fefb4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 -- 172.21.15.32:0/1304073138 shutdown_connections 2024-04-16T05:45:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 -- 172.21.15.32:0/1304073138 wait complete. 2024-04-16T05:45:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 Processor -- start 2024-04-16T05:45:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 -- start start 2024-04-16T05:45:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefb4101140 0x7fefb419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefb4106210 0x7fefb419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb41975b0 0x7fefb4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41112d0 con 0x7fefb41975b0 2024-04-16T05:45:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4111150 con 0x7fefb4106210 2024-04-16T05:45:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.548+0000 7fefb99f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4111450 con 0x7fefb4101140 2024-04-16T05:45:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb41975b0 0x7fefb4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb41975b0 0x7fefb4197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57182/0 (socket says 172.21.15.32:57182) 2024-04-16T05:45:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 -- 172.21.15.32:0/1692230303 learned_addr learned my addr 172.21.15.32:0/1692230303 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb2ffd640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefb4101140 0x7fefb419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 -- 172.21.15.32:0/1692230303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefb4101140 msgr2=0x7fefb419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb27fc640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefb4106210 0x7fefb419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefb4101140 0x7fefb419d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 -- 172.21.15.32:0/1692230303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefb4106210 msgr2=0x7fefb419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefb4106210 0x7fefb419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 -- 172.21.15.32:0/1692230303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb41980d0 con 0x7fefb41975b0 2024-04-16T05:45:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb37fe640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb41975b0 0x7fefb4197a60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009230 tx=0x7fefa402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb89f2640 1 -- 172.21.15.32:0/1692230303 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4041070 con 0x7fefb41975b0 2024-04-16T05:45:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb89f2640 1 -- 172.21.15.32:0/1692230303 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4004450 con 0x7fefb41975b0 2024-04-16T05:45:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb4198360 con 0x7fefb41975b0 2024-04-16T05:45:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb89f2640 1 -- 172.21.15.32:0/1692230303 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4034440 con 0x7fefb41975b0 2024-04-16T05:45:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.549+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41a4550 con 0x7fefb41975b0 2024-04-16T05:45:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.550+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb4068e20 con 0x7fefb41975b0 2024-04-16T05:45:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.551+0000 7fefb89f2640 1 -- 172.21.15.32:0/1692230303 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fefa4044070 con 0x7fefb41975b0 2024-04-16T05:45:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.552+0000 7fefb89f2640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fefa00a3d20 0x7fefa00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.552+0000 7fefb2ffd640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fefa00a3d20 0x7fefa00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.552+0000 7fefb89f2640 1 -- 172.21.15.32:0/1692230303 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7fefa40f2740 con 0x7fefb41975b0 2024-04-16T05:45:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.553+0000 7fefb2ffd640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fefa00a3d20 0x7fefa00a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefac005c40 tx=0x7fefac008230 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:19.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.557+0000 7fefb89f2640 1 -- 172.21.15.32:0/1692230303 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefa40beb60 con 0x7fefb41975b0 2024-04-16T05:45:19.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.744+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fefb4101af0 con 0x7fefb41975b0 2024-04-16T05:45:19.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.744+0000 7fefb89f2640 1 -- 172.21.15.32:0/1692230303 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fefa40be460 con 0x7fefb41975b0 2024-04-16T05:45:19.743 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:45:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.746+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fefa00a3d20 msgr2=0x7fefa00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.746+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fefa00a3d20 0x7fefa00a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefac005c40 tx=0x7fefac008230 comp rx=0 tx=0).stop 2024-04-16T05:45:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.746+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb41975b0 msgr2=0x7fefb4197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.746+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb41975b0 0x7fefb4197a60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009230 tx=0x7fefa402cbf0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.747+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 shutdown_connections 2024-04-16T05:45:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.747+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fefa00a3d20 0x7fefa00a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.747+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb41975b0 0x7fefb4197a60 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.747+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefb4106210 0x7fefb419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.747+0000 7fefb99f4640 1 --2- 172.21.15.32:0/1692230303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefb4101140 0x7fefb419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.747+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 >> 172.21.15.32:0/1692230303 conn(0x7fefb4078000 msgr2=0x7fefb40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.747+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 shutdown_connections 2024-04-16T05:45:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.747+0000 7fefb99f4640 1 -- 172.21.15.32:0/1692230303 wait complete. 2024-04-16T05:45:19.756 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:45:19.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.866+0000 7f9c57300640 1 -- 172.21.15.32:0/2050153939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5006a870 msgr2=0x7f9c5006ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.866+0000 7f9c57300640 1 --2- 172.21.15.32:0/2050153939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5006a870 0x7f9c5006ac50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009900 tx=0x7f9c4c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.866+0000 7f9c57300640 1 -- 172.21.15.32:0/2050153939 shutdown_connections 2024-04-16T05:45:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.866+0000 7f9c57300640 1 --2- 172.21.15.32:0/2050153939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c50107330 0x7f9c5010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.866+0000 7f9c57300640 1 --2- 172.21.15.32:0/2050153939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c5006b220 0x7f9c50106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.866+0000 7f9c57300640 1 --2- 172.21.15.32:0/2050153939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5006a870 0x7f9c5006ac50 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.866+0000 7f9c57300640 1 -- 172.21.15.32:0/2050153939 >> 172.21.15.32:0/2050153939 conn(0x7f9c50076d00 msgr2=0x7f9c50077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 -- 172.21.15.32:0/2050153939 shutdown_connections 2024-04-16T05:45:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 -- 172.21.15.32:0/2050153939 wait complete. 2024-04-16T05:45:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 Processor -- start 2024-04-16T05:45:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 -- start start 2024-04-16T05:45:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5006a870 0x7f9c5019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c50107330 0x7f9c5019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c501986c0 0x7f9c50198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c501123c0 con 0x7f9c50107330 2024-04-16T05:45:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50112240 con 0x7f9c501986c0 2024-04-16T05:45:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c57300640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50112540 con 0x7f9c5006a870 2024-04-16T05:45:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.867+0000 7f9c5489b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c50107330 0x7f9c5019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5489b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c50107330 0x7f9c5019eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57196/0 (socket says 172.21.15.32:57196) 2024-04-16T05:45:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5489b640 1 -- 172.21.15.32:0/330934684 learned_addr learned my addr 172.21.15.32:0/330934684 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5509c640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5006a870 0x7f9c5019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5589d640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c501986c0 0x7f9c50198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5489b640 1 -- 172.21.15.32:0/330934684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5006a870 msgr2=0x7f9c5019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5489b640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5006a870 0x7f9c5019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5489b640 1 -- 172.21.15.32:0/330934684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c501986c0 msgr2=0x7f9c50198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5489b640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c501986c0 0x7f9c50198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5489b640 1 -- 172.21.15.32:0/330934684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c501992c0 con 0x7f9c50107330 2024-04-16T05:45:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5589d640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c501986c0 0x7f9c50198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c5489b640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c50107330 0x7f9c5019eaf0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c970 tx=0x7f9c4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c467fc640 1 -- 172.21.15.32:0/330934684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40007bb0 con 0x7f9c50107330 2024-04-16T05:45:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c467fc640 1 -- 172.21.15.32:0/330934684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c40007d50 con 0x7f9c50107330 2024-04-16T05:45:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c501a5350 con 0x7f9c50107330 2024-04-16T05:45:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c467fc640 1 -- 172.21.15.32:0/330934684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000f450 con 0x7f9c50107330 2024-04-16T05:45:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.868+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c501a5750 con 0x7f9c50107330 2024-04-16T05:45:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.869+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c50069e50 con 0x7f9c50107330 2024-04-16T05:45:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.870+0000 7f9c467fc640 1 -- 172.21.15.32:0/330934684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9c40004f40 con 0x7f9c50107330 2024-04-16T05:45:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.871+0000 7f9c467fc640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c3c0a3d20 0x7f9c3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.871+0000 7f9c5509c640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c3c0a3d20 0x7f9c3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.871+0000 7f9c467fc640 1 -- 172.21.15.32:0/330934684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f9c400d0cb0 con 0x7f9c50107330 2024-04-16T05:45:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.871+0000 7f9c5509c640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c3c0a3d20 0x7f9c3c0a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c005dd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:19.875+0000 7f9c467fc640 1 -- 172.21.15.32:0/330934684 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c40014030 con 0x7f9c50107330 2024-04-16T05:45:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.059+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9c50069270 con 0x7f9c50107330 2024-04-16T05:45:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.060+0000 7f9c467fc640 1 -- 172.21.15.32:0/330934684 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7f9c4009cfc0 con 0x7f9c50107330 2024-04-16T05:45:20.059 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:45:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c3c0a3d20 msgr2=0x7f9c3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c3c0a3d20 0x7f9c3c0a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c005dd0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c50107330 msgr2=0x7f9c5019eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c50107330 0x7f9c5019eaf0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c970 tx=0x7f9c4000ce30 comp rx=0 tx=0).stop 2024-04-16T05:45:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 shutdown_connections 2024-04-16T05:45:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c501986c0 0x7f9c50198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f9c3c0a3d20 0x7f9c3c0a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c50107330 0x7f9c5019eaf0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 --2- 172.21.15.32:0/330934684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c5006a870 0x7f9c5019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 >> 172.21.15.32:0/330934684 conn(0x7f9c50076d00 msgr2=0x7f9c500ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 shutdown_connections 2024-04-16T05:45:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.062+0000 7f9c57300640 1 -- 172.21.15.32:0/330934684 wait complete. 2024-04-16T05:45:20.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:19 smithi032 ceph-mon[32351]: pgmap v322: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:45:20.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:19 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3274788965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:19 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/254306483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:19 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1692230303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-16T05:45:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:19 smithi134 ceph-mon[36153]: pgmap v322: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:45:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:19 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3274788965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:19 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/254306483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:19 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1692230303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:19 smithi082 ceph-mon[36195]: pgmap v322: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:45:20.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:19 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3274788965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:19 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/254306483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:19 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1692230303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 -- 172.21.15.32:0/319066483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101af0 msgr2=0x7fba60105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 --2- 172.21.15.32:0/319066483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101af0 0x7fba60105b30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fba50009ca0 tx=0x7fba5002c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 -- 172.21.15.32:0/319066483 shutdown_connections 2024-04-16T05:45:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 --2- 172.21.15.32:0/319066483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60106210 0x7fba6010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 --2- 172.21.15.32:0/319066483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101af0 0x7fba60105b30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 --2- 172.21.15.32:0/319066483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba60101140 0x7fba60101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 -- 172.21.15.32:0/319066483 >> 172.21.15.32:0/319066483 conn(0x7fba60077280 msgr2=0x7fba60077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 -- 172.21.15.32:0/319066483 shutdown_connections 2024-04-16T05:45:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.178+0000 7fba66c74640 1 -- 172.21.15.32:0/319066483 wait complete. 2024-04-16T05:45:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba66c74640 1 Processor -- start 2024-04-16T05:45:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba66c74640 1 -- start start 2024-04-16T05:45:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba66c74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101140 0x7fba601a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba66c74640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60106210 0x7fba601a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba66c74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6019a900 0x7fba6019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba66c74640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba601112a0 con 0x7fba60101140 2024-04-16T05:45:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba66c74640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fba60111120 con 0x7fba60106210 2024-04-16T05:45:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba66c74640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fba60111420 con 0x7fba6019a900 2024-04-16T05:45:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba64a10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101140 0x7fba601a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba64a10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101140 0x7fba601a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57210/0 (socket says 172.21.15.32:57210) 2024-04-16T05:45:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba64a10640 1 -- 172.21.15.32:0/4088426013 learned_addr learned my addr 172.21.15.32:0/4088426013 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba5ffff640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60106210 0x7fba601a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba65211640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6019a900 0x7fba6019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba64a10640 1 -- 172.21.15.32:0/4088426013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6019a900 msgr2=0x7fba6019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba64a10640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6019a900 0x7fba6019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba64a10640 1 -- 172.21.15.32:0/4088426013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60106210 msgr2=0x7fba601a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba64a10640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60106210 0x7fba601a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba64a10640 1 -- 172.21.15.32:0/4088426013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6019b420 con 0x7fba60101140 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.179+0000 7fba5ffff640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60106210 0x7fba601a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.180+0000 7fba64a10640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101140 0x7fba601a0750 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fba5800c970 tx=0x7fba5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.180+0000 7fba5dffb640 1 -- 172.21.15.32:0/4088426013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58007bb0 con 0x7fba60101140 2024-04-16T05:45:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.180+0000 7fba5dffb640 1 -- 172.21.15.32:0/4088426013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba58007d50 con 0x7fba60101140 2024-04-16T05:45:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.180+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba6019b710 con 0x7fba60101140 2024-04-16T05:45:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.180+0000 7fba5dffb640 1 -- 172.21.15.32:0/4088426013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5800f450 con 0x7fba60101140 2024-04-16T05:45:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.180+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba601a7910 con 0x7fba60101140 2024-04-16T05:45:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.181+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba60069270 con 0x7fba60101140 2024-04-16T05:45:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.182+0000 7fba5dffb640 1 -- 172.21.15.32:0/4088426013 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba58005020 con 0x7fba60101140 2024-04-16T05:45:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.182+0000 7fba5dffb640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fba4c0a3d20 0x7fba4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.183+0000 7fba5ffff640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fba4c0a3d20 0x7fba4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.183+0000 7fba5dffb640 1 -- 172.21.15.32:0/4088426013 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7fba580d0ab0 con 0x7fba60101140 2024-04-16T05:45:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.183+0000 7fba5ffff640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fba4c0a3d20 0x7fba4c0a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fba5002c700 tx=0x7fba50005c30 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.187+0000 7fba5dffb640 1 -- 172.21.15.32:0/4088426013 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba58014030 con 0x7fba60101140 2024-04-16T05:45:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.425+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fba6019be30 con 0x7fba60101140 2024-04-16T05:45:20.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.434+0000 7fba5dffb640 1 -- 172.21.15.32:0/4088426013 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746063 (secure 0 0 0) 0x7fba5809cd60 con 0x7fba60101140 2024-04-16T05:45:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:45:20.442 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:45:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 26, 2024-04-16T05:45:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "active_gid": 15429, 2024-04-16T05:45:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "active_name": "x", 2024-04-16T05:45:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addrs": { 2024-04-16T05:45:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-16T05:45:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3496546430 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3496546430 2024-04-16T05:45:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addr": "172.21.15.32:6801/3496546430", 2024-04-16T05:45:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "active_change": "2024-04-16T05:34:56.002899+0000", 2024-04-16T05:45:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "active_mgr_features": 4540701547738038271, 2024-04-16T05:45:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "available": true, 2024-04-16T05:45:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "standbys": [ 2024-04-16T05:45:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 24994, 2024-04-16T05:45:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "z", 2024-04-16T05:45:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540701547738038271, 2024-04-16T05:45:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2024-04-16T05:45:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2024-04-16T05:45:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T05:45:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2024-04-16T05:45:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2024-04-16T05:45:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2024-04-16T05:45:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2024-04-16T05:45:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2024-04-16T05:45:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2024-04-16T05:45:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2024-04-16T05:45:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2024-04-16T05:45:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2024-04-16T05:45:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2024-04-16T05:45:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2024-04-16T05:45:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T05:45:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2024-04-16T05:45:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T05:45:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T05:45:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2024-04-16T05:45:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2024-04-16T05:45:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-04-16T05:45:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-04-16T05:45:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T05:45:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2024-04-16T05:45:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2024-04-16T05:45:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2024-04-16T05:45:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2024-04-16T05:45:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T05:45:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T05:45:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T05:45:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T05:45:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2024-04-16T05:45:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T05:45:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2024-04-16T05:45:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2024-04-16T05:45:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2024-04-16T05:45:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T05:45:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2024-04-16T05:45:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2024-04-16T05:45:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-04-16T05:45:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-04-16T05:45:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2024-04-16T05:45:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2024-04-16T05:45:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2024-04-16T05:45:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "read", 2024-04-16T05:45:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap", 2024-04-16T05:45:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap-read" 2024-04-16T05:45:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2024-04-16T05:45:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2024-04-16T05:45:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2024-04-16T05:45:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T05:45:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2024-04-16T05:45:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2024-04-16T05:45:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T05:45:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T05:45:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2024-04-16T05:45:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2024-04-16T05:45:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2024-04-16T05:45:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2024-04-16T05:45:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-04-16T05:45:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2024-04-16T05:45:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T05:45:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2024-04-16T05:45:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2024-04-16T05:45:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T05:45:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T05:45:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2024-04-16T05:45:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2024-04-16T05:45:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2024-04-16T05:45:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2024-04-16T05:45:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2024-04-16T05:45:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T05:45:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T05:45:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2024-04-16T05:45:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2024-04-16T05:45:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T05:45:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T05:45:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2024-04-16T05:45:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2024-04-16T05:45:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2024-04-16T05:45:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2024-04-16T05:45:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2024-04-16T05:45:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T05:45:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2024-04-16T05:45:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2024-04-16T05:45:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T05:45:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2024-04-16T05:45:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2024-04-16T05:45:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2024-04-16T05:45:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T05:45:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2024-04-16T05:45:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2024-04-16T05:45:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2024-04-16T05:45:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T05:45:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2024-04-16T05:45:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2024-04-16T05:45:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2024-04-16T05:45:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2024-04-16T05:45:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T05:45:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2024-04-16T05:45:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T05:45:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2024-04-16T05:45:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2024-04-16T05:45:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T05:45:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2024-04-16T05:45:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2024-04-16T05:45:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_samba": { 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_samba", 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Samba/SMB container image", 2024-04-16T05:45:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T05:45:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2024-04-16T05:45:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2024-04-16T05:45:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2024-04-16T05:45:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T05:45:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2024-04-16T05:45:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2024-04-16T05:45:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T05:45:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2024-04-16T05:45:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2024-04-16T05:45:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2024-04-16T05:45:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2024-04-16T05:45:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2024-04-16T05:45:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "grafana_dashboards_path": { 2024-04-16T05:45:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "grafana_dashboards_path", 2024-04-16T05:45:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "hw_monitoring": { 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hw_monitoring", 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2024-04-16T05:45:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2024-04-16T05:45:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T05:45:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2024-04-16T05:45:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2024-04-16T05:45:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T05:45:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T05:45:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T05:45:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T05:45:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T05:45:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T05:45:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T05:45:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2024-04-16T05:45:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T05:45:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2024-04-16T05:45:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2024-04-16T05:45:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2024-04-16T05:45:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T05:45:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2024-04-16T05:45:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-04-16T05:45:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-04-16T05:45:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2024-04-16T05:45:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2024-04-16T05:45:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2024-04-16T05:45:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T05:45:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "oob_default_addr": { 2024-04-16T05:45:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "oob_default_addr", 2024-04-16T05:45:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "169.254.1.1", 2024-04-16T05:45:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T05:45:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T05:45:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T05:45:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2024-04-16T05:45:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2024-04-16T05:45:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2024-04-16T05:45:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2024-04-16T05:45:20.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.593 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.593 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.593 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.593 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.593 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.593 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T05:45:20.594 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.594 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.594 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2024-04-16T05:45:20.594 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2024-04-16T05:45:20.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.595 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.595 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.595 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.595 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.595 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.595 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.595 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T05:45:20.595 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.596 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T05:45:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2024-04-16T05:45:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2024-04-16T05:45:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T05:45:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2024-04-16T05:45:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2024-04-16T05:45:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2024-04-16T05:45:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2024-04-16T05:45:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T05:45:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T05:45:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2024-04-16T05:45:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2024-04-16T05:45:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T05:45:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2024-04-16T05:45:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2024-04-16T05:45:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2024-04-16T05:45:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2024-04-16T05:45:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2024-04-16T05:45:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2024-04-16T05:45:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T05:45:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2024-04-16T05:45:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2024-04-16T05:45:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T05:45:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2024-04-16T05:45:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2024-04-16T05:45:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T05:45:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T05:45:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T05:45:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T05:45:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T05:45:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T05:45:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2024-04-16T05:45:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T05:45:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T05:45:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T05:45:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T05:45:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T05:45:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T05:45:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T05:45:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T05:45:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T05:45:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T05:45:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T05:45:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T05:45:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T05:45:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T05:45:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T05:45:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T05:45:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T05:45:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2024-04-16T05:45:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2024-04-16T05:45:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T05:45:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T05:45:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T05:45:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T05:45:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T05:45:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T05:45:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T05:45:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T05:45:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T05:45:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "[]", 2024-04-16T05:45:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "{}", 2024-04-16T05:45:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T05:45:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T05:45:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T05:45:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T05:45:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T05:45:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T05:45:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T05:45:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T05:45:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T05:45:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T05:45:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T05:45:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T05:45:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T05:45:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T05:45:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T05:45:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T05:45:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T05:45:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T05:45:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T05:45:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T05:45:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T05:45:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T05:45:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T05:45:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T05:45:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T05:45:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T05:45:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T05:45:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2024-04-16T05:45:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T05:45:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T05:45:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2024-04-16T05:45:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T05:45:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T05:45:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T05:45:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T05:45:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T05:45:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T05:45:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T05:45:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T05:45:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T05:45:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2024-04-16T05:45:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2024-04-16T05:45:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2024-04-16T05:45:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-04-16T05:45:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-04-16T05:45:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2024-04-16T05:45:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2024-04-16T05:45:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T05:45:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-04-16T05:45:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2024-04-16T05:45:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-04-16T05:45:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-04-16T05:45:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2024-04-16T05:45:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-04-16T05:45:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-04-16T05:45:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-04-16T05:45:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-04-16T05:45:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-04-16T05:45:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-04-16T05:45:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2024-04-16T05:45:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2024-04-16T05:45:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2024-04-16T05:45:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2024-04-16T05:45:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2024-04-16T05:45:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2024-04-16T05:45:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2024-04-16T05:45:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2024-04-16T05:45:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T05:45:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2024-04-16T05:45:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2024-04-16T05:45:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2024-04-16T05:45:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T05:45:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2024-04-16T05:45:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2024-04-16T05:45:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2024-04-16T05:45:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2024-04-16T05:45:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T05:45:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2024-04-16T05:45:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2024-04-16T05:45:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T05:45:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2024-04-16T05:45:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T05:45:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2024-04-16T05:45:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-04-16T05:45:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2024-04-16T05:45:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2024-04-16T05:45:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2024-04-16T05:45:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2024-04-16T05:45:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T05:45:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2024-04-16T05:45:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2024-04-16T05:45:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2024-04-16T05:45:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T05:45:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2024-04-16T05:45:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2024-04-16T05:45:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2024-04-16T05:45:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2024-04-16T05:45:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2024-04-16T05:45:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T05:45:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2024-04-16T05:45:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2024-04-16T05:45:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2024-04-16T05:45:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-04-16T05:45:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-04-16T05:45:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2024-04-16T05:45:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2024-04-16T05:45:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2024-04-16T05:45:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T05:45:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2024-04-16T05:45:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2024-04-16T05:45:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2024-04-16T05:45:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2024-04-16T05:45:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2024-04-16T05:45:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2024-04-16T05:45:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2024-04-16T05:45:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2024-04-16T05:45:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2024-04-16T05:45:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2024-04-16T05:45:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2024-04-16T05:45:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2024-04-16T05:45:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2024-04-16T05:45:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2024-04-16T05:45:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2024-04-16T05:45:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T05:45:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2024-04-16T05:45:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2024-04-16T05:45:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2024-04-16T05:45:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2024-04-16T05:45:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2024-04-16T05:45:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2024-04-16T05:45:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2024-04-16T05:45:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2024-04-16T05:45:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2024-04-16T05:45:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2024-04-16T05:45:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T05:45:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T05:45:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2024-04-16T05:45:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2024-04-16T05:45:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2024-04-16T05:45:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2024-04-16T05:45:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.886 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.886 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.886 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.886 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.886 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.886 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2024-04-16T05:45:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-04-16T05:45:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2024-04-16T05:45:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2024-04-16T05:45:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2024-04-16T05:45:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2024-04-16T05:45:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2024-04-16T05:45:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2024-04-16T05:45:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2024-04-16T05:45:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2024-04-16T05:45:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-04-16T05:45:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2024-04-16T05:45:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2024-04-16T05:45:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T05:45:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2024-04-16T05:45:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2024-04-16T05:45:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T05:45:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-04-16T05:45:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2024-04-16T05:45:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2024-04-16T05:45:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2024-04-16T05:45:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2024-04-16T05:45:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2024-04-16T05:45:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2024-04-16T05:45:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2024-04-16T05:45:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2024-04-16T05:45:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2024-04-16T05:45:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T05:45:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T05:45:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2024-04-16T05:45:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2024-04-16T05:45:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2024-04-16T05:45:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T05:45:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2024-04-16T05:45:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2024-04-16T05:45:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-04-16T05:45:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-04-16T05:45:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2024-04-16T05:45:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2024-04-16T05:45:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2024-04-16T05:45:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-04-16T05:45:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-04-16T05:45:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-04-16T05:45:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-04-16T05:45:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2024-04-16T05:45:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2024-04-16T05:45:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2024-04-16T05:45:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2024-04-16T05:45:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T05:45:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2024-04-16T05:45:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2024-04-16T05:45:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2024-04-16T05:45:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2024-04-16T05:45:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2024-04-16T05:45:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-04-16T05:45:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-04-16T05:45:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2024-04-16T05:45:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2024-04-16T05:45:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2024-04-16T05:45:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2024-04-16T05:45:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2024-04-16T05:45:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T05:45:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2024-04-16T05:45:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2024-04-16T05:45:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2024-04-16T05:45:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2024-04-16T05:45:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2024-04-16T05:45:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2024-04-16T05:45:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2024-04-16T05:45:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2024-04-16T05:45:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2024-04-16T05:45:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2024-04-16T05:45:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2024-04-16T05:45:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2024-04-16T05:45:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2024-04-16T05:45:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2024-04-16T05:45:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2024-04-16T05:45:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2024-04-16T05:45:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2024-04-16T05:45:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2024-04-16T05:45:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2024-04-16T05:45:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2024-04-16T05:45:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2024-04-16T05:45:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2024-04-16T05:45:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-04-16T05:45:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2024-04-16T05:45:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2024-04-16T05:45:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2024-04-16T05:45:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2024-04-16T05:45:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2024-04-16T05:45:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2024-04-16T05:45:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2024-04-16T05:45:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2024-04-16T05:45:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T05:45:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2024-04-16T05:45:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2024-04-16T05:45:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2024-04-16T05:45:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2024-04-16T05:45:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2024-04-16T05:45:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2024-04-16T05:45:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2024-04-16T05:45:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2024-04-16T05:45:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T05:45:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2024-04-16T05:45:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2024-04-16T05:45:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T05:45:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2024-04-16T05:45:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2024-04-16T05:45:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T05:45:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2024-04-16T05:45:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2024-04-16T05:45:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2024-04-16T05:45:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2024-04-16T05:45:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2024-04-16T05:45:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2024-04-16T05:45:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-04-16T05:45:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-04-16T05:45:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2024-04-16T05:45:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2024-04-16T05:45:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2024-04-16T05:45:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2024-04-16T05:45:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2024-04-16T05:45:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2024-04-16T05:45:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2024-04-16T05:45:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2024-04-16T05:45:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2024-04-16T05:45:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2024-04-16T05:45:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2024-04-16T05:45:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2024-04-16T05:45:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T05:45:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2024-04-16T05:45:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2024-04-16T05:45:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2024-04-16T05:45:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2024-04-16T05:45:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T05:45:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2024-04-16T05:45:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2024-04-16T05:45:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T05:45:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_no_wait": { 2024-04-16T05:45:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T05:45:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T05:45:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2024-04-16T05:45:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2024-04-16T05:45:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2024-04-16T05:45:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2024-04-16T05:45:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2024-04-16T05:45:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2024-04-16T05:45:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2024-04-16T05:45:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T05:45:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 25043, 2024-04-16T05:45:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "y", 2024-04-16T05:45:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540701547738038271, 2024-04-16T05:45:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2024-04-16T05:45:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2024-04-16T05:45:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T05:45:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2024-04-16T05:45:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2024-04-16T05:45:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2024-04-16T05:45:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2024-04-16T05:45:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2024-04-16T05:45:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2024-04-16T05:45:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2024-04-16T05:45:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2024-04-16T05:45:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2024-04-16T05:45:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2024-04-16T05:45:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2024-04-16T05:45:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2024-04-16T05:45:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2024-04-16T05:45:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2024-04-16T05:45:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2024-04-16T05:45:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T05:45:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2024-04-16T05:45:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2024-04-16T05:45:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2024-04-16T05:45:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T05:45:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2024-04-16T05:45:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2024-04-16T05:45:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2024-04-16T05:45:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T05:45:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2024-04-16T05:45:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2024-04-16T05:45:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2024-04-16T05:45:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2024-04-16T05:45:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2024-04-16T05:45:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2024-04-16T05:45:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T05:45:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2024-04-16T05:45:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T05:45:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T05:45:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2024-04-16T05:45:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2024-04-16T05:45:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2024-04-16T05:45:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2024-04-16T05:45:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2024-04-16T05:45:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2024-04-16T05:45:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T05:45:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2024-04-16T05:45:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2024-04-16T05:45:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2024-04-16T05:45:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2024-04-16T05:45:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2024-04-16T05:45:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-04-16T05:45:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-04-16T05:45:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T05:45:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2024-04-16T05:45:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2024-04-16T05:45:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-04-16T05:45:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2024-04-16T05:45:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2024-04-16T05:45:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "read", 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap", 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap-read" 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T05:45:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2024-04-16T05:45:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T05:45:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T05:45:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2024-04-16T05:45:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T05:45:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2024-04-16T05:45:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2024-04-16T05:45:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2024-04-16T05:45:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T05:45:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2024-04-16T05:45:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2024-04-16T05:45:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T05:45:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T05:45:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2024-04-16T05:45:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2024-04-16T05:45:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T05:45:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2024-04-16T05:45:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T05:45:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2024-04-16T05:45:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2024-04-16T05:45:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T05:45:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T05:45:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2024-04-16T05:45:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2024-04-16T05:45:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T05:45:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2024-04-16T05:45:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2024-04-16T05:45:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T05:45:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2024-04-16T05:45:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2024-04-16T05:45:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2024-04-16T05:45:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2024-04-16T05:45:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T05:45:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2024-04-16T05:45:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2024-04-16T05:45:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T05:45:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T05:45:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2024-04-16T05:45:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2024-04-16T05:45:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2024-04-16T05:45:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T05:45:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2024-04-16T05:45:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T05:45:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2024-04-16T05:45:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2024-04-16T05:45:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2024-04-16T05:45:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T05:45:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2024-04-16T05:45:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T05:45:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2024-04-16T05:45:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2024-04-16T05:45:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T05:45:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2024-04-16T05:45:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2024-04-16T05:45:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2024-04-16T05:45:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T05:45:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2024-04-16T05:45:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2024-04-16T05:45:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_samba": { 2024-04-16T05:45:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_samba", 2024-04-16T05:45:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T05:45:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Samba/SMB container image", 2024-04-16T05:45:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2024-04-16T05:45:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2024-04-16T05:45:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T05:45:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T05:45:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2024-04-16T05:45:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2024-04-16T05:45:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T05:45:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2024-04-16T05:45:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T05:45:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2024-04-16T05:45:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T05:45:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2024-04-16T05:45:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2024-04-16T05:45:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2024-04-16T05:45:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T05:45:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2024-04-16T05:45:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "grafana_dashboards_path": { 2024-04-16T05:45:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "grafana_dashboards_path", 2024-04-16T05:45:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T05:45:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2024-04-16T05:45:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2024-04-16T05:45:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "hw_monitoring": { 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hw_monitoring", 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2024-04-16T05:45:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2024-04-16T05:45:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2024-04-16T05:45:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2024-04-16T05:45:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T05:45:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T05:45:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T05:45:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T05:45:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T05:45:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T05:45:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2024-04-16T05:45:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T05:45:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2024-04-16T05:45:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2024-04-16T05:45:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2024-04-16T05:45:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2024-04-16T05:45:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T05:45:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2024-04-16T05:45:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2024-04-16T05:45:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2024-04-16T05:45:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-04-16T05:45:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-04-16T05:45:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T05:45:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "oob_default_addr": { 2024-04-16T05:45:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "oob_default_addr", 2024-04-16T05:45:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "169.254.1.1", 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2024-04-16T05:45:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2024-04-16T05:45:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T05:45:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2024-04-16T05:45:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2024-04-16T05:45:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T05:45:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2024-04-16T05:45:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2024-04-16T05:45:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T05:45:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2024-04-16T05:45:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2024-04-16T05:45:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T05:45:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2024-04-16T05:45:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2024-04-16T05:45:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2024-04-16T05:45:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T05:45:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2024-04-16T05:45:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2024-04-16T05:45:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T05:45:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2024-04-16T05:45:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2024-04-16T05:45:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T05:45:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T05:45:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2024-04-16T05:45:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2024-04-16T05:45:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2024-04-16T05:45:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T05:45:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2024-04-16T05:45:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2024-04-16T05:45:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2024-04-16T05:45:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2024-04-16T05:45:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T05:45:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2024-04-16T05:45:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2024-04-16T05:45:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2024-04-16T05:45:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T05:45:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2024-04-16T05:45:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T05:45:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T05:45:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T05:45:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T05:45:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2024-04-16T05:45:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T05:45:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T05:45:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T05:45:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T05:45:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T05:45:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T05:45:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T05:45:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T05:45:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T05:45:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T05:45:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T05:45:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T05:45:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T05:45:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T05:45:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T05:45:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T05:45:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T05:45:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T05:45:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T05:45:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2024-04-16T05:45:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2024-04-16T05:45:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T05:45:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T05:45:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T05:45:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T05:45:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T05:45:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T05:45:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T05:45:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T05:45:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T05:45:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T05:45:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T05:45:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "[]", 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T05:45:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T05:45:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "{}", 2024-04-16T05:45:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T05:45:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T05:45:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T05:45:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T05:45:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T05:45:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T05:45:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T05:45:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T05:45:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T05:45:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T05:45:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T05:45:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T05:45:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T05:45:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T05:45:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T05:45:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T05:45:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T05:45:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T05:45:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T05:45:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T05:45:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T05:45:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2024-04-16T05:45:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T05:45:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T05:45:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T05:45:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T05:45:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T05:45:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T05:45:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T05:45:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T05:45:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T05:45:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T05:45:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T05:45:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T05:45:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T05:45:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T05:45:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T05:45:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2024-04-16T05:45:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2024-04-16T05:45:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2024-04-16T05:45:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2024-04-16T05:45:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2024-04-16T05:45:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2024-04-16T05:45:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2024-04-16T05:45:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2024-04-16T05:45:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2024-04-16T05:45:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-04-16T05:45:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2024-04-16T05:45:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-04-16T05:45:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-04-16T05:45:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2024-04-16T05:45:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2024-04-16T05:45:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2024-04-16T05:45:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-04-16T05:45:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-04-16T05:45:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-04-16T05:45:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-04-16T05:45:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-04-16T05:45:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-04-16T05:45:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2024-04-16T05:45:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2024-04-16T05:45:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2024-04-16T05:45:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2024-04-16T05:45:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2024-04-16T05:45:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2024-04-16T05:45:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T05:45:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2024-04-16T05:45:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2024-04-16T05:45:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2024-04-16T05:45:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T05:45:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2024-04-16T05:45:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2024-04-16T05:45:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2024-04-16T05:45:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2024-04-16T05:45:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2024-04-16T05:45:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2024-04-16T05:45:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-04-16T05:45:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T05:45:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2024-04-16T05:45:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2024-04-16T05:45:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T05:45:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T05:45:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2024-04-16T05:45:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2024-04-16T05:45:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2024-04-16T05:45:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2024-04-16T05:45:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2024-04-16T05:45:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-04-16T05:45:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2024-04-16T05:45:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2024-04-16T05:45:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2024-04-16T05:45:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2024-04-16T05:45:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2024-04-16T05:45:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2024-04-16T05:45:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2024-04-16T05:45:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2024-04-16T05:45:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2024-04-16T05:45:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T05:45:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2024-04-16T05:45:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2024-04-16T05:45:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2024-04-16T05:45:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T05:45:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2024-04-16T05:45:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2024-04-16T05:45:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2024-04-16T05:45:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2024-04-16T05:45:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2024-04-16T05:45:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2024-04-16T05:45:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2024-04-16T05:45:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2024-04-16T05:45:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2024-04-16T05:45:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T05:45:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2024-04-16T05:45:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2024-04-16T05:45:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2024-04-16T05:45:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2024-04-16T05:45:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2024-04-16T05:45:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2024-04-16T05:45:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2024-04-16T05:45:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2024-04-16T05:45:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2024-04-16T05:45:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2024-04-16T05:45:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2024-04-16T05:45:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2024-04-16T05:45:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T05:45:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2024-04-16T05:45:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2024-04-16T05:45:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2024-04-16T05:45:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2024-04-16T05:45:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2024-04-16T05:45:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T05:45:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2024-04-16T05:45:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2024-04-16T05:45:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2024-04-16T05:45:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2024-04-16T05:45:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2024-04-16T05:45:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2024-04-16T05:45:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T05:45:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2024-04-16T05:45:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2024-04-16T05:45:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2024-04-16T05:45:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2024-04-16T05:45:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2024-04-16T05:45:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2024-04-16T05:45:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T05:45:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T05:45:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2024-04-16T05:45:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2024-04-16T05:45:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2024-04-16T05:45:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2024-04-16T05:45:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T05:45:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2024-04-16T05:45:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2024-04-16T05:45:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2024-04-16T05:45:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2024-04-16T05:45:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2024-04-16T05:45:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2024-04-16T05:45:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-04-16T05:45:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2024-04-16T05:45:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2024-04-16T05:45:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T05:45:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2024-04-16T05:45:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2024-04-16T05:45:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T05:45:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-04-16T05:45:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-04-16T05:45:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2024-04-16T05:45:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2024-04-16T05:45:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2024-04-16T05:45:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2024-04-16T05:45:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T05:45:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T05:45:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2024-04-16T05:45:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2024-04-16T05:45:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T05:45:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T05:45:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2024-04-16T05:45:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2024-04-16T05:45:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2024-04-16T05:45:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T05:45:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2024-04-16T05:45:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T05:45:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2024-04-16T05:45:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2024-04-16T05:45:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2024-04-16T05:45:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-04-16T05:45:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-04-16T05:45:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2024-04-16T05:45:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-04-16T05:45:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-04-16T05:45:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-04-16T05:45:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-04-16T05:45:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2024-04-16T05:45:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2024-04-16T05:45:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2024-04-16T05:45:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2024-04-16T05:45:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T05:45:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2024-04-16T05:45:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2024-04-16T05:45:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2024-04-16T05:45:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2024-04-16T05:45:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2024-04-16T05:45:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-04-16T05:45:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2024-04-16T05:45:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2024-04-16T05:45:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2024-04-16T05:45:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2024-04-16T05:45:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2024-04-16T05:45:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2024-04-16T05:45:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2024-04-16T05:45:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2024-04-16T05:45:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2024-04-16T05:45:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2024-04-16T05:45:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2024-04-16T05:45:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2024-04-16T05:45:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2024-04-16T05:45:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2024-04-16T05:45:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2024-04-16T05:45:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2024-04-16T05:45:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2024-04-16T05:45:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2024-04-16T05:45:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2024-04-16T05:45:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2024-04-16T05:45:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2024-04-16T05:45:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2024-04-16T05:45:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2024-04-16T05:45:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2024-04-16T05:45:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2024-04-16T05:45:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-04-16T05:45:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2024-04-16T05:45:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2024-04-16T05:45:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T05:45:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2024-04-16T05:45:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2024-04-16T05:45:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2024-04-16T05:45:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2024-04-16T05:45:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2024-04-16T05:45:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2024-04-16T05:45:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2024-04-16T05:45:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2024-04-16T05:45:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2024-04-16T05:45:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T05:45:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T05:45:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2024-04-16T05:45:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2024-04-16T05:45:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T05:45:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2024-04-16T05:45:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2024-04-16T05:45:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2024-04-16T05:45:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2024-04-16T05:45:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-04-16T05:45:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-04-16T05:45:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2024-04-16T05:45:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2024-04-16T05:45:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2024-04-16T05:45:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2024-04-16T05:45:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2024-04-16T05:45:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2024-04-16T05:45:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2024-04-16T05:45:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2024-04-16T05:45:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2024-04-16T05:45:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2024-04-16T05:45:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2024-04-16T05:45:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2024-04-16T05:45:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2024-04-16T05:45:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2024-04-16T05:45:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2024-04-16T05:45:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2024-04-16T05:45:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2024-04-16T05:45:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2024-04-16T05:45:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T05:45:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2024-04-16T05:45:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2024-04-16T05:45:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2024-04-16T05:45:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2024-04-16T05:45:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T05:45:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_no_wait": { 2024-04-16T05:45:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T05:45:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2024-04-16T05:45:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2024-04-16T05:45:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2024-04-16T05:45:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2024-04-16T05:45:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2024-04-16T05:45:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2024-04-16T05:45:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2024-04-16T05:45:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T05:45:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "modules": [ 2024-04-16T05:45:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2024-04-16T05:45:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "dashboard", 2024-04-16T05:45:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "iostat", 2024-04-16T05:45:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "nfs", 2024-04-16T05:45:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "restful", 2024-04-16T05:45:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_schedule" 2024-04-16T05:45:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2024-04-16T05:45:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2024-04-16T05:45:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T05:45:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2024-04-16T05:45:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2024-04-16T05:45:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2024-04-16T05:45:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2024-04-16T05:45:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2024-04-16T05:45:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2024-04-16T05:45:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2024-04-16T05:45:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2024-04-16T05:45:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2024-04-16T05:45:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2024-04-16T05:45:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2024-04-16T05:45:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2024-04-16T05:45:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2024-04-16T05:45:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2024-04-16T05:45:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2024-04-16T05:45:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2024-04-16T05:45:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2024-04-16T05:45:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2024-04-16T05:45:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2024-04-16T05:45:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2024-04-16T05:45:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-04-16T05:45:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-04-16T05:45:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T05:45:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2024-04-16T05:45:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2024-04-16T05:45:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2024-04-16T05:45:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2024-04-16T05:45:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T05:45:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2024-04-16T05:45:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2024-04-16T05:45:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2024-04-16T05:45:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2024-04-16T05:45:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2024-04-16T05:45:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T05:45:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2024-04-16T05:45:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2024-04-16T05:45:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2024-04-16T05:45:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T05:45:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T05:45:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2024-04-16T05:45:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2024-04-16T05:45:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-04-16T05:45:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-04-16T05:45:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T05:45:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T05:45:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2024-04-16T05:45:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2024-04-16T05:45:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-04-16T05:45:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2024-04-16T05:45:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "read", 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap", 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap-read" 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2024-04-16T05:45:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2024-04-16T05:45:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2024-04-16T05:45:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2024-04-16T05:45:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T05:45:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T05:45:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2024-04-16T05:45:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-04-16T05:45:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T05:45:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T05:45:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2024-04-16T05:45:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2024-04-16T05:45:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2024-04-16T05:45:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2024-04-16T05:45:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T05:45:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T05:45:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2024-04-16T05:45:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2024-04-16T05:45:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T05:45:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2024-04-16T05:45:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T05:45:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2024-04-16T05:45:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T05:45:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2024-04-16T05:45:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2024-04-16T05:45:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2024-04-16T05:45:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T05:45:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2024-04-16T05:45:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2024-04-16T05:45:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T05:45:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2024-04-16T05:45:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2024-04-16T05:45:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T05:45:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2024-04-16T05:45:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2024-04-16T05:45:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T05:45:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2024-04-16T05:45:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2024-04-16T05:45:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T05:45:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2024-04-16T05:45:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2024-04-16T05:45:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T05:45:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2024-04-16T05:45:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T05:45:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2024-04-16T05:45:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2024-04-16T05:45:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2024-04-16T05:45:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T05:45:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2024-04-16T05:45:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2024-04-16T05:45:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T05:45:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2024-04-16T05:45:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2024-04-16T05:45:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2024-04-16T05:45:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2024-04-16T05:45:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2024-04-16T05:45:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T05:45:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2024-04-16T05:45:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2024-04-16T05:45:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2024-04-16T05:45:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T05:45:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2024-04-16T05:45:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2024-04-16T05:45:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2024-04-16T05:45:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2024-04-16T05:45:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2024-04-16T05:45:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T05:45:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2024-04-16T05:45:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T05:45:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2024-04-16T05:45:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2024-04-16T05:45:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T05:45:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2024-04-16T05:45:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2024-04-16T05:45:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2024-04-16T05:45:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2024-04-16T05:45:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2024-04-16T05:45:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T05:45:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2024-04-16T05:45:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_samba": { 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_samba", 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Samba/SMB container image", 2024-04-16T05:45:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2024-04-16T05:45:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2024-04-16T05:45:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T05:45:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T05:45:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T05:45:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2024-04-16T05:45:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T05:45:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2024-04-16T05:45:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2024-04-16T05:45:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2024-04-16T05:45:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2024-04-16T05:45:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2024-04-16T05:45:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2024-04-16T05:45:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "grafana_dashboards_path": { 2024-04-16T05:45:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "grafana_dashboards_path", 2024-04-16T05:45:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T05:45:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T05:45:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2024-04-16T05:45:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2024-04-16T05:45:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2024-04-16T05:45:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "hw_monitoring": { 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hw_monitoring", 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2024-04-16T05:45:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2024-04-16T05:45:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2024-04-16T05:45:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2024-04-16T05:45:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T05:45:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T05:45:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T05:45:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T05:45:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T05:45:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2024-04-16T05:45:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T05:45:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2024-04-16T05:45:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2024-04-16T05:45:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T05:45:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2024-04-16T05:45:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2024-04-16T05:45:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-04-16T05:45:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-04-16T05:45:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2024-04-16T05:45:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "oob_default_addr": { 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "oob_default_addr", 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "169.254.1.1", 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2024-04-16T05:45:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2024-04-16T05:45:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2024-04-16T05:45:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T05:45:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2024-04-16T05:45:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2024-04-16T05:45:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2024-04-16T05:45:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T05:45:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2024-04-16T05:45:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2024-04-16T05:45:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T05:45:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2024-04-16T05:45:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2024-04-16T05:45:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2024-04-16T05:45:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2024-04-16T05:45:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2024-04-16T05:45:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2024-04-16T05:45:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T05:45:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2024-04-16T05:45:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2024-04-16T05:45:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T05:45:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2024-04-16T05:45:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2024-04-16T05:45:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T05:45:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T05:45:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2024-04-16T05:45:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2024-04-16T05:45:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T05:45:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T05:45:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2024-04-16T05:45:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2024-04-16T05:45:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2024-04-16T05:45:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T05:45:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2024-04-16T05:45:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T05:45:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T05:45:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2024-04-16T05:45:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T05:45:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T05:45:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T05:45:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T05:45:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T05:45:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T05:45:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T05:45:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T05:45:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T05:45:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T05:45:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T05:45:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T05:45:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T05:45:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T05:45:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T05:45:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T05:45:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T05:45:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T05:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T05:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T05:45:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T05:45:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T05:45:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T05:45:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "[]", 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "{}", 2024-04-16T05:45:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T05:45:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T05:45:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T05:45:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T05:45:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T05:45:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T05:45:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T05:45:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T05:45:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T05:45:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T05:45:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T05:45:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T05:45:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T05:45:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T05:45:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T05:45:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T05:45:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2024-04-16T05:45:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T05:45:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T05:45:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2024-04-16T05:45:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T05:45:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T05:45:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T05:45:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T05:45:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T05:45:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T05:45:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T05:45:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T05:45:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2024-04-16T05:45:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2024-04-16T05:45:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2024-04-16T05:45:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2024-04-16T05:45:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2024-04-16T05:45:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2024-04-16T05:45:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2024-04-16T05:45:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2024-04-16T05:45:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2024-04-16T05:45:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2024-04-16T05:45:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2024-04-16T05:45:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-04-16T05:45:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-04-16T05:45:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2024-04-16T05:45:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2024-04-16T05:45:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2024-04-16T05:45:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-04-16T05:45:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-04-16T05:45:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2024-04-16T05:45:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T05:45:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2024-04-16T05:45:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2024-04-16T05:45:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2024-04-16T05:45:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2024-04-16T05:45:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2024-04-16T05:45:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2024-04-16T05:45:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2024-04-16T05:45:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2024-04-16T05:45:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-04-16T05:45:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2024-04-16T05:45:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2024-04-16T05:45:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2024-04-16T05:45:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2024-04-16T05:45:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2024-04-16T05:45:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2024-04-16T05:45:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2024-04-16T05:45:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2024-04-16T05:45:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2024-04-16T05:45:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-04-16T05:45:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2024-04-16T05:45:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2024-04-16T05:45:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2024-04-16T05:45:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2024-04-16T05:45:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2024-04-16T05:45:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2024-04-16T05:45:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T05:45:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2024-04-16T05:45:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2024-04-16T05:45:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2024-04-16T05:45:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T05:45:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2024-04-16T05:45:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2024-04-16T05:45:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2024-04-16T05:45:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T05:45:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2024-04-16T05:45:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2024-04-16T05:45:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2024-04-16T05:45:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2024-04-16T05:45:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2024-04-16T05:45:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2024-04-16T05:45:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2024-04-16T05:45:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2024-04-16T05:45:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T05:45:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2024-04-16T05:45:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2024-04-16T05:45:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2024-04-16T05:45:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2024-04-16T05:45:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2024-04-16T05:45:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2024-04-16T05:45:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2024-04-16T05:45:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2024-04-16T05:45:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T05:45:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2024-04-16T05:45:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2024-04-16T05:45:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2024-04-16T05:45:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2024-04-16T05:45:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2024-04-16T05:45:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2024-04-16T05:45:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T05:45:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2024-04-16T05:45:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2024-04-16T05:45:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2024-04-16T05:45:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2024-04-16T05:45:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2024-04-16T05:45:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T05:45:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2024-04-16T05:45:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2024-04-16T05:45:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2024-04-16T05:45:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2024-04-16T05:45:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2024-04-16T05:45:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2024-04-16T05:45:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2024-04-16T05:45:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2024-04-16T05:45:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2024-04-16T05:45:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-04-16T05:45:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2024-04-16T05:45:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2024-04-16T05:45:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T05:45:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2024-04-16T05:45:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2024-04-16T05:45:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2024-04-16T05:45:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T05:45:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-04-16T05:45:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-04-16T05:45:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-04-16T05:45:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-04-16T05:45:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-04-16T05:45:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2024-04-16T05:45:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2024-04-16T05:45:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T05:45:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T05:45:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2024-04-16T05:45:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2024-04-16T05:45:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T05:45:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T05:45:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2024-04-16T05:45:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2024-04-16T05:45:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2024-04-16T05:45:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2024-04-16T05:45:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-04-16T05:45:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T05:45:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2024-04-16T05:45:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T05:45:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2024-04-16T05:45:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2024-04-16T05:45:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2024-04-16T05:45:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2024-04-16T05:45:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2024-04-16T05:45:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-04-16T05:45:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-04-16T05:45:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-04-16T05:45:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-04-16T05:45:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-04-16T05:45:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2024-04-16T05:45:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2024-04-16T05:45:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2024-04-16T05:45:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T05:45:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2024-04-16T05:45:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2024-04-16T05:45:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2024-04-16T05:45:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-04-16T05:45:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-04-16T05:45:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2024-04-16T05:45:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2024-04-16T05:45:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2024-04-16T05:45:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2024-04-16T05:45:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T05:45:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2024-04-16T05:45:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2024-04-16T05:45:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2024-04-16T05:45:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2024-04-16T05:45:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2024-04-16T05:45:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2024-04-16T05:45:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2024-04-16T05:45:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2024-04-16T05:45:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2024-04-16T05:45:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-04-16T05:45:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2024-04-16T05:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2024-04-16T05:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2024-04-16T05:45:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2024-04-16T05:45:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2024-04-16T05:45:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2024-04-16T05:45:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-04-16T05:45:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2024-04-16T05:45:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2024-04-16T05:45:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2024-04-16T05:45:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2024-04-16T05:45:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2024-04-16T05:45:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2024-04-16T05:45:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2024-04-16T05:45:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2024-04-16T05:45:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2024-04-16T05:45:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2024-04-16T05:45:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2024-04-16T05:45:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2024-04-16T05:45:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T05:45:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2024-04-16T05:45:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2024-04-16T05:45:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T05:45:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2024-04-16T05:45:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2024-04-16T05:45:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T05:45:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2024-04-16T05:45:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2024-04-16T05:45:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T05:45:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2024-04-16T05:45:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2024-04-16T05:45:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T05:45:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2024-04-16T05:45:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2024-04-16T05:45:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2024-04-16T05:45:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2024-04-16T05:45:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2024-04-16T05:45:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2024-04-16T05:45:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T05:45:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-04-16T05:45:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2024-04-16T05:45:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2024-04-16T05:45:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2024-04-16T05:45:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2024-04-16T05:45:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2024-04-16T05:45:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2024-04-16T05:45:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2024-04-16T05:45:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2024-04-16T05:45:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T05:45:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2024-04-16T05:45:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2024-04-16T05:45:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T05:45:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2024-04-16T05:45:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2024-04-16T05:45:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2024-04-16T05:45:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2024-04-16T05:45:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2024-04-16T05:45:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-04-16T05:45:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T05:45:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_no_wait": { 2024-04-16T05:45:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T05:45:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-04-16T05:45:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T05:45:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2024-04-16T05:45:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-04-16T05:45:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-04-16T05:45:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-04-16T05:45:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2024-04-16T05:45:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2024-04-16T05:45:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-04-16T05:45:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2024-04-16T05:45:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2024-04-16T05:45:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2024-04-16T05:45:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-04-16T05:45:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-04-16T05:45:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-04-16T05:45:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-04-16T05:45:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-04-16T05:45:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-04-16T05:45:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-04-16T05:45:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-04-16T05:45:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-04-16T05:45:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-04-16T05:45:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-04-16T05:45:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-04-16T05:45:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-04-16T05:45:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-04-16T05:45:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2024-04-16T05:45:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2024-04-16T05:45:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2024-04-16T05:45:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-04-16T05:45:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2024-04-16T05:45:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2024-04-16T05:45:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2024-04-16T05:45:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-04-16T05:45:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-04-16T05:45:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-04-16T05:45:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T05:45:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-04-16T05:45:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-04-16T05:45:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-04-16T05:45:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-04-16T05:45:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-04-16T05:45:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-04-16T05:45:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-04-16T05:45:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "services": { 2024-04-16T05:45:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "dashboard": "https://172.21.15.32:8443/" 2024-04-16T05:45:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "always_on_modules": { 2024-04-16T05:45:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "octopus": [ 2024-04-16T05:45:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-04-16T05:45:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-04-16T05:45:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-04-16T05:45:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-04-16T05:45:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-04-16T05:45:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-04-16T05:45:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-04-16T05:45:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-04-16T05:45:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-04-16T05:45:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pacific": [ 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-04-16T05:45:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "quincy": [ 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-04-16T05:45:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-04-16T05:45:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-04-16T05:45:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-04-16T05:45:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-04-16T05:45:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-04-16T05:45:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-04-16T05:45:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-04-16T05:45:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "reef": [ 2024-04-16T05:45:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-04-16T05:45:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-04-16T05:45:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-04-16T05:45:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-04-16T05:45:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-04-16T05:45:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-04-16T05:45:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-04-16T05:45:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "squid": [ 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-04-16T05:45:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-04-16T05:45:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-04-16T05:45:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-04-16T05:45:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-04-16T05:45:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-04-16T05:45:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-04-16T05:45:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "last_failure_osd_epoch": 92, 2024-04-16T05:45:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "active_clients": [ 2024-04-16T05:45:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2024-04-16T05:45:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-16T05:45:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-16T05:45:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-04-16T05:45:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3324344289 2024-04-16T05:45:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "libcephsqlite", 2024-04-16T05:45:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-16T05:45:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-16T05:45:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-04-16T05:45:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1852501144 2024-04-16T05:45:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2024-04-16T05:45:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1555160763 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-04-16T05:45:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-16T05:45:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-16T05:45:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-04-16T05:45:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3069576058 2024-04-16T05:45:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-04-16T05:45:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-16T05:45:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-16T05:45:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-04-16T05:45:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2108291074 2024-04-16T05:45:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-04-16T05:45:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3752683166 2024-04-16T05:45:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-16T05:45:22.653 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:45:22.655 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:20 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/330934684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:22.655 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:20 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4088426013' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T05:45:22.655 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:21 smithi082 ceph-mon[36195]: pgmap v323: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:45:22.655 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:20 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/330934684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:22.655 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:20 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4088426013' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T05:45:22.655 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:21 smithi134 ceph-mon[36153]: pgmap v323: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:45:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fba4c0a3d20 msgr2=0x7fba4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fba4c0a3d20 0x7fba4c0a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fba5002c700 tx=0x7fba50005c30 comp rx=0 tx=0).stop 2024-04-16T05:45:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101140 msgr2=0x7fba601a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101140 0x7fba601a0750 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fba5800c970 tx=0x7fba5800ce30 comp rx=0 tx=0).stop 2024-04-16T05:45:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 shutdown_connections 2024-04-16T05:45:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6019a900 0x7fba6019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60106210 0x7fba601a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fba4c0a3d20 0x7fba4c0a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 --2- 172.21.15.32:0/4088426013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba60101140 0x7fba601a0750 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 >> 172.21.15.32:0/4088426013 conn(0x7fba60077280 msgr2=0x7fba600ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 shutdown_connections 2024-04-16T05:45:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:20.439+0000 7fba66c74640 1 -- 172.21.15.32:0/4088426013 wait complete. 2024-04-16T05:45:22.665 DEBUG:tasks.check_counter:Getting stats from a 2024-04-16T05:45:22.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-16T05:45:22.666 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:20 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/330934684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:22.666 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:20 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4088426013' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T05:45:22.666 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:21 smithi032 ceph-mon[32351]: pgmap v323: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T05:45:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:45:24.102 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:23 smithi032 ceph-mon[32351]: pgmap v324: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:45:24.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:23 smithi134 ceph-mon[36153]: pgmap v324: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:45:24.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:23 smithi082 ceph-mon[36195]: pgmap v324: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T05:45:24.923 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:45:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 221, 2024-04-16T05:45:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 210, 2024-04-16T05:45:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 1589488, 2024-04-16T05:45:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 128704, 2024-04-16T05:45:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2024-04-16T05:45:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.044160699, 2024-04-16T05:45:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.014887734, 2024-04-16T05:45:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.032769142, 2024-04-16T05:45:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.004388851, 2024-04-16T05:45:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 209, 2024-04-16T05:45:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.011046799, 2024-04-16T05:45:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000052855 2024-04-16T05:45:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1589488, 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 128704 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 6, 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 162, 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 3312, 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 184784, 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.013665253, 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.009222726, 2024-04-16T05:45:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.002209669, 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 160, 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.010455774, 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000065348 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 184784 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2024-04-16T05:45:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.000344353, 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000286173, 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2024-04-16T05:45:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2024-04-16T05:45:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2024-04-16T05:45:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2024-04-16T05:45:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2024-04-16T05:45:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 149011, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 32, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2024-04-16T05:45:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2024-04-16T05:45:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2024-04-16T05:45:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-04-16T05:45:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-04-16T05:45:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-04-16T05:45:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 223, 2024-04-16T05:45:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 1558333, 2024-04-16T05:45:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 223, 2024-04-16T05:45:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-04-16T05:45:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-04-16T05:45:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 223, 2024-04-16T05:45:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 1558333, 2024-04-16T05:45:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-04-16T05:45:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:24.942 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:45:25.630 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-16T05:45:25.630 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-16T05:45:25.630 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-16T05:45:25.631 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-16T05:45:25.631 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-16T05:45:25.631 DEBUG:tasks.check_counter:Getting stats from d 2024-04-16T05:45:25.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-16T05:45:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:45:25.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:25 smithi032 ceph-mon[32351]: pgmap v325: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:45:26.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:25 smithi134 ceph-mon[36153]: pgmap v325: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:45:26.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:25 smithi082 ceph-mon[36195]: pgmap v325: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-16T05:45:27.604 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:45:27.604 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:27.604 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 226, 2024-04-16T05:45:27.604 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 215, 2024-04-16T05:45:27.604 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 1590848, 2024-04-16T05:45:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 131904, 2024-04-16T05:45:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2024-04-16T05:45:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.046797780, 2024-04-16T05:45:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.013996032, 2024-04-16T05:45:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.037254345, 2024-04-16T05:45:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.004288278, 2024-04-16T05:45:27.606 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:27.606 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 214, 2024-04-16T05:45:27.606 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.010260996, 2024-04-16T05:45:27.606 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000047948 2024-04-16T05:45:27.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.606 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:27.606 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:27.606 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:27.607 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:27.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.607 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1590848, 2024-04-16T05:45:27.607 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 131904 2024-04-16T05:45:27.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.607 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:27.607 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 2, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 165, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 320, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 185216, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.015299553, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.010868567, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.001967411, 2024-04-16T05:45:27.608 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:27.609 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:27.609 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 162, 2024-04-16T05:45:27.609 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.013863783, 2024-04-16T05:45:27.609 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000085578 2024-04-16T05:45:27.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.609 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:27.609 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:27.609 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:45:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 185216 2024-04-16T05:45:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 4, 2024-04-16T05:45:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 4, 2024-04-16T05:45:27.611 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 2992, 2024-04-16T05:45:27.611 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 448, 2024-04-16T05:45:27.611 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2024-04-16T05:45:27.611 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:27.611 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.000643670, 2024-04-16T05:45:27.611 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000039937, 2024-04-16T05:45:27.611 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000619350, 2024-04-16T05:45:27.611 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 3, 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000821315, 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000273771 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:27.612 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:27.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.613 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-16T05:45:27.613 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T05:45:27.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.613 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2024-04-16T05:45:27.613 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2024-04-16T05:45:27.613 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2024-04-16T05:45:27.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.614 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2024-04-16T05:45:27.614 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:27.614 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:27.614 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:27.614 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:27.614 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:27.614 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:27.615 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:27.615 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:27.615 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:27.615 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:27.615 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:27.615 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:27.615 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:27.615 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:27.616 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:27.616 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:27.616 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:27.616 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:27.616 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:27.616 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:27.616 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:27.616 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:27.617 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:27.617 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:27.617 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:27.617 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:27.617 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:27.617 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:27.617 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:27.617 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2024-04-16T05:45:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 149017, 2024-04-16T05:45:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 31, 2024-04-16T05:45:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2024-04-16T05:45:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2024-04-16T05:45:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:27.620 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2024-04-16T05:45:27.620 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2024-04-16T05:45:27.620 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:27.620 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:27.620 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:27.620 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2024-04-16T05:45:27.620 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2024-04-16T05:45:27.620 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2024-04-16T05:45:27.621 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:27.621 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2024-04-16T05:45:27.621 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:27.621 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2024-04-16T05:45:27.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:27.621 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:27.621 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-04-16T05:45:27.621 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-04-16T05:45:27.622 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-04-16T05:45:27.622 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 228, 2024-04-16T05:45:27.622 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 1558983, 2024-04-16T05:45:27.622 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:27.622 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 228, 2024-04-16T05:45:27.622 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-04-16T05:45:27.622 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-04-16T05:45:27.622 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 228, 2024-04-16T05:45:27.623 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 1558983, 2024-04-16T05:45:27.623 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-04-16T05:45:27.623 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:27.623 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:27.623 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:27.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:27.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:27.623 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:45:27.977 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:27 smithi032 ceph-mon[32351]: pgmap v326: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:28.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:27 smithi134 ceph-mon[36153]: pgmap v326: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:28.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:27 smithi082 ceph-mon[36195]: pgmap v326: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:28.429 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-16T05:45:28.429 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-16T05:45:28.430 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-16T05:45:28.430 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-16T05:45:28.430 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-16T05:45:28.430 DEBUG:tasks.check_counter:Getting stats from g 2024-04-16T05:45:28.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-16T05:45:28.686 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:45:30.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:29 smithi134 ceph-mon[36153]: pgmap v327: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:30.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:29 smithi082 ceph-mon[36195]: pgmap v327: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:30.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:29 smithi032 ceph-mon[32351]: pgmap v327: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:30.450 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:45:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 223, 2024-04-16T05:45:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 212, 2024-04-16T05:45:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 1590032, 2024-04-16T05:45:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 129984, 2024-04-16T05:45:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2024-04-16T05:45:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.045724476, 2024-04-16T05:45:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.014749372, 2024-04-16T05:45:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.034580662, 2024-04-16T05:45:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.004414072, 2024-04-16T05:45:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 211, 2024-04-16T05:45:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.010398593, 2024-04-16T05:45:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000049282 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1590032, 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 129984 2024-04-16T05:45:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.454 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:30.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 2, 2024-04-16T05:45:30.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 161, 2024-04-16T05:45:30.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 320, 2024-04-16T05:45:30.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 184512, 2024-04-16T05:45:30.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:30.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:30.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.013603064, 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.009571028, 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.001807045, 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 158, 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.012774850, 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000080853 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.455 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:30.456 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:30.456 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:30.456 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:30.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.456 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:45:30.456 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 184512 2024-04-16T05:45:30.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.456 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 4, 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 4, 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 2992, 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 448, 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.000628008, 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000037040, 2024-04-16T05:45:30.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000580268, 2024-04-16T05:45:30.458 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:30.458 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:30.458 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 3, 2024-04-16T05:45:30.458 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000746609, 2024-04-16T05:45:30.458 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000248869 2024-04-16T05:45:30.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.458 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:30.458 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2024-04-16T05:45:30.459 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2024-04-16T05:45:30.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.460 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2024-04-16T05:45:30.460 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:30.460 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:30.460 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:30.460 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:30.460 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:30.460 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:30.461 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:30.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:30.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:30.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:30.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:30.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:30.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:30.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:30.462 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:30.463 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:30.463 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:30.463 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:30.463 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:30.463 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:30.463 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:30.463 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:30.463 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 149017, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 31, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:30.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2024-04-16T05:45:30.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2024-04-16T05:45:30.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:30.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:30.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:30.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:30.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2024-04-16T05:45:30.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2024-04-16T05:45:30.466 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:30.466 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:30.466 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:30.466 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2024-04-16T05:45:30.466 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2024-04-16T05:45:30.466 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2024-04-16T05:45:30.466 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:30.466 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 225, 2024-04-16T05:45:30.467 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 1558593, 2024-04-16T05:45:30.468 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:30.468 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 225, 2024-04-16T05:45:30.468 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-04-16T05:45:30.468 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-04-16T05:45:30.468 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 225, 2024-04-16T05:45:30.468 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 1558593, 2024-04-16T05:45:30.468 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-04-16T05:45:30.468 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:30.469 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:30.469 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:30.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:30.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:30.469 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:45:31.173 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-16T05:45:31.174 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-16T05:45:31.174 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-16T05:45:31.174 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-16T05:45:31.174 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-16T05:45:31.174 DEBUG:tasks.check_counter:Getting stats from j 2024-04-16T05:45:31.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-16T05:45:31.431 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.a/config 2024-04-16T05:45:32.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:31 smithi134 ceph-mon[36153]: pgmap v328: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:32.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:31 smithi082 ceph-mon[36195]: pgmap v328: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:32.157 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:31 smithi032 ceph-mon[32351]: pgmap v328: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:33.003 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-16T05:45:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 229, 2024-04-16T05:45:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 218, 2024-04-16T05:45:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 1591664, 2024-04-16T05:45:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 133824, 2024-04-16T05:45:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2024-04-16T05:45:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.048498145, 2024-04-16T05:45:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.014529177, 2024-04-16T05:45:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.037820958, 2024-04-16T05:45:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.004543565, 2024-04-16T05:45:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 217, 2024-04-16T05:45:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.010818359, 2024-04-16T05:45:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000049854 2024-04-16T05:45:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1591664, 2024-04-16T05:45:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 133824 2024-04-16T05:45:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 6, 2024-04-16T05:45:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 168, 2024-04-16T05:45:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 3312, 2024-04-16T05:45:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 185840, 2024-04-16T05:45:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.016098395, 2024-04-16T05:45:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.010809466, 2024-04-16T05:45:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.002750745, 2024-04-16T05:45:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 166, 2024-04-16T05:45:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.012420745, 2024-04-16T05:45:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000074823 2024-04-16T05:45:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-16T05:45:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 185840 2024-04-16T05:45:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2024-04-16T05:45:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2024-04-16T05:45:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.000377306, 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000312336, 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2024-04-16T05:45:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2024-04-16T05:45:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2024-04-16T05:45:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2024-04-16T05:45:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2024-04-16T05:45:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 149017, 2024-04-16T05:45:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 31, 2024-04-16T05:45:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2024-04-16T05:45:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2024-04-16T05:45:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2024-04-16T05:45:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2024-04-16T05:45:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2024-04-16T05:45:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-16T05:45:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-04-16T05:45:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-04-16T05:45:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-04-16T05:45:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 231, 2024-04-16T05:45:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 1559373, 2024-04-16T05:45:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 231, 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 231, 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 1559373, 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-04-16T05:45:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-04-16T05:45:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-16T05:45:33.022 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-16T05:45:33.748 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-16T05:45:34.223 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-16T05:45:34.223 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-16T05:45:34.223 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-16T05:45:34.223 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-16T05:45:34.223 DEBUG:tasks.check_counter:Getting stats from b 2024-04-16T05:45:34.223 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-16T05:45:34.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:33 smithi082 ceph-mon[36195]: pgmap v329: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:34.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:33 smithi134 ceph-mon[36153]: pgmap v329: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:34.225 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:33 smithi032 ceph-mon[32351]: pgmap v329: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:34.474 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:45:35.881 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:35 smithi082 ceph-mon[36195]: pgmap v330: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:35.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:35 smithi032 ceph-mon[32351]: pgmap v330: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:36.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:35 smithi134 ceph-mon[36153]: pgmap v330: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-16T05:45:36.339 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:36.339 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 938, 2024-04-16T05:45:36.339 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 903, 2024-04-16T05:45:36.339 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 217556, 2024-04-16T05:45:36.339 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 686408, 2024-04-16T05:45:36.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 10, 2024-04-16T05:45:36.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-04-16T05:45:36.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.109648433, 2024-04-16T05:45:36.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.040526538, 2024-04-16T05:45:36.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.029835354, 2024-04-16T05:45:36.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.019235281, 2024-04-16T05:45:36.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:36.340 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 904, 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.068603630, 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000075888 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 914, 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000195130, 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000213 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.341 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-16T05:45:36.342 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 319584 2024-04-16T05:45:36.342 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.342 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:36.342 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1671, 2024-04-16T05:45:36.342 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 967, 2024-04-16T05:45:36.342 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 2147042, 2024-04-16T05:45:36.342 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 2291039, 2024-04-16T05:45:36.342 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 172, 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 9, 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.237852046, 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.051104376, 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.132020772, 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.058122389, 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 965, 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.081714238, 2024-04-16T05:45:36.343 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000084677 2024-04-16T05:45:36.344 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.344 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:36.344 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1264, 2024-04-16T05:45:36.344 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000208207, 2024-04-16T05:45:36.344 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000164 2024-04-16T05:45:36.344 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.344 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2043792, 2024-04-16T05:45:36.344 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 284224 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1590, 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 848, 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 497972, 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 550960, 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 25, 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 7, 2024-04-16T05:45:36.345 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.182159533, 2024-04-16T05:45:36.346 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.045381741, 2024-04-16T05:45:36.346 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.066528892, 2024-04-16T05:45:36.346 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.063208137, 2024-04-16T05:45:36.346 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:36.346 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 844, 2024-04-16T05:45:36.346 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.078059994, 2024-04-16T05:45:36.346 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000092488 2024-04-16T05:45:36.346 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1687, 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000322424, 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000191 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 137232 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.347 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-16T05:45:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-04-16T05:45:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-16T05:45:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-04-16T05:45:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-16T05:45:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-16T05:45:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 482, 2024-04-16T05:45:36.348 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.122750625, 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000254669 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000010443, 2024-04-16T05:45:36.349 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000005221 2024-04-16T05:45:36.350 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.350 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.350 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-04-16T05:45:36.350 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 166, 2024-04-16T05:45:36.350 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 166, 2024-04-16T05:45:36.350 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-04-16T05:45:36.350 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 166, 2024-04-16T05:45:36.350 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.137667640, 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000829323 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 27, 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-04-16T05:45:36.351 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-04-16T05:45:36.352 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 27, 2024-04-16T05:45:36.352 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 7, 2024-04-16T05:45:36.352 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 20, 2024-04-16T05:45:36.352 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-04-16T05:45:36.352 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 17, 2024-04-16T05:45:36.352 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-04-16T05:45:36.352 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 5, 2024-04-16T05:45:36.352 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 6, 2024-04-16T05:45:36.353 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-04-16T05:45:36.353 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 234, 2024-04-16T05:45:36.353 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 226, 2024-04-16T05:45:36.353 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-04-16T05:45:36.353 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-04-16T05:45:36.353 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:45:36.353 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-04-16T05:45:36.353 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-04-16T05:45:36.354 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-04-16T05:45:36.354 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-04-16T05:45:36.354 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-04-16T05:45:36.354 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-04-16T05:45:36.354 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-04-16T05:45:36.354 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-04-16T05:45:36.354 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-04-16T05:45:36.354 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 256, 2024-04-16T05:45:36.355 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-04-16T05:45:36.355 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 4, 2024-04-16T05:45:36.355 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 234, 2024-04-16T05:45:36.355 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 6, 2024-04-16T05:45:36.355 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 256, 2024-04-16T05:45:36.355 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-04-16T05:45:36.355 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:45:36.355 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 176, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 20, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 156, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 80, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 8, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 5, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:45:36.356 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:45:36.357 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:45:36.357 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 16, 2024-04-16T05:45:36.357 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 3, 2024-04-16T05:45:36.357 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-04-16T05:45:36.357 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 17 2024-04-16T05:45:36.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.357 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-04-16T05:45:36.357 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-04-16T05:45:36.358 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-04-16T05:45:36.358 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-04-16T05:45:36.358 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-04-16T05:45:36.358 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 4, 2024-04-16T05:45:36.358 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-04-16T05:45:36.358 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-04-16T05:45:36.358 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:45:36.358 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-04-16T05:45:36.359 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:45:36.360 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:45:36.360 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 20, 2024-04-16T05:45:36.360 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-04-16T05:45:36.360 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-04-16T05:45:36.360 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:45:36.360 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-04-16T05:45:36.360 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-04-16T05:45:36.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 112, 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 53, 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 53, 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 59, 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 1, 2024-04-16T05:45:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 8, 2024-04-16T05:45:36.362 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 7, 2024-04-16T05:45:36.362 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 7, 2024-04-16T05:45:36.362 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 1, 2024-04-16T05:45:36.362 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 1, 2024-04-16T05:45:36.362 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-04-16T05:45:36.362 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 1, 2024-04-16T05:45:36.362 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4285170, 2024-04-16T05:45:36.362 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4319791, 2024-04-16T05:45:36.363 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-04-16T05:45:36.363 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-04-16T05:45:36.363 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 90, 2024-04-16T05:45:36.363 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.163085571, 2024-04-16T05:45:36.363 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001812061 2024-04-16T05:45:36.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.363 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-04-16T05:45:36.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 25, 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 45, 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 20, 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 20, 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 20, 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 27, 2024-04-16T05:45:36.364 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 27, 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 6, 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 20, 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 14, 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 51360, 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 231800 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-04-16T05:45:36.365 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 205, 2024-04-16T05:45:36.366 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 1, 2024-04-16T05:45:36.366 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 166, 2024-04-16T05:45:36.366 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 31, 2024-04-16T05:45:36.366 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 2, 2024-04-16T05:45:36.366 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-04-16T05:45:36.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-04-16T05:45:36.366 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003011899, 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000752974 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 113, 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.024924324, 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000220569 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-04-16T05:45:36.367 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.368 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.368 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.368 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-04-16T05:45:36.368 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.368 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.368 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000500352, 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000250176 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.369 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-04-16T05:45:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-04-16T05:45:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000368388, 2024-04-16T05:45:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000184194 2024-04-16T05:45:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-04-16T05:45:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-04-16T05:45:36.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.372 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.372 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.372 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-04-16T05:45:36.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-04-16T05:45:36.372 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001642718, 2024-04-16T05:45:36.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000164271 2024-04-16T05:45:36.373 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.373 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-04-16T05:45:36.373 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 8, 2024-04-16T05:45:36.373 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.011806658, 2024-04-16T05:45:36.373 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001475832 2024-04-16T05:45:36.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.374 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-04-16T05:45:36.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.374 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.374 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-04-16T05:45:36.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.050806969, 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.005080696 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000281693, 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000281693 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.375 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-04-16T05:45:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-04-16T05:45:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-04-16T05:45:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002223396, 2024-04-16T05:45:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001111698 2024-04-16T05:45:36.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.377 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-04-16T05:45:36.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.377 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.377 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-04-16T05:45:36.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.378 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.378 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.378 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-04-16T05:45:36.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.378 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.379 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-04-16T05:45:36.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-04-16T05:45:36.379 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.018404024, 2024-04-16T05:45:36.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.004601006 2024-04-16T05:45:36.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.379 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-04-16T05:45:36.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.380 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.380 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.380 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.380 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-04-16T05:45:36.380 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-04-16T05:45:36.380 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001744334, 2024-04-16T05:45:36.380 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000872167 2024-04-16T05:45:36.380 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.381 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.382 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.382 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-04-16T05:45:36.382 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.382 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.382 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.382 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.382 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-04-16T05:45:36.382 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 8, 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.021952885, 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002744110 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.383 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-04-16T05:45:36.384 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.384 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.384 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.384 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.384 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:45:36.384 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:45:36.384 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-04-16T05:45:36.384 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.385 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.385 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.385 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.385 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.385 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-04-16T05:45:36.385 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-04-16T05:45:36.385 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 5, 2024-04-16T05:45:36.385 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 3, 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 27, 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 13, 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 608, 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-16T05:45:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:36.387 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:36.387 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:36.387 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:36.387 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:36.387 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:36.387 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:36.387 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:36.387 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:36.388 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:36.388 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:36.388 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:36.388 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:36.388 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:36.388 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:36.388 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:36.388 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:36.389 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:36.390 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:36.390 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:36.390 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:36.390 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:36.390 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:36.390 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-16T05:45:36.390 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:36.390 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:36.391 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:36.391 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:36.391 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 350245, 2024-04-16T05:45:36.391 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 143, 2024-04-16T05:45:36.391 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 880, 2024-04-16T05:45:36.391 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 10, 2024-04-16T05:45:36.391 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-16T05:45:36.391 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9320, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 178, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:36.392 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:36.393 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-16T05:45:36.393 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 109781, 2024-04-16T05:45:36.393 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 703, 2024-04-16T05:45:36.393 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:36.393 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-16T05:45:36.393 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:36.393 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-16T05:45:36.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 1009, 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 163162, 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 1009, 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-04-16T05:45:36.394 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1009, 2024-04-16T05:45:36.395 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 17.061659247, 2024-04-16T05:45:36.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.016909473 2024-04-16T05:45:36.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.395 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-04-16T05:45:36.395 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-04-16T05:45:36.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1009, 2024-04-16T05:45:36.395 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1099 2024-04-16T05:45:36.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.396 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 1009, 2024-04-16T05:45:36.396 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 296, 2024-04-16T05:45:36.396 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 713, 2024-04-16T05:45:36.396 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-04-16T05:45:36.396 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-04-16T05:45:36.396 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 8, 2024-04-16T05:45:36.396 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 27, 2024-04-16T05:45:36.396 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 20, 2024-04-16T05:45:36.397 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 90, 2024-04-16T05:45:36.397 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 40, 2024-04-16T05:45:36.397 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-04-16T05:45:36.397 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-04-16T05:45:36.397 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-04-16T05:45:36.397 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-04-16T05:45:36.397 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 506, 2024-04-16T05:45:36.397 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 276, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 50, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-04-16T05:45:36.398 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-04-16T05:45:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-04-16T05:45:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:45:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-04-16T05:45:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:45:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-04-16T05:45:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-04-16T05:45:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-04-16T05:45:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-04-16T05:45:36.400 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-04-16T05:45:36.401 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-04-16T05:45:36.401 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-04-16T05:45:36.401 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-04-16T05:45:36.401 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-04-16T05:45:36.401 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 100, 2024-04-16T05:45:36.401 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-04-16T05:45:36.401 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 16, 2024-04-16T05:45:36.401 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2024-04-16T05:45:36.402 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2024-04-16T05:45:36.402 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-04-16T05:45:36.402 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-04-16T05:45:36.402 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 75, 2024-04-16T05:45:36.402 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-04-16T05:45:36.402 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 7 2024-04-16T05:45:36.402 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.402 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.403 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-04-16T05:45:36.404 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-04-16T05:45:36.404 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:45:36.404 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-04-16T05:45:36.404 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-04-16T05:45:36.404 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-04-16T05:45:36.404 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-04-16T05:45:36.404 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-04-16T05:45:36.404 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.405 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:36.405 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:36.405 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-16T05:45:36.405 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:36.405 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 4195, 2024-04-16T05:45:36.405 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 2424270, 2024-04-16T05:45:36.405 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:36.405 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 4195, 2024-04-16T05:45:36.406 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:36.406 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:36.406 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 4195, 2024-04-16T05:45:36.406 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2424270, 2024-04-16T05:45:36.406 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:36.406 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.406 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.406 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:36.407 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:36.408 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:36.408 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 1009, 2024-04-16T05:45:36.408 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 163162, 2024-04-16T05:45:36.408 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1009, 2024-04-16T05:45:36.408 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 163162, 2024-04-16T05:45:36.408 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:36.408 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.408 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:36.409 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:36.410 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:36.410 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:36.410 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 1009, 2024-04-16T05:45:36.410 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 1009, 2024-04-16T05:45:36.410 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1009, 2024-04-16T05:45:36.410 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1009, 2024-04-16T05:45:36.410 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:36.410 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.411 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.411 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.411 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.411 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.411 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:45:36.411 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:36.411 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-16T05:45:36.411 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-04-16T05:45:36.412 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x5560ac412500": { 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-16T05:45:36.413 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:36.414 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 112, 2024-04-16T05:45:36.414 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 125487, 2024-04-16T05:45:36.414 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:36.414 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 112, 2024-04-16T05:45:36.414 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:36.414 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:36.414 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 90, 2024-04-16T05:45:36.414 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 125487, 2024-04-16T05:45:36.415 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:36.415 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:36.415 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:36.415 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:36.415 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.415 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:36.415 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-16T05:45:37.164 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-16T05:45:37.164 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-16T05:45:37.164 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-16T05:45:37.164 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-16T05:45:37.164 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-16T05:45:37.164 DEBUG:tasks.check_counter:Getting stats from e 2024-04-16T05:45:37.165 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-16T05:45:37.405 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:45:38.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:37 smithi134 ceph-mon[36153]: pgmap v331: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:38.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:37 smithi082 ceph-mon[36195]: pgmap v331: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:38.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:37 smithi032 ceph-mon[32351]: pgmap v331: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:39.123 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-16T05:45:39.123 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:39.123 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2226, 2024-04-16T05:45:39.124 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2228, 2024-04-16T05:45:39.124 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 630217, 2024-04-16T05:45:39.124 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 707875, 2024-04-16T05:45:39.124 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-04-16T05:45:39.124 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-04-16T05:45:39.124 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.176620756, 2024-04-16T05:45:39.124 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.058079739, 2024-04-16T05:45:39.124 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.042491941, 2024-04-16T05:45:39.125 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.061608192, 2024-04-16T05:45:39.125 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:39.125 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2226, 2024-04-16T05:45:39.125 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.074126116, 2024-04-16T05:45:39.125 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000033300 2024-04-16T05:45:39.125 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.125 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:39.125 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.126 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.126 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.126 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.126 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:39.126 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:39.126 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.126 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:39.126 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 8, 2024-04-16T05:45:39.127 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 170, 2024-04-16T05:45:39.127 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3868, 2024-04-16T05:45:39.127 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 450730, 2024-04-16T05:45:39.127 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-04-16T05:45:39.127 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-04-16T05:45:39.127 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.010622057, 2024-04-16T05:45:39.127 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.006541357, 2024-04-16T05:45:39.127 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002136271, 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000043913, 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 168, 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.017474984, 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000104017 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.128 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.129 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.129 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.129 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-16T05:45:39.129 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 450032 2024-04-16T05:45:39.129 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.129 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:39.130 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 247, 2024-04-16T05:45:39.130 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 235, 2024-04-16T05:45:39.130 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1619244, 2024-04-16T05:45:39.130 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 136910, 2024-04-16T05:45:39.130 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-04-16T05:45:39.130 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-04-16T05:45:39.130 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.047461912, 2024-04-16T05:45:39.130 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.014694984, 2024-04-16T05:45:39.131 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.036128498, 2024-04-16T05:45:39.131 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.005737671, 2024-04-16T05:45:39.131 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:39.131 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 234, 2024-04-16T05:45:39.131 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.013188043, 2024-04-16T05:45:39.131 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000056359 2024-04-16T05:45:39.131 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.131 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1618976, 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 136592 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-16T05:45:39.132 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-04-16T05:45:39.133 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-16T05:45:39.133 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.133 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-04-16T05:45:39.133 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-16T05:45:39.133 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-16T05:45:39.133 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3755, 2024-04-16T05:45:39.133 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.093402073, 2024-04-16T05:45:39.133 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000024874 2024-04-16T05:45:39.134 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.134 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.134 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-04-16T05:45:39.134 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-16T05:45:39.134 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-16T05:45:39.134 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.134 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.134 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.136 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.136 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-04-16T05:45:39.136 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-04-16T05:45:39.136 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-04-16T05:45:39.136 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-04-16T05:45:39.136 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-04-16T05:45:39.136 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-04-16T05:45:39.136 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-04-16T05:45:39.137 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 0, 2024-04-16T05:45:39.137 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-04-16T05:45:39.137 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 0, 2024-04-16T05:45:39.137 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-04-16T05:45:39.137 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 0, 2024-04-16T05:45:39.137 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-04-16T05:45:39.137 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-04-16T05:45:39.137 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-04-16T05:45:39.138 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 1, 2024-04-16T05:45:39.138 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-04-16T05:45:39.138 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-04-16T05:45:39.138 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-04-16T05:45:39.138 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-04-16T05:45:39.138 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:45:39.138 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-04-16T05:45:39.139 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-04-16T05:45:39.139 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-04-16T05:45:39.139 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-04-16T05:45:39.139 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-04-16T05:45:39.139 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-04-16T05:45:39.139 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-04-16T05:45:39.139 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-04-16T05:45:39.139 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:45:39.140 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:45:39.141 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:45:39.141 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:45:39.141 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:45:39.141 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-04-16T05:45:39.141 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:45:39.141 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:45:39.141 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:45:39.142 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:45:39.142 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:45:39.142 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:45:39.142 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-04-16T05:45:39.142 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:45:39.142 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-04-16T05:45:39.142 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-04-16T05:45:39.143 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-04-16T05:45:39.144 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:45:39.144 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-04-16T05:45:39.144 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-04-16T05:45:39.144 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-04-16T05:45:39.144 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-04-16T05:45:39.144 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-04-16T05:45:39.144 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:45:39.144 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:45:39.145 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-04-16T05:45:39.146 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-04-16T05:45:39.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.146 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-04-16T05:45:39.146 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-04-16T05:45:39.146 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2024-04-16T05:45:39.146 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 4, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-04-16T05:45:39.147 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-04-16T05:45:39.148 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-04-16T05:45:39.148 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-04-16T05:45:39.148 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-04-16T05:45:39.148 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4195424, 2024-04-16T05:45:39.148 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 4195283, 2024-04-16T05:45:39.148 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-04-16T05:45:39.148 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.148 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 4 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 3, 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 3, 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-04-16T05:45:39.149 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 1, 2024-04-16T05:45:39.150 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 1, 2024-04-16T05:45:39.150 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-04-16T05:45:39.150 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 0, 2024-04-16T05:45:39.150 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 0, 2024-04-16T05:45:39.150 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-04-16T05:45:39.150 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-04-16T05:45:39.150 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-04-16T05:45:39.150 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-04-16T05:45:39.151 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 48172, 2024-04-16T05:45:39.151 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215416 2024-04-16T05:45:39.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.151 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-04-16T05:45:39.151 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-04-16T05:45:39.151 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-04-16T05:45:39.151 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-04-16T05:45:39.151 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.152 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.153 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.153 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.153 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-04-16T05:45:39.153 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.153 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.153 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.153 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.153 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.155 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-04-16T05:45:39.155 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.155 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.155 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.155 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.155 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-04-16T05:45:39.155 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.155 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-04-16T05:45:39.156 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.157 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.157 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.157 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-04-16T05:45:39.157 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.157 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.157 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.158 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-04-16T05:45:39.158 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.158 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.158 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.158 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.158 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-04-16T05:45:39.158 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.159 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.159 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.159 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.159 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-04-16T05:45:39.160 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.160 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.160 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.160 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.160 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-04-16T05:45:39.160 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.160 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.161 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.161 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.161 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-04-16T05:45:39.161 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.161 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.161 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.161 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.161 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-04-16T05:45:39.162 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.162 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.162 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.162 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.162 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-04-16T05:45:39.162 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.162 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.162 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.163 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.164 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.164 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.164 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-04-16T05:45:39.164 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.164 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.164 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.164 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.164 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.165 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.166 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-04-16T05:45:39.166 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.166 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.166 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.166 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.166 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-04-16T05:45:39.166 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.166 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.167 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.167 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.167 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-04-16T05:45:39.167 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.167 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.167 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.167 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.167 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-04-16T05:45:39.168 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.168 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.168 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.168 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.168 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-04-16T05:45:39.168 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.168 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-04-16T05:45:39.169 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.170 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.170 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.170 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.170 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:45:39.170 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:45:39.170 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-04-16T05:45:39.170 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 0, 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 1, 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-04-16T05:45:39.171 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-04-16T05:45:39.172 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-04-16T05:45:39.172 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-04-16T05:45:39.172 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-04-16T05:45:39.172 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 1472, 2024-04-16T05:45:39.172 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-04-16T05:45:39.172 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.172 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-16T05:45:39.172 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:39.173 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:39.174 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:39.174 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:39.174 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:39.174 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:39.174 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:39.174 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:39.174 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:39.174 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:39.175 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:39.175 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:39.175 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:39.175 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:39.175 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:39.175 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:39.175 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:39.175 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:39.176 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:39.177 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:39.177 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 181353, 2024-04-16T05:45:39.177 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 48, 2024-04-16T05:45:39.177 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:39.177 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:39.177 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-16T05:45:39.177 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-16T05:45:39.177 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:39.178 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:39.178 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:39.178 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:39.178 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9320, 2024-04-16T05:45:39.178 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 178, 2024-04-16T05:45:39.178 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:39.178 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:39.178 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 8512, 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 13, 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.179 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-04-16T05:45:39.180 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-04-16T05:45:39.180 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-04-16T05:45:39.180 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 2229, 2024-04-16T05:45:39.180 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 21, 2024-04-16T05:45:39.180 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 3, 2024-04-16T05:45:39.180 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 2229, 2024-04-16T05:45:39.180 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-04-16T05:45:39.180 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2229, 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.141977134, 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000512327 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2229, 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2230 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.181 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 2229, 2024-04-16T05:45:39.182 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 2229, 2024-04-16T05:45:39.182 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2024-04-16T05:45:39.182 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-04-16T05:45:39.182 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-04-16T05:45:39.182 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 1458, 2024-04-16T05:45:39.182 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-04-16T05:45:39.182 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 769, 2024-04-16T05:45:39.182 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2024-04-16T05:45:39.183 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2024-04-16T05:45:39.183 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-04-16T05:45:39.183 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-04-16T05:45:39.183 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-04-16T05:45:39.183 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-04-16T05:45:39.183 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2024-04-16T05:45:39.183 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-04-16T05:45:39.183 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-04-16T05:45:39.184 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-04-16T05:45:39.185 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:45:39.185 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-04-16T05:45:39.185 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:45:39.185 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-04-16T05:45:39.185 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-04-16T05:45:39.185 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-04-16T05:45:39.185 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-04-16T05:45:39.185 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-04-16T05:45:39.186 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-04-16T05:45:39.186 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-04-16T05:45:39.186 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-04-16T05:45:39.186 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-04-16T05:45:39.186 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-04-16T05:45:39.186 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-04-16T05:45:39.186 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-04-16T05:45:39.186 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 100, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 11, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 4, 2024-04-16T05:45:39.187 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 4, 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-04-16T05:45:39.188 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:45:39.189 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-04-16T05:45:39.189 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-04-16T05:45:39.189 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.189 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-04-16T05:45:39.189 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-04-16T05:45:39.189 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.189 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-04-16T05:45:39.189 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-04-16T05:45:39.190 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:45:39.190 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-04-16T05:45:39.190 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-04-16T05:45:39.190 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-04-16T05:45:39.190 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-04-16T05:45:39.190 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-04-16T05:45:39.190 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.190 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2477, 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 2014772, 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2477, 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:39.191 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:39.192 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2477, 2024-04-16T05:45:39.192 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2014772, 2024-04-16T05:45:39.192 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:39.192 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.192 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.192 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.192 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.192 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:39.193 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2229, 2024-04-16T05:45:39.194 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 1125, 2024-04-16T05:45:39.194 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2229, 2024-04-16T05:45:39.194 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1125, 2024-04-16T05:45:39.194 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:39.194 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.194 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.194 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.194 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.195 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.195 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-04-16T05:45:39.195 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:39.195 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-04-16T05:45:39.195 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:39.195 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:39.195 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:39.195 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2229, 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 2229, 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2229, 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2229, 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.196 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.197 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.197 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.197 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:45:39.197 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:39.197 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-16T05:45:39.197 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:39.197 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:39.197 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:39.198 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:39.198 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:39.198 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:39.198 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:39.198 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-04-16T05:45:39.198 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-04-16T05:45:39.198 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:39.198 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55ec747a46e0": { 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:39.199 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:39.200 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:39.200 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:39.200 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:39.200 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:39.200 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:39.200 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-04-16T05:45:39.200 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-04-16T05:45:39.200 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:39.201 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:39.201 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:39.201 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:39.201 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.201 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:39.201 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-16T05:45:39.938 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-16T05:45:39.938 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-16T05:45:39.939 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-16T05:45:39.939 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:45:39.939 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:45:39.939 DEBUG:tasks.check_counter:Getting stats from h 2024-04-16T05:45:39.939 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-16T05:45:39.964 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:39 smithi082 ceph-mon[36195]: pgmap v332: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:40.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:39 smithi134 ceph-mon[36153]: pgmap v332: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:40.176 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:45:40.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:39 smithi032 ceph-mon[32351]: pgmap v332: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:41.986 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:41 smithi082 ceph-mon[36195]: pgmap v333: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:41.987 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-16T05:45:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 403, 2024-04-16T05:45:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 569, 2024-04-16T05:45:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 103824, 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 106655, 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.064315735, 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.029495334, 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.025621055, 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004001264, 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 567, 2024-04-16T05:45:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.045002767, 2024-04-16T05:45:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000079369 2024-04-16T05:45:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 918, 2024-04-16T05:45:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000230353, 2024-04-16T05:45:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000250 2024-04-16T05:45:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 342, 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 326, 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1654619, 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 211667, 2024-04-16T05:45:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 12, 2024-04-16T05:45:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-04-16T05:45:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.056929974, 2024-04-16T05:45:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.017594866, 2024-04-16T05:45:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.043645575, 2024-04-16T05:45:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.006069453, 2024-04-16T05:45:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 325, 2024-04-16T05:45:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.028863180, 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000088809 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 104, 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000018898, 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000181 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1632832, 2024-04-16T05:45:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 140688 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 54, 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 214, 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 11319, 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 927639, 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 49, 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.025547762, 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.007657670, 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.014411841, 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000272493, 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 212, 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.027403818, 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000129263 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 40, 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000003271, 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000081 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 449680 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-16T05:45:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-04-16T05:45:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-16T05:45:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-04-16T05:45:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-16T05:45:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-16T05:45:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 43, 2024-04-16T05:45:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001313425, 2024-04-16T05:45:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000030544 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000025123, 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000012561 2024-04-16T05:45:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 11, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-04-16T05:45:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-04-16T05:45:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 10, 2024-04-16T05:45:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-04-16T05:45:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-04-16T05:45:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-04-16T05:45:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-04-16T05:45:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-04-16T05:45:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 1, 2024-04-16T05:45:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-04-16T05:45:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-04-16T05:45:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-04-16T05:45:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-04-16T05:45:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-04-16T05:45:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-04-16T05:45:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:45:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-04-16T05:45:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 4, 2024-04-16T05:45:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 234, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 6, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:45:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-04-16T05:45:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:45:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:45:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:45:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:45:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:45:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:45:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-04-16T05:45:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:45:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-04-16T05:45:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-04-16T05:45:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-04-16T05:45:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-04-16T05:45:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-04-16T05:45:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-04-16T05:45:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 1, 2024-04-16T05:45:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-04-16T05:45:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-04-16T05:45:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-04-16T05:45:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:45:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-04-16T05:45:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-04-16T05:45:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-04-16T05:45:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:45:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 4, 2024-04-16T05:45:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 4, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 2, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-04-16T05:45:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-04-16T05:45:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-04-16T05:45:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-04-16T05:45:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-04-16T05:45:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4195424, 2024-04-16T05:45:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-04-16T05:45:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-04-16T05:45:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-04-16T05:45:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005243095, 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001747698 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 11, 2024-04-16T05:45:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 1, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 52964, 2024-04-16T05:45:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 223608 2024-04-16T05:45:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-04-16T05:45:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-04-16T05:45:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-04-16T05:45:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-04-16T05:45:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 4, 2024-04-16T05:45:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-04-16T05:45:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-04-16T05:45:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-04-16T05:45:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-04-16T05:45:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-04-16T05:45:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-04-16T05:45:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-04-16T05:45:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-04-16T05:45:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-04-16T05:45:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-04-16T05:45:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-04-16T05:45:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-04-16T05:45:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-04-16T05:45:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-04-16T05:45:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-04-16T05:45:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-04-16T05:45:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-04-16T05:45:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-04-16T05:45:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 0, 2024-04-16T05:45:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 1, 2024-04-16T05:45:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-04-16T05:45:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 0, 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 62, 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 245831, 2024-04-16T05:45:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 75, 2024-04-16T05:45:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-16T05:45:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-16T05:45:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9320, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 178, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-16T05:45:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 50872, 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 198, 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-04-16T05:45:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-04-16T05:45:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 38, 2024-04-16T05:45:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 11332, 2024-04-16T05:45:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 28, 2024-04-16T05:45:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 38, 2024-04-16T05:45:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-04-16T05:45:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 38, 2024-04-16T05:45:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.217995675, 2024-04-16T05:45:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.005736728 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 38, 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 60 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 38, 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-04-16T05:45:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 38, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 10, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 3, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 6, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-04-16T05:45:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-04-16T05:45:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-04-16T05:45:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-04-16T05:45:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6, 2024-04-16T05:45:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-04-16T05:45:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-04-16T05:45:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-04-16T05:45:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-04-16T05:45:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 20, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-04-16T05:45:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:45:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-04-16T05:45:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-04-16T05:45:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-04-16T05:45:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-04-16T05:45:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-04-16T05:45:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-04-16T05:45:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-04-16T05:45:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-04-16T05:45:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-04-16T05:45:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-04-16T05:45:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-04-16T05:45:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-04-16T05:45:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-04-16T05:45:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-04-16T05:45:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-04-16T05:45:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 100, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 11, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 9, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 9, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-04-16T05:45:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 14, 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 1 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-04-16T05:45:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-04-16T05:45:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-04-16T05:45:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-04-16T05:45:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-04-16T05:45:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:45:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-04-16T05:45:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 795, 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1681129, 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 795, 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 795, 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1681129, 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-04-16T05:45:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-16T05:45:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 38, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 11332, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 38, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 11332, 2024-04-16T05:45:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-04-16T05:45:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 38, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 38, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 38, 2024-04-16T05:45:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 38, 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-04-16T05:45:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-04-16T05:45:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x5648fff44500": { 2024-04-16T05:45:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 4, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1120, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 4, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 3, 2024-04-16T05:45:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1120, 2024-04-16T05:45:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:42.061 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-16T05:45:42.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:41 smithi134 ceph-mon[36153]: pgmap v333: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:42.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:41 smithi032 ceph-mon[32351]: pgmap v333: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:42.775 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-16T05:45:42.776 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-16T05:45:42.776 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-16T05:45:42.776 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:45:42.776 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:45:42.776 DEBUG:tasks.check_counter:Getting stats from k 2024-04-16T05:45:42.776 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-16T05:45:43.056 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.b/config 2024-04-16T05:45:44.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:43 smithi134 ceph-mon[36153]: pgmap v334: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:44.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:43 smithi082 ceph-mon[36195]: pgmap v334: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:44.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:43 smithi032 ceph-mon[32351]: pgmap v334: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:44.810 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-16T05:45:44.810 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:44.811 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-04-16T05:45:44.811 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-04-16T05:45:44.811 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-04-16T05:45:44.811 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-04-16T05:45:44.811 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:44.811 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-04-16T05:45:44.811 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.001140056, 2024-04-16T05:45:44.811 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T05:45:44.812 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000244260, 2024-04-16T05:45:44.812 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:44.812 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:44.812 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:44.812 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:44.812 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:44.812 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.812 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 6, 2024-04-16T05:45:44.814 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 171, 2024-04-16T05:45:44.814 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3312, 2024-04-16T05:45:44.814 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 186368, 2024-04-16T05:45:44.814 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:44.814 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:44.814 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.011968296, 2024-04-16T05:45:44.814 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.007828725, 2024-04-16T05:45:44.814 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001698029, 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 169, 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.019951503, 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000118056 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:44.815 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:44.816 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:44.816 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.816 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-16T05:45:44.816 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 186368 2024-04-16T05:45:44.816 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.816 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:44.816 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 232, 2024-04-16T05:45:44.816 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 221, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1592480, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 135744, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.051073855, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.015324526, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.038835923, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.005792280, 2024-04-16T05:45:44.817 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:44.818 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 220, 2024-04-16T05:45:44.818 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.013482456, 2024-04-16T05:45:44.818 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000061283 2024-04-16T05:45:44.818 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.818 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:44.818 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:44.818 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:44.818 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1592480, 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 135744 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.819 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-16T05:45:44.820 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:44.820 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:44.820 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:44.820 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:44.820 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:44.820 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:44.820 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:44.820 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:44.821 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:44.821 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:44.821 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:44.821 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:44.821 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:44.821 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:44.821 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:44.821 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:44.822 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:44.822 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:44.822 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:44.822 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:44.822 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:44.822 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:44.822 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:44.822 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-16T05:45:44.823 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:44.824 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:44.824 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:44.824 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:44.824 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148967, 2024-04-16T05:45:44.824 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 29, 2024-04-16T05:45:44.824 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:44.824 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:44.824 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:44.825 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:44.826 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:44.826 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-16T05:45:44.826 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-04-16T05:45:44.826 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-04-16T05:45:44.826 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:44.826 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-16T05:45:44.826 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:44.826 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 234, 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1559763, 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:44.827 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 234, 2024-04-16T05:45:44.828 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-16T05:45:44.828 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-16T05:45:44.828 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 234, 2024-04-16T05:45:44.828 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1559763, 2024-04-16T05:45:44.828 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-16T05:45:44.828 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-16T05:45:44.828 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-16T05:45:44.828 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-16T05:45:44.829 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:44.829 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-16T05:45:44.829 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-16T05:45:45.524 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-16T05:45:45.524 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-16T05:45:45.524 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-16T05:45:45.525 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-16T05:45:45.525 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-16T05:45:45.525 DEBUG:tasks.check_counter:Getting stats from c 2024-04-16T05:45:45.525 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-16T05:45:45.776 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:45:46.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:45 smithi134 ceph-mon[36153]: pgmap v335: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:46.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:45 smithi082 ceph-mon[36195]: pgmap v335: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:46.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:45 smithi032 ceph-mon[32351]: pgmap v335: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-16T05:45:47.570 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1844, 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1830, 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2221805, 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 643778, 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.204324462, 2024-04-16T05:45:47.571 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.066695885, 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.074564273, 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.056834926, 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1830, 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.073154513, 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000039975 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.572 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1622128, 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 137200 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.573 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 6, 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 171, 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1976, 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 454330, 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.011349186, 2024-04-16T05:45:47.574 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.006885096, 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.002208456, 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000130864, 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 168, 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.018280942, 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000108815 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.575 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 452992 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:47.576 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 774, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 776, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 190510, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 246139, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.077192613, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.028616338, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.016042197, 2024-04-16T05:45:47.577 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.024919422, 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 774, 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.028137021, 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000036352 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.578 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-16T05:45:47.579 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3943, 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.120065077, 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000030450 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.581 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.582 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 14, 2024-04-16T05:45:47.583 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 5, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 9, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 7, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-04-16T05:45:47.584 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-16T05:45:47.585 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-04-16T05:45:47.586 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:45:47.587 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:45:47.588 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-16T05:45:47.589 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-16T05:45:47.590 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-16T05:45:47.591 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-16T05:45:47.592 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 112, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-04-16T05:45:47.593 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 8, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4285170, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4319791, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 4319655, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.594 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 112 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 15, 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 15, 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-04-16T05:45:47.595 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 10, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 10, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 0, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 14, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 14, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-16T05:45:47.596 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 47968, 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215416 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-04-16T05:45:47.597 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.598 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-16T05:45:47.599 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.600 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.602 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-16T05:45:47.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-16T05:45:47.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-16T05:45:47.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.609 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-16T05:45:47.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-16T05:45:47.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-16T05:45:47.615 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 5, 2024-04-16T05:45:47.616 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 3, 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 3, 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 2920, 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.617 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:47.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:47.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:47.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 189494, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 53, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-16T05:45:47.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9320, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 178, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:47.623 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 61037, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 318, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.624 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2371, 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 26, 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 4, 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2371, 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-16T05:45:47.625 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2371, 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.080975199, 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000455915 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2371, 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2372 2024-04-16T05:45:47.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2371, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2371, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1542, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-04-16T05:45:47.627 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 826, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-16T05:45:47.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-16T05:45:47.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-16T05:45:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-16T05:45:47.631 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 100, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 11, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 5, 2024-04-16T05:45:47.632 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 5, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-04-16T05:45:47.633 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-16T05:45:47.634 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.635 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2620, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2162818, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2620, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:47.636 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2620, 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2162818, 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:47.638 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2371, 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 125497, 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2371, 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 125497, 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.639 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:47.640 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2371, 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2371, 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2371, 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2371, 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.641 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.643 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55ca0c1da6e0": { 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:47.645 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:47.646 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-16T05:45:48.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:47 smithi134 ceph-mon[36153]: pgmap v336: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:48.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:47 smithi082 ceph-mon[36195]: pgmap v336: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:48.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:47 smithi032 ceph-mon[32351]: pgmap v336: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:48.470 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-16T05:45:48.470 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-16T05:45:48.470 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-16T05:45:48.470 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:45:48.470 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:45:48.470 DEBUG:tasks.check_counter:Getting stats from f 2024-04-16T05:45:48.471 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-16T05:45:48.717 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:45:50.027 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:49 smithi134 ceph-mon[36153]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:50.148 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:49 smithi082 ceph-mon[36195]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:50.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:49 smithi032 ceph-mon[32351]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:50.282 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-16T05:45:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 782, 2024-04-16T05:45:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 784, 2024-04-16T05:45:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 207095, 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 248740, 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.067732112, 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.023387149, 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.016831997, 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.019544277, 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 782, 2024-04-16T05:45:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.027214665, 2024-04-16T05:45:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000034801 2024-04-16T05:45:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 960, 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 946, 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1796702, 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 363469, 2024-04-16T05:45:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.117494378, 2024-04-16T05:45:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.033657749, 2024-04-16T05:45:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.056721842, 2024-04-16T05:45:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.031690054, 2024-04-16T05:45:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 946, 2024-04-16T05:45:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.035624468, 2024-04-16T05:45:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000037658 2024-04-16T05:45:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1622512, 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 138320 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 776, 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 941, 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 260384, 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 697508, 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-04-16T05:45:50.288 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.106449623, 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.038200237, 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.025126895, 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.031047188, 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 938, 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.063248580, 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000067429 2024-04-16T05:45:50.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:50.290 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.290 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.290 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T05:45:50.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 451376 2024-04-16T05:45:50.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3699, 2024-04-16T05:45:50.291 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.113397504, 2024-04-16T05:45:50.292 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000030656 2024-04-16T05:45:50.292 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.292 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-16T05:45:50.292 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-16T05:45:50.292 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-16T05:45:50.292 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.292 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.293 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.294 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.294 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.294 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-16T05:45:50.294 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-16T05:45:50.294 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-16T05:45:50.294 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-16T05:45:50.294 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-04-16T05:45:50.294 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-16T05:45:50.295 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-16T05:45:50.296 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-16T05:45:50.297 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-16T05:45:50.297 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-16T05:45:50.297 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-16T05:45:50.297 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-16T05:45:50.297 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-16T05:45:50.297 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-16T05:45:50.297 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:45:50.297 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:45:50.298 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:45:50.299 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-16T05:45:50.300 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:45:50.300 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-16T05:45:50.300 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-16T05:45:50.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.300 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-16T05:45:50.300 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-16T05:45:50.300 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-16T05:45:50.300 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-16T05:45:50.301 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-16T05:45:50.302 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-16T05:45:50.302 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:45:50.302 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:45:50.302 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-16T05:45:50.302 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-16T05:45:50.302 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:45:50.302 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:45:50.302 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-16T05:45:50.303 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 6, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-16T05:45:50.304 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-04-16T05:45:50.305 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-16T05:45:50.305 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-16T05:45:50.305 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-16T05:45:50.305 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-16T05:45:50.305 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4195956, 2024-04-16T05:45:50.305 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 4195815, 2024-04-16T05:45:50.305 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-16T05:45:50.305 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 6 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 3, 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 3, 2024-04-16T05:45:50.306 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 1, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 1, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 0, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 0, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 0, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-04-16T05:45:50.307 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 47728, 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215416 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-04-16T05:45:50.308 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-04-16T05:45:50.309 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-16T05:45:50.309 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.309 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.309 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.309 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-16T05:45:50.309 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.309 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-16T05:45:50.310 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.311 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.311 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.311 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-16T05:45:50.311 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.311 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.311 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.312 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.313 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.314 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.314 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-16T05:45:50.314 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.314 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.314 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.314 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-16T05:45:50.315 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.315 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.315 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.315 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-16T05:45:50.315 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.316 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.316 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.316 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-16T05:45:50.316 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.316 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.316 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.317 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.318 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.318 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-16T05:45:50.318 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.318 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.318 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.318 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.320 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-16T05:45:50.320 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.320 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.320 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.320 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-16T05:45:50.320 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.320 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-16T05:45:50.321 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.322 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.322 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.322 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-16T05:45:50.322 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.322 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.322 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.323 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.324 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.325 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.325 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-16T05:45:50.325 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.325 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.325 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.325 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-16T05:45:50.326 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.326 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.326 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.326 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:45:50.326 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:45:50.326 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-16T05:45:50.326 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 0, 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 2, 2024-04-16T05:45:50.327 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1600, 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:50.329 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:50.329 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:50.329 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:50.329 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:50.329 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:50.329 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:50.329 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:50.329 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:50.330 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:50.331 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:50.331 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:50.331 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:50.331 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:50.331 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:50.331 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:50.331 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:50.331 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:50.332 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 189339, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 50, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:50.333 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:50.334 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:50.334 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9320, 2024-04-16T05:45:50.334 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 178, 2024-04-16T05:45:50.334 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:50.334 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:50.334 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:50.334 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-16T05:45:50.337 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 8512, 2024-04-16T05:45:50.337 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 13, 2024-04-16T05:45:50.337 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:50.337 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-16T05:45:50.338 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:50.338 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-16T05:45:50.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.338 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-16T05:45:50.338 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-16T05:45:50.338 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-16T05:45:50.338 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2264, 2024-04-16T05:45:50.338 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-04-16T05:45:50.339 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-04-16T05:45:50.339 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2264, 2024-04-16T05:45:50.339 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-16T05:45:50.339 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2264, 2024-04-16T05:45:50.339 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.063975578, 2024-04-16T05:45:50.339 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000469953 2024-04-16T05:45:50.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.339 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2264, 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2265 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2264, 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2264, 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-16T05:45:50.340 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1480, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 782, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-16T05:45:50.341 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-16T05:45:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-04-16T05:45:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-16T05:45:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-16T05:45:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-16T05:45:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-04-16T05:45:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-04-16T05:45:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:45:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-16T05:45:50.343 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-16T05:45:50.344 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-16T05:45:50.344 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-16T05:45:50.344 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-16T05:45:50.344 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-16T05:45:50.344 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-16T05:45:50.344 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-16T05:45:50.344 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-16T05:45:50.344 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 100, 2024-04-16T05:45:50.345 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-16T05:45:50.346 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 11, 2024-04-16T05:45:50.346 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 5, 2024-04-16T05:45:50.346 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 5, 2024-04-16T05:45:50.346 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-16T05:45:50.346 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-16T05:45:50.346 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-04-16T05:45:50.346 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-04-16T05:45:50.346 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-16T05:45:50.347 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-16T05:45:50.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-16T05:45:50.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.349 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:50.349 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:50.349 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-16T05:45:50.349 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:50.349 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2514, 2024-04-16T05:45:50.349 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2022190, 2024-04-16T05:45:50.349 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2514, 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2514, 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2022190, 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.350 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:50.351 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2264, 2024-04-16T05:45:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1657, 2024-04-16T05:45:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2264, 2024-04-16T05:45:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1657, 2024-04-16T05:45:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.352 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:50.354 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.354 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:50.354 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2264, 2024-04-16T05:45:50.354 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2264, 2024-04-16T05:45:50.354 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2264, 2024-04-16T05:45:50.354 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2264, 2024-04-16T05:45:50.354 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:50.354 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:50.355 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:50.356 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:50.356 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.356 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:50.356 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:50.356 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:50.356 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-16T05:45:50.356 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-16T05:45:50.356 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55feb88906e0": { 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-16T05:45:50.357 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-16T05:45:50.358 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:50.359 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:50.359 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:50.359 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:50.359 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.359 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:50.359 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-16T05:45:51.083 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-16T05:45:51.084 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-16T05:45:51.084 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-16T05:45:51.084 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:45:51.084 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:45:51.084 DEBUG:tasks.check_counter:Getting stats from i 2024-04-16T05:45:51.084 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-16T05:45:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:45:52.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:51 smithi134 ceph-mon[36153]: pgmap v338: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:52.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:51 smithi082 ceph-mon[36195]: pgmap v338: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:52.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:51 smithi032 ceph-mon[32351]: pgmap v338: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:53.002 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-16T05:45:53.003 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:53.003 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 35, 2024-04-16T05:45:53.003 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 171, 2024-04-16T05:45:53.003 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 8750, 2024-04-16T05:45:53.003 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 385117, 2024-04-16T05:45:53.003 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 14, 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.017272761, 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.006580158, 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.004201218, 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000569209, 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 167, 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.034603103, 2024-04-16T05:45:53.004 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000207204 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 17, 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000002257, 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000132 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2096, 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 325936 2024-04-16T05:45:53.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.006 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:53.006 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 348, 2024-04-16T05:45:53.006 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 330, 2024-04-16T05:45:53.006 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1655017, 2024-04-16T05:45:53.006 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 173836, 2024-04-16T05:45:53.006 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-04-16T05:45:53.006 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-04-16T05:45:53.006 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.056756992, 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.018672853, 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.041768989, 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.006070292, 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 332, 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.026572644, 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000080038 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.007 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 101, 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000021402, 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000211 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1631936, 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 142160 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 444, 2024-04-16T05:45:53.008 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 642, 2024-04-16T05:45:53.009 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 109089, 2024-04-16T05:45:53.009 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 717867, 2024-04-16T05:45:53.009 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 48, 2024-04-16T05:45:53.009 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:53.009 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.073715554, 2024-04-16T05:45:53.009 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.026086860, 2024-04-16T05:45:53.009 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.039886309, 2024-04-16T05:45:53.009 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.004483600, 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 639, 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.034813382, 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000054481 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 992, 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000259460, 2024-04-16T05:45:53.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000261 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 133136 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.011 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 55, 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001768566, 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000032155 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-16T05:45:53.012 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000017906, 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000008953 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 6, 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 4, 2024-04-16T05:45:53.013 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-16T05:45:53.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4, 2024-04-16T05:45:53.014 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002130598, 2024-04-16T05:45:53.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000532649 2024-04-16T05:45:53.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.014 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-16T05:45:53.014 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 2, 2024-04-16T05:45:53.014 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-16T05:45:53.014 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 11, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 11, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 10, 2024-04-16T05:45:53.015 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 1, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 12, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 6, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 2, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-16T05:45:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 1000, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 1, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-16T05:45:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-16T05:45:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T05:45:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 2 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-16T05:45:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 3, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-16T05:45:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-16T05:45:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 6, 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-16T05:45:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 6, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 2, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-16T05:45:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4195956, 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5, 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.009712717, 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001942543 2024-04-16T05:45:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-04-16T05:45:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-16T05:45:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 14, 2024-04-16T05:45:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 14, 2024-04-16T05:45:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-04-16T05:45:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 13, 2024-04-16T05:45:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 14, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 11, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 12, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 1, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 3, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 3, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 51880, 2024-04-16T05:45:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223608 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 10, 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 6, 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 8, 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 1, 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-16T05:45:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001004151, 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000502075 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-16T05:45:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-16T05:45:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001126447, 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000563223 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-16T05:45:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-16T05:45:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-16T05:45:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-16T05:45:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-16T05:45:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-16T05:45:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-16T05:45:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-16T05:45:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-16T05:45:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-16T05:45:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 0, 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 2, 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 0, 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 3, 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 3, 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 85, 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-16T05:45:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 247319, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 82, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-16T05:45:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9320, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 178, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-16T05:45:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 56568, 2024-04-16T05:45:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 227, 2024-04-16T05:45:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-16T05:45:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-16T05:45:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 44, 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 12189, 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 31, 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 44, 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 44, 2024-04-16T05:45:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.236994576, 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005386240 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 44, 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 68 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 44, 2024-04-16T05:45:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 0, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 44, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 0, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 5, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 8, 2024-04-16T05:45:53.057 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-16T05:45:53.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-16T05:45:53.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-16T05:45:53.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-16T05:45:53.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 6, 2024-04-16T05:45:53.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-16T05:45:53.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-16T05:45:53.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-16T05:45:53.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T05:45:53.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-16T05:45:53.060 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-16T05:45:53.060 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-16T05:45:53.060 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-16T05:45:53.060 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-16T05:45:53.060 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-16T05:45:53.060 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-16T05:45:53.060 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-16T05:45:53.060 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-16T05:45:53.061 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-16T05:45:53.061 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-16T05:45:53.061 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-16T05:45:53.061 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-16T05:45:53.061 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-16T05:45:53.061 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-16T05:45:53.061 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-16T05:45:53.061 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-16T05:45:53.062 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-16T05:45:53.062 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-16T05:45:53.062 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 100, 2024-04-16T05:45:53.062 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-16T05:45:53.062 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 11, 2024-04-16T05:45:53.062 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 8, 2024-04-16T05:45:53.062 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 8, 2024-04-16T05:45:53.062 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 17, 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 2 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-16T05:45:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-16T05:45:53.064 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-16T05:45:53.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.064 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-16T05:45:53.064 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-16T05:45:53.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.064 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-16T05:45:53.064 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-16T05:45:53.064 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:53.065 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-16T05:45:53.066 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:53.066 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 823, 2024-04-16T05:45:53.066 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1681109, 2024-04-16T05:45:53.066 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:53.066 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 823, 2024-04-16T05:45:53.066 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:53.066 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:53.066 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 823, 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1681109, 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-16T05:45:53.067 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 44, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 12189, 2024-04-16T05:45:53.068 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 44, 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 12189, 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-16T05:45:53.069 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:53.070 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-16T05:45:53.070 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:53.070 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:53.070 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:53.070 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:53.070 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:53.070 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 44, 2024-04-16T05:45:53.070 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 44, 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 44, 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 44, 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-16T05:45:53.071 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:53.072 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-16T05:45:53.073 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-16T05:45:53.073 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:53.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.073 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.073 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:53.073 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55ec4e6f0500": { 2024-04-16T05:45:53.074 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:53.074 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-16T05:45:53.074 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:53.074 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 6, 2024-04-16T05:45:53.074 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1652, 2024-04-16T05:45:53.074 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:53.074 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 6, 2024-04-16T05:45:53.074 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 5, 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1652, 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.075 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:53.076 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-16T05:45:53.767 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-16T05:45:53.767 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-16T05:45:53.767 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-16T05:45:53.767 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-16T05:45:53.767 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T05:45:53.767 DEBUG:tasks.check_counter:Getting stats from l 2024-04-16T05:45:53.768 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea shell --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-16T05:45:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/mon.c/config 2024-04-16T05:45:54.129 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:53 smithi134 ceph-mon[36153]: pgmap v339: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:54.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:53 smithi082 ceph-mon[36195]: pgmap v339: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:54.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:53 smithi032 ceph-mon[32351]: pgmap v339: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:45:55.600 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-16T05:45:55.600 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T05:45:55.600 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 0, 2024-04-16T05:45:55.600 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2, 2024-04-16T05:45:55.600 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 0, 2024-04-16T05:45:55.600 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 0, 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 0, 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.000890956, 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.000113050, 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:55.601 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:55.602 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:55.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.602 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:55.602 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:55.602 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:55.602 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:55.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.603 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T05:45:55.603 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T05:45:55.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.603 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T05:45:55.603 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 6, 2024-04-16T05:45:55.603 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 173, 2024-04-16T05:45:55.603 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3312, 2024-04-16T05:45:55.603 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 186720, 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.012601346, 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.008089529, 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.002077092, 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 171, 2024-04-16T05:45:55.604 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.022860853, 2024-04-16T05:45:55.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000133689 2024-04-16T05:45:55.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.605 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:55.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:55.605 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:55.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:55.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.606 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-16T05:45:55.606 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 186720 2024-04-16T05:45:55.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.607 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T05:45:55.607 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 236, 2024-04-16T05:45:55.607 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 225, 2024-04-16T05:45:55.607 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1593568, 2024-04-16T05:45:55.607 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 138304, 2024-04-16T05:45:55.607 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 2, 2024-04-16T05:45:55.607 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2024-04-16T05:45:55.607 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.051587487, 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.015091504, 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.039112562, 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.006223068, 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 224, 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.012129840, 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000054151 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.608 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-16T05:45:55.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:55.609 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:55.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:55.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.609 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1593568, 2024-04-16T05:45:55.609 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 138304 2024-04-16T05:45:55.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.609 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 0, 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-16T05:45:55.610 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T05:45:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-16T05:45:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T05:45:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T05:45:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T05:45:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T05:45:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T05:45:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-16T05:45:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T05:45:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T05:45:55.613 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T05:45:55.613 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-16T05:45:55.613 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-16T05:45:55.613 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-16T05:45:55.613 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T05:45:55.613 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T05:45:55.614 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-16T05:45:55.614 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-16T05:45:55.614 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-16T05:45:55.614 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-16T05:45:55.614 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T05:45:55.614 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-16T05:45:55.614 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T05:45:55.614 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-16T05:45:55.615 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 148945, 2024-04-16T05:45:55.615 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 29, 2024-04-16T05:45:55.615 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-16T05:45:55.615 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-16T05:45:55.615 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-16T05:45:55.615 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-16T05:45:55.615 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-16T05:45:55.615 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-16T05:45:55.616 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-16T05:45:55.616 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-16T05:45:55.616 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 0, 2024-04-16T05:45:55.616 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 0, 2024-04-16T05:45:55.616 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T05:45:55.616 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-16T05:45:55.616 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 928, 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 2, 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-16T05:45:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T05:45:55.618 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-16T05:45:55.618 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-16T05:45:55.618 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-16T05:45:55.618 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 238, 2024-04-16T05:45:55.618 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1560283, 2024-04-16T05:45:55.618 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-16T05:45:55.618 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 238, 2024-04-16T05:45:55.618 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 238, 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1560283, 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:55.619 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-16T05:45:55.620 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-16T05:45:56.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:55 smithi134 ceph-mon[36153]: pgmap v340: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:56.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:55 smithi082 ceph-mon[36195]: pgmap v340: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:56.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:55 smithi032 ceph-mon[32351]: pgmap v340: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:56.309 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-16T05:45:56.309 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-16T05:45:56.310 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-16T05:45:56.310 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-16T05:45:56.310 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-16T05:45:56.310 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a87fcff6340baeedd43af126e793f3241b6400ea/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2024-04-16T05:45:56.318 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-16T05:45:56.329 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-16T05:45:56.329 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:45:56.330 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:45:56.356 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:45:56.357 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:45:56.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:45:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 -- 172.21.15.32:0/1812397645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc101150 msgr2=0x7fb7fc101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 --2- 172.21.15.32:0/1812397645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc101150 0x7fb7fc101530 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009900 tx=0x7fb7e402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 -- 172.21.15.32:0/1812397645 shutdown_connections 2024-04-16T05:45:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 --2- 172.21.15.32:0/1812397645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc106220 0x7fb7fc10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 --2- 172.21.15.32:0/1812397645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7fc101b00 0x7fb7fc105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 --2- 172.21.15.32:0/1812397645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc101150 0x7fb7fc101530 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 -- 172.21.15.32:0/1812397645 >> 172.21.15.32:0/1812397645 conn(0x7fb7fc078000 msgr2=0x7fb7fc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 -- 172.21.15.32:0/1812397645 shutdown_connections 2024-04-16T05:45:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.572+0000 7fb802f8f640 1 -- 172.21.15.32:0/1812397645 wait complete. 2024-04-16T05:45:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb802f8f640 1 Processor -- start 2024-04-16T05:45:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb802f8f640 1 -- start start 2024-04-16T05:45:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb802f8f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7fc101150 0x7fb7fc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb802f8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc106220 0x7fb7fc199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb802f8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc19a180 0x7fb7fc1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb802f8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1112d0 con 0x7fb7fc19a180 2024-04-16T05:45:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb802f8f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc111150 con 0x7fb7fc106220 2024-04-16T05:45:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb802f8f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc111450 con 0x7fb7fc101150 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc19a180 0x7fb7fc1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc19a180 0x7fb7fc1a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48408/0 (socket says 172.21.15.32:48408) 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb800d2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7fc101150 0x7fb7fc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 -- 172.21.15.32:0/2494655775 learned_addr learned my addr 172.21.15.32:0/2494655775 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb7fbfff640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc106220 0x7fb7fc199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 -- 172.21.15.32:0/2494655775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7fc101150 msgr2=0x7fb7fc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7fc101150 0x7fb7fc1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 -- 172.21.15.32:0/2494655775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc106220 msgr2=0x7fb7fc199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc106220 0x7fb7fc199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 -- 172.21.15.32:0/2494655775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc1a4f00 con 0x7fb7fc19a180 2024-04-16T05:45:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.573+0000 7fb80152c640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc19a180 0x7fb7fc1a4730 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c970 tx=0x7fb7f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.574+0000 7fb7f9ffb640 1 -- 172.21.15.32:0/2494655775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4007bb0 con 0x7fb7fc19a180 2024-04-16T05:45:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.574+0000 7fb7f9ffb640 1 -- 172.21.15.32:0/2494655775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f4007d50 con 0x7fb7fc19a180 2024-04-16T05:45:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.574+0000 7fb7f9ffb640 1 -- 172.21.15.32:0/2494655775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4018800 con 0x7fb7fc19a180 2024-04-16T05:45:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.574+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc1a51f0 con 0x7fb7fc19a180 2024-04-16T05:45:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.574+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc1a56f0 con 0x7fb7fc19a180 2024-04-16T05:45:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.575+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7fc068e20 con 0x7fb7fc19a180 2024-04-16T05:45:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.576+0000 7fb7f9ffb640 1 -- 172.21.15.32:0/2494655775 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb7f400fa50 con 0x7fb7fc19a180 2024-04-16T05:45:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.576+0000 7fb7f9ffb640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7e80a3d20 0x7fb7e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:56.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.577+0000 7fb800d2b640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7e80a3d20 0x7fb7e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:56.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.577+0000 7fb7f9ffb640 1 -- 172.21.15.32:0/2494655775 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7fb7f40d0b40 con 0x7fb7fc19a180 2024-04-16T05:45:56.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.577+0000 7fb800d2b640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7e80a3d20 0x7fb7e80a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4006010 tx=0x7fb7e4005dd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:56.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.581+0000 7fb7f9ffb640 1 -- 172.21.15.32:0/2494655775 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7f4014030 con 0x7fb7fc19a180 2024-04-16T05:45:56.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.770+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7fc109450 con 0x7fb7fc19a180 2024-04-16T05:45:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.772+0000 7fb7f9ffb640 1 -- 172.21.15.32:0/2494655775 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25177 (secure 0 0 0) 0x7fb7f409cdf0 con 0x7fb7fc19a180 2024-04-16T05:45:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:45:56.771 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":100,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:45:12.798785+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1356969979":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6800/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/558122612":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2362160984":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1729232468":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:45:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7e80a3d20 msgr2=0x7fb7e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7e80a3d20 0x7fb7e80a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4006010 tx=0x7fb7e4005dd0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc19a180 msgr2=0x7fb7fc1a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc19a180 0x7fb7fc1a4730 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c970 tx=0x7fb7f400ce30 comp rx=0 tx=0).stop 2024-04-16T05:45:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 shutdown_connections 2024-04-16T05:45:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb7e80a3d20 0x7fb7e80a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7fc19a180 0x7fb7fc1a4730 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc106220 0x7fb7fc199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 --2- 172.21.15.32:0/2494655775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7fc101150 0x7fb7fc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 >> 172.21.15.32:0/2494655775 conn(0x7fb7fc078000 msgr2=0x7fb7fc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 shutdown_connections 2024-04-16T05:45:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.774+0000 7fb802f8f640 1 -- 172.21.15.32:0/2494655775 wait complete. 2024-04-16T05:45:56.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T05:27:56.907609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T05:32:32.418635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T05:32:33.067994+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T05:32:36.906465+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T05:45:56.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T05:45:56.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 -- 172.21.15.32:0/284808419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100ec0 msgr2=0x7fa81410ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:56.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 --2- 172.21.15.32:0/284808419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100ec0 0x7fa81410ee90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009cd0 tx=0x7fa80c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 -- 172.21.15.32:0/284808419 shutdown_connections 2024-04-16T05:45:56.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 --2- 172.21.15.32:0/284808419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100ec0 0x7fa81410ee90 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 --2- 172.21.15.32:0/284808419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa814100500 0x7fa814100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 --2- 172.21.15.32:0/284808419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814106fe0 0x7fa8141073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 -- 172.21.15.32:0/284808419 >> 172.21.15.32:0/284808419 conn(0x7fa814077490 msgr2=0x7fa8140fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:56.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 -- 172.21.15.32:0/284808419 shutdown_connections 2024-04-16T05:45:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.895+0000 7fa81be14640 1 -- 172.21.15.32:0/284808419 wait complete. 2024-04-16T05:45:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81be14640 1 Processor -- start 2024-04-16T05:45:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81be14640 1 -- start start 2024-04-16T05:45:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81be14640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100500 0x7fa8141016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81be14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa814100ec0 0x7fa814101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81be14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814106fe0 0x7fa814102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81be14640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa814112490 con 0x7fa814100500 2024-04-16T05:45:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81be14640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa814112310 con 0x7fa814100ec0 2024-04-16T05:45:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81be14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa814112610 con 0x7fa814106fe0 2024-04-16T05:45:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa819bb0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100500 0x7fa8141016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa819bb0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100500 0x7fa8141016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48418/0 (socket says 172.21.15.32:48418) 2024-04-16T05:45:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa819bb0640 1 -- 172.21.15.32:0/4156534180 learned_addr learned my addr 172.21.15.32:0/4156534180 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa81a3b1640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814106fe0 0x7fa814102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa8193af640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa814100ec0 0x7fa814101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa819bb0640 1 -- 172.21.15.32:0/4156534180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814106fe0 msgr2=0x7fa814102160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa819bb0640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814106fe0 0x7fa814102160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa819bb0640 1 -- 172.21.15.32:0/4156534180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa814100ec0 msgr2=0x7fa814101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa819bb0640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa814100ec0 0x7fa814101c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.896+0000 7fa819bb0640 1 -- 172.21.15.32:0/4156534180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8141a6e50 con 0x7fa814100500 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.897+0000 7fa819bb0640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100500 0x7fa8141016e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa81000d770 tx=0x7fa81000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.897+0000 7fa80affd640 1 -- 172.21.15.32:0/4156534180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810017070 con 0x7fa814100500 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.897+0000 7fa80affd640 1 -- 172.21.15.32:0/4156534180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa810005500 con 0x7fa814100500 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.897+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8141a7140 con 0x7fa814100500 2024-04-16T05:45:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.897+0000 7fa80affd640 1 -- 172.21.15.32:0/4156534180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810004e00 con 0x7fa814100500 2024-04-16T05:45:56.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.897+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8141a7640 con 0x7fa814100500 2024-04-16T05:45:56.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.898+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa814102c10 con 0x7fa814100500 2024-04-16T05:45:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.899+0000 7fa80affd640 1 -- 172.21.15.32:0/4156534180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa810004fa0 con 0x7fa814100500 2024-04-16T05:45:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.900+0000 7fa80affd640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8000a3d20 0x7fa8000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.900+0000 7fa80affd640 1 -- 172.21.15.32:0/4156534180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7fa8100d06b0 con 0x7fa814100500 2024-04-16T05:45:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.900+0000 7fa8193af640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8000a3d20 0x7fa8000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.900+0000 7fa8193af640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8000a3d20 0x7fa8000a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa804005fd0 tx=0x7fa804005850 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:56.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:56.904+0000 7fa80affd640 1 -- 172.21.15.32:0/4156534180 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa81009c960 con 0x7fa814100500 2024-04-16T05:45:57.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.091+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa81410a4e0 con 0x7fa814100500 2024-04-16T05:45:57.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.091+0000 7fa80affd640 1 -- 172.21.15.32:0/4156534180 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fa81009c260 con 0x7fa814100500 2024-04-16T05:45:57.090 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-16T05:45:57.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8000a3d20 msgr2=0x7fa8000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8000a3d20 0x7fa8000a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa804005fd0 tx=0x7fa804005850 comp rx=0 tx=0).stop 2024-04-16T05:45:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100500 msgr2=0x7fa8141016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100500 0x7fa8141016e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa81000d770 tx=0x7fa81000dc30 comp rx=0 tx=0).stop 2024-04-16T05:45:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 shutdown_connections 2024-04-16T05:45:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814106fe0 0x7fa814102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa814100ec0 0x7fa814101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa8000a3d20 0x7fa8000a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 --2- 172.21.15.32:0/4156534180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa814100500 0x7fa8141016e0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 >> 172.21.15.32:0/4156534180 conn(0x7fa814077490 msgr2=0x7fa814104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:57.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 shutdown_connections 2024-04-16T05:45:57.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.093+0000 7fa81be14640 1 -- 172.21.15.32:0/4156534180 wait complete. 2024-04-16T05:45:57.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:56 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2494655775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:57.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T05:45:57.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:56 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2494655775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:57.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:56 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2494655775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:45:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 -- 172.21.15.32:0/3498340549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d0103280 msgr2=0x7f90d0103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 --2- 172.21.15.32:0/3498340549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d0103280 0x7f90d0103660 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009900 tx=0x7f90cc02bd60 comp rx=0 tx=0).stop 2024-04-16T05:45:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 -- 172.21.15.32:0/3498340549 shutdown_connections 2024-04-16T05:45:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 --2- 172.21.15.32:0/3498340549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d010c5c0 0x7f90d010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 --2- 172.21.15.32:0/3498340549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90d0103ba0 0x7f90d010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 --2- 172.21.15.32:0/3498340549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d0103280 0x7f90d0103660 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 -- 172.21.15.32:0/3498340549 >> 172.21.15.32:0/3498340549 conn(0x7f90d0078000 msgr2=0x7f90d0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 -- 172.21.15.32:0/3498340549 shutdown_connections 2024-04-16T05:45:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 -- 172.21.15.32:0/3498340549 wait complete. 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 Processor -- start 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.213+0000 7f90d85d4640 1 -- start start 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d85d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90d0103280 0x7f90d0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d85d4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d0103ba0 0x7f90d0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d85d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d010c5c0 0x7f90d0101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d85d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01123f0 con 0x7f90d010c5c0 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d85d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f90d0112270 con 0x7f90d0103280 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d85d4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90d0112570 con 0x7f90d0103ba0 2024-04-16T05:45:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d010c5c0 0x7f90d0101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d010c5c0 0x7f90d0101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48434/0 (socket says 172.21.15.32:48434) 2024-04-16T05:45:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 -- 172.21.15.32:0/828621963 learned_addr learned my addr 172.21.15.32:0/828621963 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6370640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90d0103280 0x7f90d0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d5b6f640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d0103ba0 0x7f90d0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 -- 172.21.15.32:0/828621963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d0103ba0 msgr2=0x7f90d0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d0103ba0 0x7f90d0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 -- 172.21.15.32:0/828621963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90d0103280 msgr2=0x7f90d0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90d0103280 0x7f90d0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 -- 172.21.15.32:0/828621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d01a6d10 con 0x7f90d010c5c0 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6370640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90d0103280 0x7f90d0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d6b71640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d010c5c0 0x7f90d0101950 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f90c800eaa0 tx=0x7f90c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90c77fe640 1 -- 172.21.15.32:0/828621963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c800cdb0 con 0x7f90d010c5c0 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90c77fe640 1 -- 172.21.15.32:0/828621963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c8005020 con 0x7f90d010c5c0 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d01a7000 con 0x7f90d010c5c0 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90c77fe640 1 -- 172.21.15.32:0/828621963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c80116b0 con 0x7f90d010c5c0 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.214+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d01a7520 con 0x7f90d010c5c0 2024-04-16T05:45:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.215+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d0068e20 con 0x7f90d010c5c0 2024-04-16T05:45:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.217+0000 7f90c77fe640 1 -- 172.21.15.32:0/828621963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f90c8005560 con 0x7f90d010c5c0 2024-04-16T05:45:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.217+0000 7f90c77fe640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f90bc0a3d20 0x7f90bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.217+0000 7f90d6370640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f90bc0a3d20 0x7f90bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.217+0000 7f90c77fe640 1 -- 172.21.15.32:0/828621963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f90c80d0df0 con 0x7f90d010c5c0 2024-04-16T05:45:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.218+0000 7f90d6370640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f90bc0a3d20 0x7f90bc0a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90cc006010 tx=0x7f90cc005a20 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:57.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.222+0000 7f90c77fe640 1 -- 172.21.15.32:0/828621963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90c8016030 con 0x7f90d010c5c0 2024-04-16T05:45:57.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.408+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90d01a7880 con 0x7f90d010c5c0 2024-04-16T05:45:57.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.408+0000 7f90c77fe640 1 -- 172.21.15.32:0/828621963 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f90c809d100 con 0x7f90d010c5c0 2024-04-16T05:45:57.407 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:45:57.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f90bc0a3d20 msgr2=0x7f90bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f90bc0a3d20 0x7f90bc0a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90cc006010 tx=0x7f90cc005a20 comp rx=0 tx=0).stop 2024-04-16T05:45:57.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d010c5c0 msgr2=0x7f90d0101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d010c5c0 0x7f90d0101950 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f90c800eaa0 tx=0x7f90c800ef60 comp rx=0 tx=0).stop 2024-04-16T05:45:57.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 shutdown_connections 2024-04-16T05:45:57.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f90bc0a3d20 0x7f90bc0a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d010c5c0 0x7f90d0101950 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d0103ba0 0x7f90d0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 --2- 172.21.15.32:0/828621963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90d0103280 0x7f90d0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 >> 172.21.15.32:0/828621963 conn(0x7f90d0078000 msgr2=0x7f90d00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 shutdown_connections 2024-04-16T05:45:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.410+0000 7f90d85d4640 1 -- 172.21.15.32:0/828621963 wait complete. 2024-04-16T05:45:57.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T05:45:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 -- 172.21.15.32:0/3685280824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 msgr2=0x7fb564100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 --2- 172.21.15.32:0/3685280824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 0x7fb564100f80 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb554009900 tx=0x7fb55402bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:57.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 -- 172.21.15.32:0/3685280824 shutdown_connections 2024-04-16T05:45:57.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 --2- 172.21.15.32:0/3685280824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb564105a20 0x7fb56410d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 --2- 172.21.15.32:0/3685280824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb564101550 0x7fb5641052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 --2- 172.21.15.32:0/3685280824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 0x7fb564100f80 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 -- 172.21.15.32:0/3685280824 >> 172.21.15.32:0/3685280824 conn(0x7fb564076190 msgr2=0x7fb5640765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 -- 172.21.15.32:0/3685280824 shutdown_connections 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.530+0000 7fb5698a0640 1 -- 172.21.15.32:0/3685280824 wait complete. 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5698a0640 1 Processor -- start 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5698a0640 1 -- start start 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5698a0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 0x7fb564199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5698a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb564105a20 0x7fb564199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5698a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb56419a250 0x7fb5641a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5698a0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb564111320 con 0x7fb564100ba0 2024-04-16T05:45:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5698a0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641111a0 con 0x7fb564105a20 2024-04-16T05:45:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5698a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641114a0 con 0x7fb56419a250 2024-04-16T05:45:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb562ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 0x7fb564199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb562ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 0x7fb564199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48438/0 (socket says 172.21.15.32:48438) 2024-04-16T05:45:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb562ffd640 1 -- 172.21.15.32:0/2332250986 learned_addr learned my addr 172.21.15.32:0/2332250986 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb5637fe640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb56419a250 0x7fb5641a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.531+0000 7fb55a7fc640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb564105a20 0x7fb564199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb562ffd640 1 -- 172.21.15.32:0/2332250986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb56419a250 msgr2=0x7fb5641a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb562ffd640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb56419a250 0x7fb5641a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb562ffd640 1 -- 172.21.15.32:0/2332250986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb564105a20 msgr2=0x7fb564199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb562ffd640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb564105a20 0x7fb564199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb562ffd640 1 -- 172.21.15.32:0/2332250986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5641a5000 con 0x7fb564100ba0 2024-04-16T05:45:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb5637fe640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb56419a250 0x7fb5641a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb55a7fc640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb564105a20 0x7fb564199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb562ffd640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 0x7fb564199780 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb554036040 tx=0x7fb55402ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb56889e640 1 -- 172.21.15.32:0/2332250986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554041070 con 0x7fb564100ba0 2024-04-16T05:45:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5641a5290 con 0x7fb564100ba0 2024-04-16T05:45:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb56889e640 1 -- 172.21.15.32:0/2332250986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5540054e0 con 0x7fb564100ba0 2024-04-16T05:45:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb56889e640 1 -- 172.21.15.32:0/2332250986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554034440 con 0x7fb564100ba0 2024-04-16T05:45:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.532+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5641a57d0 con 0x7fb564100ba0 2024-04-16T05:45:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.533+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb564069270 con 0x7fb564100ba0 2024-04-16T05:45:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.534+0000 7fb56889e640 1 -- 172.21.15.32:0/2332250986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb554044070 con 0x7fb564100ba0 2024-04-16T05:45:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.535+0000 7fb56889e640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb5500a3d20 0x7fb5500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.535+0000 7fb55a7fc640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb5500a3d20 0x7fb5500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.535+0000 7fb56889e640 1 -- 172.21.15.32:0/2332250986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7fb5540f29b0 con 0x7fb564100ba0 2024-04-16T05:45:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.535+0000 7fb55a7fc640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb5500a3d20 0x7fb5500a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0058c0 tx=0x7fb54c005850 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.538+0000 7fb56889e640 1 -- 172.21.15.32:0/2332250986 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5540bee60 con 0x7fb564100ba0 2024-04-16T05:45:57.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.723+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb564108bf0 con 0x7fb564100ba0 2024-04-16T05:45:57.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.723+0000 7fb56889e640 1 -- 172.21.15.32:0/2332250986 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fb554004c50 con 0x7fb564100ba0 2024-04-16T05:45:57.722 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-16T05:45:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb5500a3d20 msgr2=0x7fb5500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb5500a3d20 0x7fb5500a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0058c0 tx=0x7fb54c005850 comp rx=0 tx=0).stop 2024-04-16T05:45:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 msgr2=0x7fb564199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 0x7fb564199780 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb554036040 tx=0x7fb55402ce20 comp rx=0 tx=0).stop 2024-04-16T05:45:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 shutdown_connections 2024-04-16T05:45:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb56419a250 0x7fb5641a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb564105a20 0x7fb564199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb5500a3d20 0x7fb5500a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 --2- 172.21.15.32:0/2332250986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb564100ba0 0x7fb564199780 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 >> 172.21.15.32:0/2332250986 conn(0x7fb564076190 msgr2=0x7fb5640fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 shutdown_connections 2024-04-16T05:45:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.725+0000 7fb5698a0640 1 -- 172.21.15.32:0/2332250986 wait complete. 2024-04-16T05:45:57.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T05:45:57.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 -- 172.21.15.32:0/1573026062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101bf0 msgr2=0x7f71ec10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 --2- 172.21.15.32:0/1573026062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101bf0 0x7f71ec10dd30 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009cd0 tx=0x7f71e402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 -- 172.21.15.32:0/1573026062 shutdown_connections 2024-04-16T05:45:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 --2- 172.21.15.32:0/1573026062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101bf0 0x7f71ec10dd30 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 --2- 172.21.15.32:0/1573026062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ec101250 0x7f71ec1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 --2- 172.21.15.32:0/1573026062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71ec1070e0 0x7f71ec1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 -- 172.21.15.32:0/1573026062 >> 172.21.15.32:0/1573026062 conn(0x7f71ec076d00 msgr2=0x7f71ec077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 -- 172.21.15.32:0/1573026062 shutdown_connections 2024-04-16T05:45:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.845+0000 7f71f189a640 1 -- 172.21.15.32:0/1573026062 wait complete. 2024-04-16T05:45:57.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71f189a640 1 Processor -- start 2024-04-16T05:45:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71f189a640 1 -- start start 2024-04-16T05:45:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71f189a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101250 0x7f71ec19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71f189a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71ec101bf0 0x7f71ec19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71f189a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ec1070e0 0x7f71ec1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71f189a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec112460 con 0x7f71ec101250 2024-04-16T05:45:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71f189a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec1122e0 con 0x7f71ec1070e0 2024-04-16T05:45:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71f189a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec1125e0 con 0x7f71ec101bf0 2024-04-16T05:45:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101250 0x7f71ec19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101250 0x7f71ec19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48456/0 (socket says 172.21.15.32:48456) 2024-04-16T05:45:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 -- 172.21.15.32:0/2362031789 learned_addr learned my addr 172.21.15.32:0/2362031789 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71e3fff640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71ec101bf0 0x7f71ec19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eb7fe640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ec1070e0 0x7f71ec1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 -- 172.21.15.32:0/2362031789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71ec101bf0 msgr2=0x7f71ec19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71ec101bf0 0x7f71ec19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 -- 172.21.15.32:0/2362031789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ec1070e0 msgr2=0x7f71ec1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ec1070e0 0x7f71ec1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:57.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 -- 172.21.15.32:0/2362031789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec198f60 con 0x7f71ec101250 2024-04-16T05:45:57.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eb7fe640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ec1070e0 0x7f71ec1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:57.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.846+0000 7f71eaffd640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101250 0x7f71ec19e5e0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00d770 tx=0x7f71dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:57.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.847+0000 7f71f0898640 1 -- 172.21.15.32:0/2362031789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc017070 con 0x7f71ec101250 2024-04-16T05:45:57.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.847+0000 7f71f0898640 1 -- 172.21.15.32:0/2362031789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71dc005500 con 0x7f71ec101250 2024-04-16T05:45:57.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.847+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec199250 con 0x7f71ec101250 2024-04-16T05:45:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.847+0000 7f71f0898640 1 -- 172.21.15.32:0/2362031789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc004e00 con 0x7f71ec101250 2024-04-16T05:45:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.847+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec1a5560 con 0x7f71ec101250 2024-04-16T05:45:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.848+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec069e50 con 0x7f71ec101250 2024-04-16T05:45:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.849+0000 7f71f0898640 1 -- 172.21.15.32:0/2362031789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71dc004fa0 con 0x7f71ec101250 2024-04-16T05:45:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.850+0000 7f71f0898640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f71d80a3d20 0x7f71d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.850+0000 7f71e3fff640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f71d80a3d20 0x7f71d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.850+0000 7f71f0898640 1 -- 172.21.15.32:0/2362031789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f71dc0d0720 con 0x7f71ec101250 2024-04-16T05:45:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.850+0000 7f71e3fff640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f71d80a3d20 0x7f71d80a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f71ec199ef0 tx=0x7f71d4005f90 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:57.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:57.854+0000 7f71f0898640 1 -- 172.21.15.32:0/2362031789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71dc09c9d0 con 0x7f71ec101250 2024-04-16T05:45:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.040+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f71ec069270 con 0x7f71ec101250 2024-04-16T05:45:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.040+0000 7f71f0898640 1 -- 172.21.15.32:0/2362031789 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7f71dc09c2d0 con 0x7f71ec101250 2024-04-16T05:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-16T05:45:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f71d80a3d20 msgr2=0x7f71d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f71d80a3d20 0x7f71d80a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f71ec199ef0 tx=0x7f71d4005f90 comp rx=0 tx=0).stop 2024-04-16T05:45:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101250 msgr2=0x7f71ec19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101250 0x7f71ec19e5e0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00d770 tx=0x7f71dc00dc30 comp rx=0 tx=0).stop 2024-04-16T05:45:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 shutdown_connections 2024-04-16T05:45:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ec1070e0 0x7f71ec1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71ec101bf0 0x7f71ec19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f71d80a3d20 0x7f71d80a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 --2- 172.21.15.32:0/2362031789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ec101250 0x7f71ec19e5e0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.042+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 >> 172.21.15.32:0/2362031789 conn(0x7f71ec076d00 msgr2=0x7f71ec0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.043+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 shutdown_connections 2024-04-16T05:45:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.043+0000 7f71f189a640 1 -- 172.21.15.32:0/2362031789 wait complete. 2024-04-16T05:45:58.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:57 smithi032 ceph-mon[32351]: pgmap v341: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:58.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/4156534180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/828621963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:57 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2332250986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T05:45:58.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:57 smithi134 ceph-mon[36153]: pgmap v341: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:57 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/4156534180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:57 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/828621963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:57 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2332250986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:57 smithi082 ceph-mon[36195]: pgmap v341: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:45:58.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:57 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/4156534180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:57 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/828621963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:57 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2332250986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/234409233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 msgr2=0x7fc4dc10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/234409233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 0x7fc4dc10dd30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009cd0 tx=0x7fc4d402c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/234409233 shutdown_connections 2024-04-16T05:45:58.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/234409233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 0x7fc4dc10dd30 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/234409233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4dc101250 0x7fc4dc1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/234409233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4dc1070e0 0x7fc4dc1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/234409233 >> 172.21.15.32:0/234409233 conn(0x7fc4dc076d00 msgr2=0x7fc4dc077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:58.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/234409233 shutdown_connections 2024-04-16T05:45:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/234409233 wait complete. 2024-04-16T05:45:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.161+0000 7fc4e1a7e640 1 Processor -- start 2024-04-16T05:45:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4e1a7e640 1 -- start start 2024-04-16T05:45:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4e1a7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4dc101250 0x7fc4dc102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4e1a7e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 0x7fc4dc103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4e1a7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4dc1070e0 0x7fc4dc103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4e1a7e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1112e0 con 0x7fc4dc101bf0 2024-04-16T05:45:58.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4e1a7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc111160 con 0x7fc4dc1070e0 2024-04-16T05:45:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4e1a7e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc111460 con 0x7fc4dc101250 2024-04-16T05:45:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 0x7fc4dc103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 0x7fc4dc103140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48484/0 (socket says 172.21.15.32:48484) 2024-04-16T05:45:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 -- 172.21.15.32:0/437198531 learned_addr learned my addr 172.21.15.32:0/437198531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4db7fe640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4dc101250 0x7fc4dc102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4dbfff640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4dc1070e0 0x7fc4dc103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:58.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 -- 172.21.15.32:0/437198531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4dc101250 msgr2=0x7fc4dc102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4dc101250 0x7fc4dc102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 -- 172.21.15.32:0/437198531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4dc1070e0 msgr2=0x7fc4dc103680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4dc1070e0 0x7fc4dc103680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 -- 172.21.15.32:0/437198531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc1a6d10 con 0x7fc4dc101bf0 2024-04-16T05:45:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4dbfff640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4dc1070e0 0x7fc4dc103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.162+0000 7fc4daffd640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 0x7fc4dc103140 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000d770 tx=0x7fc4d000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.163+0000 7fc4d8ff9640 1 -- 172.21.15.32:0/437198531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0017070 con 0x7fc4dc101bf0 2024-04-16T05:45:58.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.163+0000 7fc4d8ff9640 1 -- 172.21.15.32:0/437198531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d0009820 con 0x7fc4dc101bf0 2024-04-16T05:45:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.163+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4dc1a6fa0 con 0x7fc4dc101bf0 2024-04-16T05:45:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.163+0000 7fc4d8ff9640 1 -- 172.21.15.32:0/437198531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0004cd0 con 0x7fc4dc101bf0 2024-04-16T05:45:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.163+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4dc1a7470 con 0x7fc4dc101bf0 2024-04-16T05:45:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.164+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4dc102640 con 0x7fc4dc101bf0 2024-04-16T05:45:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.165+0000 7fc4d8ff9640 1 -- 172.21.15.32:0/437198531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc4d00099d0 con 0x7fc4dc101bf0 2024-04-16T05:45:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.165+0000 7fc4d8ff9640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4c40a3d50 0x7fc4c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.166+0000 7fc4d8ff9640 1 -- 172.21.15.32:0/437198531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7fc4d00d0590 con 0x7fc4dc101bf0 2024-04-16T05:45:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.166+0000 7fc4db7fe640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4c40a3d50 0x7fc4c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.166+0000 7fc4db7fe640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4c40a3d50 0x7fc4c40a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00ab60 tx=0x7fc4cc00a2d0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:58.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.170+0000 7fc4d8ff9640 1 -- 172.21.15.32:0/437198531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4d009c8a0 con 0x7fc4dc101bf0 2024-04-16T05:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.369+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4dc1a7770 con 0x7fc4dc101bf0 2024-04-16T05:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.369+0000 7fc4d8ff9640 1 -- 172.21.15.32:0/437198531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+943 (secure 0 0 0) 0x7fc4d009c1a0 con 0x7fc4dc101bf0 2024-04-16T05:45:58.368 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1356969979 2024-04-17T05:34:56.002667+0000 2024-04-16T05:45:58.368 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2875882657 2024-04-17T05:34:56.002667+0000 2024-04-16T05:45:58.368 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2875882657 2024-04-17T05:34:56.002667+0000 2024-04-16T05:45:58.368 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/558122612 2024-04-17T05:34:56.002667+0000 2024-04-16T05:45:58.368 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2362160984 2024-04-17T05:34:56.002667+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2557553342 2024-04-17T05:25:32.817074+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1590295441 2024-04-17T05:25:32.817074+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1590295441 2024-04-17T05:25:32.817074+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2929016634 2024-04-17T05:25:32.817074+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/26695350 2024-04-17T05:25:04.672077+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3665169244 2024-04-17T05:25:04.672077+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1729232468 2024-04-17T05:34:56.002667+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/3517252554 2024-04-17T05:25:04.672077+0000 2024-04-16T05:45:58.369 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1771946956 2024-04-17T05:25:04.672077+0000 2024-04-16T05:45:58.370 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/3517252554 2024-04-17T05:25:04.672077+0000 2024-04-16T05:45:58.370 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1006904605 2024-04-17T05:25:32.817074+0000 2024-04-16T05:45:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:listed 16 entries 2024-04-16T05:45:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4c40a3d50 msgr2=0x7fc4c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4c40a3d50 0x7fc4c40a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00ab60 tx=0x7fc4cc00a2d0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 msgr2=0x7fc4dc103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 0x7fc4dc103140 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000d770 tx=0x7fc4d000db70 comp rx=0 tx=0).stop 2024-04-16T05:45:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 shutdown_connections 2024-04-16T05:45:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4dc1070e0 0x7fc4dc103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc4c40a3d50 0x7fc4c40a6210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4dc101bf0 0x7fc4dc103140 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 --2- 172.21.15.32:0/437198531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4dc101250 0x7fc4dc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.371+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 >> 172.21.15.32:0/437198531 conn(0x7fc4dc076d00 msgr2=0x7fc4dc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.372+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 shutdown_connections 2024-04-16T05:45:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.372+0000 7fc4e1a7e640 1 -- 172.21.15.32:0/437198531 wait complete. 2024-04-16T05:45:58.381 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-16T05:45:58.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:45:58.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-16T05:45:58.413 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:45:58.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:45:58.414 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-16T05:45:58.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:45:58.468 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-16T05:45:58.523 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-16T05:45:58.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:45:58.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T05:45:58.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:45:58.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-16T05:45:58.694 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-16T05:45:58.694 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-16T05:45:58.694 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T05:45:58.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057144,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498928k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1700 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249460k,nr_inodes=812365,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/151e394c32b678b770367ca0ce9c11eb06186b0555f13061f3e035e9b1975a93/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/151e394c32b678b770367ca0ce9c11eb06186b0555f13061f3e035e9b1975a93/diff,workdir=/var/lib/containers/storage/overlay/151e394c32b678b770367ca0ce9c11eb06186b0555f13061f3e035e9b1975a93/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fe617c3f11e0f43ba4a41c0dfed02c9e1094eea0328988af7a2dbfcea7272f90/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/fe617c3f11e0f43ba4a41c0dfed02c9e1094eea0328988af7a2dbfcea7272f90/diff,workdir=/var/lib/containers/storage/overlay/fe617c3f11e0f43ba4a41c0dfed02c9e1094eea0328988af7a2dbfcea7272f90/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249460k,nr_inodes=812365,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/d6a10c2c46636c58162d3b56b65696338ea5afcdcb6afde8d998c4c9f9247ab2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/d6a10c2c46636c58162d3b56b65696338ea5afcdcb6afde8d998c4c9f9247ab2/diff,workdir=/var/lib/containers/storage/overlay/d6a10c2c46636c58162d3b56b65696338ea5afcdcb6afde8d998c4c9f9247ab2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/097b3c8986d8b3628ab5f27a14306e3021921f34009af5beebc5b1172c747252/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/097b3c8986d8b3628ab5f27a14306e3021921f34009af5beebc5b1172c747252/diff,workdir=/var/lib/containers/storage/overlay/097b3c8986d8b3628ab5f27a14306e3021921f34009af5beebc5b1172c747252/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7836d69616b1ffb19045f98c7b0a58d85a59717fd76a1f94a8f2c0c26acf93ed/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/7836d69616b1ffb19045f98c7b0a58d85a59717fd76a1f94a8f2c0c26acf93ed/diff,workdir=/var/lib/containers/storage/overlay/7836d69616b1ffb19045f98c7b0a58d85a59717fd76a1f94a8f2c0c26acf93ed/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/701ccdb50ea0b6be47e143bc81dd57041f5c17d116339c3bd208a0a43255f245/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/701ccdb50ea0b6be47e143bc81dd57041f5c17d116339c3bd208a0a43255f245/diff,workdir=/var/lib/containers/storage/overlay/701ccdb50ea0b6be47e143bc81dd57041f5c17d116339c3bd208a0a43255f245/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b9576f07d5134488d6e6d231d3af13c56e98f83ed10aacded8a8772584a17a12/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/b9576f07d5134488d6e6d231d3af13c56e98f83ed10aacded8a8772584a17a12/diff,workdir=/var/lib/containers/storage/overlay/b9576f07d5134488d6e6d231d3af13c56e98f83ed10aacded8a8772584a17a12/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1361fa01d9936332082f14acaacff0f63828520b4fca8d1f7011aa203e2120c5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/1361fa01d9936332082f14acaacff0f63828520b4fca8d1f7011aa203e2120c5/diff,workdir=/var/lib/containers/storage/overlay/1361fa01d9936332082f14acaacff0f63828520b4fca8d1f7011aa203e2120c5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/feee35ca6436607927759f05875f131a64d9073d08f4f309d31acddfa842a737/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/feee35ca6436607927759f05875f131a64d9073d08f4f309d31acddfa842a737/diff,workdir=/var/lib/containers/storage/overlay/feee35ca6436607927759f05875f131a64d9073d08f4f309d31acddfa842a737/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a4acdd463a4189a4cfa3e354cd7b9dccc39d6d1ac0b171fc5e2ea1ada6588dee/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LUB6AMZH4K6KLCJCWXJZLON6NU:/var/lib/containers/storage/overlay/l/LCZAILHEN2CTJUR3TTBQWDCTIR,upperdir=/var/lib/containers/storage/overlay/a4acdd463a4189a4cfa3e354cd7b9dccc39d6d1ac0b171fc5e2ea1ada6588dee/diff,workdir=/var/lib/containers/storage/overlay/a4acdd463a4189a4cfa3e354cd7b9dccc39d6d1ac0b171fc5e2ea1ada6588dee/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498928k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-16T05:45:58.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:45:58.710 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-16T05:45:58.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T05:45:58.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-16T05:45:58.835 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-04-16T05:45:58.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:45:58.837 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-16T05:45:58.850 INFO:tasks.cephadm:Teardown begin 2024-04-16T05:45:58.851 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-16T05:45:58.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:45:58.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2362031789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:58.982 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:58 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/437198531' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:45:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 -- 172.21.15.32:0/3049591965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a74106220 msgr2=0x7f8a7410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/3049591965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a74106220 0x7f8a7410ddf0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009ca0 tx=0x7f8a6c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 -- 172.21.15.32:0/3049591965 shutdown_connections 2024-04-16T05:45:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/3049591965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a74106220 0x7f8a7410ddf0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/3049591965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a74101b00 0x7f8a74105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/3049591965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74101150 0x7f8a74101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 -- 172.21.15.32:0/3049591965 >> 172.21.15.32:0/3049591965 conn(0x7f8a74078000 msgr2=0x7f8a74078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 -- 172.21.15.32:0/3049591965 shutdown_connections 2024-04-16T05:45:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 -- 172.21.15.32:0/3049591965 wait complete. 2024-04-16T05:45:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 Processor -- start 2024-04-16T05:45:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.988+0000 7f8a7b64a640 1 -- start start 2024-04-16T05:45:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a7b64a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74101150 0x7f8a741996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a7b64a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a74106220 0x7f8a74199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a7b64a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7419a1a0 0x7f8a741a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a7b64a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74111330 con 0x7f8a7419a1a0 2024-04-16T05:45:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a7b64a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741111b0 con 0x7f8a74101150 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a7b64a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741114b0 con 0x7f8a74106220 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7419a1a0 0x7f8a741a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7419a1a0 0x7f8a741a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37376/0 (socket says 172.21.15.32:37376) 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 -- 172.21.15.32:0/2502378406 learned_addr learned my addr 172.21.15.32:0/2502378406 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a793e6640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74101150 0x7f8a741996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a78be5640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a74106220 0x7f8a74199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 -- 172.21.15.32:0/2502378406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a74106220 msgr2=0x7f8a74199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a74106220 0x7f8a74199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 -- 172.21.15.32:0/2502378406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74101150 msgr2=0x7f8a741996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74101150 0x7f8a741996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 -- 172.21.15.32:0/2502378406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a741a4f20 con 0x7f8a7419a1a0 2024-04-16T05:45:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a78be5640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a74106220 0x7f8a74199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a79be7640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7419a1a0 0x7f8a741a4750 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c02c700 tx=0x7f8a6c035b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a6a7fc640 1 -- 172.21.15.32:0/2502378406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c039040 con 0x7f8a7419a1a0 2024-04-16T05:45:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a6a7fc640 1 -- 172.21.15.32:0/2502378406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a6c044070 con 0x7f8a7419a1a0 2024-04-16T05:45:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a741a51b0 con 0x7f8a7419a1a0 2024-04-16T05:45:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.989+0000 7f8a6a7fc640 1 -- 172.21.15.32:0/2502378406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c004330 con 0x7f8a7419a1a0 2024-04-16T05:45:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.990+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741a56f0 con 0x7f8a7419a1a0 2024-04-16T05:45:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.990+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a74068e20 con 0x7f8a7419a1a0 2024-04-16T05:45:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.992+0000 7f8a6a7fc640 1 -- 172.21.15.32:0/2502378406 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a6c003970 con 0x7f8a7419a1a0 2024-04-16T05:45:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.992+0000 7f8a6a7fc640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8a600a3da0 0x7f8a600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.992+0000 7f8a793e6640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8a600a3da0 0x7f8a600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.992+0000 7f8a6a7fc640 1 -- 172.21.15.32:0/2502378406 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f8a6c0f35e0 con 0x7f8a7419a1a0 2024-04-16T05:45:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.993+0000 7f8a793e6640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8a600a3da0 0x7f8a600a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000abe0 tx=0x7f8a7000a2d0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:58.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:58.996+0000 7f8a6a7fc640 1 -- 172.21.15.32:0/2502378406 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a6c0bfa00 con 0x7f8a7419a1a0 2024-04-16T05:45:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2362031789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:59.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:58 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/437198531' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:45:59.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2362031789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T05:45:59.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:58 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/437198531' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T05:45:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.222+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a74101b00 con 0x7f8a7419a1a0 2024-04-16T05:45:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.223+0000 7f8a6a7fc640 1 -- 172.21.15.32:0/2502378406 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f8a6c0bf300 con 0x7f8a7419a1a0 2024-04-16T05:45:59.221 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:45:59.222 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:45:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8a600a3da0 msgr2=0x7f8a600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8a600a3da0 0x7f8a600a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000abe0 tx=0x7f8a7000a2d0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7419a1a0 msgr2=0x7f8a741a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7419a1a0 0x7f8a741a4750 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c02c700 tx=0x7f8a6c035b60 comp rx=0 tx=0).stop 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 shutdown_connections 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f8a600a3da0 0x7f8a600a6260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7419a1a0 0x7f8a741a4750 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a74106220 0x7f8a74199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 --2- 172.21.15.32:0/2502378406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74101150 0x7f8a741996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 >> 172.21.15.32:0/2502378406 conn(0x7f8a74078000 msgr2=0x7f8a740fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 shutdown_connections 2024-04-16T05:45:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.225+0000 7f8a7b64a640 1 -- 172.21.15.32:0/2502378406 wait complete. 2024-04-16T05:45:59.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T05:45:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 -- 172.21.15.32:0/1791791506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 msgr2=0x7f595810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 --2- 172.21.15.32:0/1791791506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 0x7f595810ea90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5950009ca0 tx=0x7f595002c180 comp rx=0 tx=0).stop 2024-04-16T05:45:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 -- 172.21.15.32:0/1791791506 shutdown_connections 2024-04-16T05:45:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 --2- 172.21.15.32:0/1791791506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 0x7f595810ea90 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 --2- 172.21.15.32:0/1791791506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958103c00 0x7f595810c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 --2- 172.21.15.32:0/1791791506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59581032e0 0x7f59581036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 -- 172.21.15.32:0/1791791506 >> 172.21.15.32:0/1791791506 conn(0x7f5958078060 msgr2=0x7f5958078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 -- 172.21.15.32:0/1791791506 shutdown_connections 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.342+0000 7f595f40d640 1 -- 172.21.15.32:0/1791791506 wait complete. 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595f40d640 1 Processor -- start 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595f40d640 1 -- start start 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595f40d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59581032e0 0x7f595819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595f40d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5958103c00 0x7f595819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595f40d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 0x7f5958198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595f40d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59581123d0 con 0x7f595810c620 2024-04-16T05:45:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595f40d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112250 con 0x7f59581032e0 2024-04-16T05:45:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595f40d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112550 con 0x7f5958103c00 2024-04-16T05:45:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d9aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 0x7f5958198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d9aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 0x7f5958198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37406/0 (socket says 172.21.15.32:37406) 2024-04-16T05:45:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d9aa640 1 -- 172.21.15.32:0/1329247937 learned_addr learned my addr 172.21.15.32:0/1329247937 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595c9a8640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5958103c00 0x7f595819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d9aa640 1 -- 172.21.15.32:0/1329247937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5958103c00 msgr2=0x7f595819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d9aa640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5958103c00 0x7f595819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d9aa640 1 -- 172.21.15.32:0/1329247937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59581032e0 msgr2=0x7f595819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d1a9640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59581032e0 0x7f595819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d9aa640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59581032e0 0x7f595819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d9aa640 1 -- 172.21.15.32:0/1329247937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5958198f30 con 0x7f595810c620 2024-04-16T05:45:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595c9a8640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5958103c00 0x7f595819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.343+0000 7f595d1a9640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59581032e0 0x7f595819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:45:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.344+0000 7f595d9aa640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 0x7f5958198670 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f595002c790 tx=0x7f5950035e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.344+0000 7f594e7fc640 1 -- 172.21.15.32:0/1329247937 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950039040 con 0x7f595810c620 2024-04-16T05:45:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.344+0000 7f594e7fc640 1 -- 172.21.15.32:0/1329247937 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5950044070 con 0x7f595810c620 2024-04-16T05:45:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.344+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59581991c0 con 0x7f595810c620 2024-04-16T05:45:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.344+0000 7f594e7fc640 1 -- 172.21.15.32:0/1329247937 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950032940 con 0x7f595810c620 2024-04-16T05:45:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.344+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581a5530 con 0x7f595810c620 2024-04-16T05:45:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.345+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595806ae30 con 0x7f595810c620 2024-04-16T05:45:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.347+0000 7f594e7fc640 1 -- 172.21.15.32:0/1329247937 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5950032ae0 con 0x7f595810c620 2024-04-16T05:45:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.347+0000 7f594e7fc640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f59440a3da0 0x7f59440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.348+0000 7f595d1a9640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f59440a3da0 0x7f59440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.348+0000 7f594e7fc640 1 -- 172.21.15.32:0/1329247937 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f59500f36a0 con 0x7f595810c620 2024-04-16T05:45:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.348+0000 7f595d1a9640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f59440a3da0 0x7f59440a6260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f59540058c0 tx=0x7f5954005d30 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.351+0000 7f594e7fc640 1 -- 172.21.15.32:0/1329247937 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59500bfa50 con 0x7f595810c620 2024-04-16T05:45:59.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.577+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5958069270 con 0x7f595810c620 2024-04-16T05:45:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.578+0000 7f594e7fc640 1 -- 172.21.15.32:0/1329247937 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11876 (secure 0 0 0) 0x7f5958069270 con 0x7f595810c620 2024-04-16T05:45:59.576 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:45:59.576 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3476798351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3476798351},{"type":"v1","addr":"172.21.15.32:6835","nonce":3476798351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3911772856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3911772856},{"type":"v1","addr":"172.21.15.32:6837","nonce":3911772856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1699429500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1699429500},{"type":"v1","addr":"172.21.15.32:6839","nonce":1699429500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2946647099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2946647099},{"type":"v1","addr":"172.21.15.32:6841","nonce":2946647099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1600676004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1600676004},{"type":"v1","addr":"172.21.15.134:6833","nonce":1600676004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1596016884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1596016884},{"type":"v1","addr":"172.21.15.82:6833","nonce":1596016884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T05:32:34.109354+0000","modified":"2024-04-16T05:34:54.496133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24461,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1431936644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1431936644},{"type":"v1","addr":"172.21.15.134:6835","nonce":1431936644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1367430466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1367430466},{"type":"v1","addr":"172.21.15.82:6835","nonce":1367430466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/2091760007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2091760007},{"type":"v1","addr":"172.21.15.134:6837","nonce":2091760007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6839/3919403886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3919403886},{"type":"v1","addr":"172.21.15.134:6839","nonce":3919403886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.82:6837/1413733447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1413733447},{"type":"v1","addr":"172.21.15.82:6837","nonce":1413733447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/2453097092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2453097092},{"type":"v1","addr":"172.21.15.82:6839","nonce":2453097092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24460,24461]},"id":1}]} 2024-04-16T05:45:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-04-16T05:45:59.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f59440a3da0 msgr2=0x7f59440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f59440a3da0 0x7f59440a6260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f59540058c0 tx=0x7f5954005d30 comp rx=0 tx=0).stop 2024-04-16T05:45:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 msgr2=0x7f5958198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 0x7f5958198670 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f595002c790 tx=0x7f5950035e00 comp rx=0 tx=0).stop 2024-04-16T05:45:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 shutdown_connections 2024-04-16T05:45:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f59440a3da0 0x7f59440a6260 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595810c620 0x7f5958198670 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5958103c00 0x7f595819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 --2- 172.21.15.32:0/1329247937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59581032e0 0x7f595819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 >> 172.21.15.32:0/1329247937 conn(0x7f5958078060 msgr2=0x7f59580ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:59.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 shutdown_connections 2024-04-16T05:45:59.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.580+0000 7f595f40d640 1 -- 172.21.15.32:0/1329247937 wait complete. 2024-04-16T05:45:59.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T05:45:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 -- 172.21.15.32:0/1140602208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4101140 msgr2=0x7f3dd4101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 --2- 172.21.15.32:0/1140602208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4101140 0x7f3dd4101520 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009900 tx=0x7f3dd002bc30 comp rx=0 tx=0).stop 2024-04-16T05:45:59.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 -- 172.21.15.32:0/1140602208 shutdown_connections 2024-04-16T05:45:59.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 --2- 172.21.15.32:0/1140602208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd4106210 0x7f3dd410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 --2- 172.21.15.32:0/1140602208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dd4101af0 0x7f3dd4105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 --2- 172.21.15.32:0/1140602208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4101140 0x7f3dd4101520 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 -- 172.21.15.32:0/1140602208 >> 172.21.15.32:0/1140602208 conn(0x7f3dd4077280 msgr2=0x7f3dd4077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 -- 172.21.15.32:0/1140602208 shutdown_connections 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.697+0000 7f3ddc688640 1 -- 172.21.15.32:0/1140602208 wait complete. 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddc688640 1 Processor -- start 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddc688640 1 -- start start 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddc688640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd4101140 0x7f3dd419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddc688640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dd4106210 0x7f3dd419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddc688640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4198660 0x7f3dd4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddc688640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd41123e0 con 0x7f3dd4198660 2024-04-16T05:45:59.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddc688640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112260 con 0x7f3dd4106210 2024-04-16T05:45:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddc688640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112560 con 0x7f3dd4101140 2024-04-16T05:45:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddac25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4198660 0x7f3dd4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddac25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4198660 0x7f3dd4198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37430/0 (socket says 172.21.15.32:37430) 2024-04-16T05:45:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3ddac25640 1 -- 172.21.15.32:0/74445194 learned_addr learned my addr 172.21.15.32:0/74445194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:45:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.698+0000 7f3dd9c23640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dd4106210 0x7f3dd419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3dda424640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd4101140 0x7f3dd419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3ddac25640 1 -- 172.21.15.32:0/74445194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd4101140 msgr2=0x7f3dd419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3ddac25640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd4101140 0x7f3dd419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3ddac25640 1 -- 172.21.15.32:0/74445194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dd4106210 msgr2=0x7f3dd419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3ddac25640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dd4106210 0x7f3dd419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3ddac25640 1 -- 172.21.15.32:0/74445194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd4199260 con 0x7f3dd4198660 2024-04-16T05:45:59.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3dd9c23640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dd4106210 0x7f3dd419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:45:59.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3ddac25640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4198660 0x7f3dd4198ae0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00eb10 tx=0x7f3dcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:59.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3dcb7fe640 1 -- 172.21.15.32:0/74445194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc004b20 con 0x7f3dd4198660 2024-04-16T05:45:59.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3dcb7fe640 1 -- 172.21.15.32:0/74445194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dcc004cc0 con 0x7f3dd4198660 2024-04-16T05:45:59.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd41a5340 con 0x7f3dd4198660 2024-04-16T05:45:59.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3dcb7fe640 1 -- 172.21.15.32:0/74445194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc010760 con 0x7f3dd4198660 2024-04-16T05:45:59.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.699+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd41a5740 con 0x7f3dd4198660 2024-04-16T05:45:59.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.700+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd4102610 con 0x7f3dd4198660 2024-04-16T05:45:59.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.701+0000 7f3dcb7fe640 1 -- 172.21.15.32:0/74445194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3dcc005000 con 0x7f3dd4198660 2024-04-16T05:45:59.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.702+0000 7f3dcb7fe640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3dc00a3d20 0x7f3dc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:45:59.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.702+0000 7f3dcb7fe640 1 -- 172.21.15.32:0/74445194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7f3dcc0d16d0 con 0x7f3dd4198660 2024-04-16T05:45:59.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.702+0000 7f3dda424640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3dc00a3d20 0x7f3dc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:45:59.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.702+0000 7f3dda424640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3dc00a3d20 0x7f3dc00a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0006010 tx=0x7f3dd0000bc0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:45:59.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.705+0000 7f3dcb7fe640 1 -- 172.21.15.32:0/74445194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3dcc09d9e0 con 0x7f3dd4198660 2024-04-16T05:45:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.894+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dd4069270 con 0x7f3dd4198660 2024-04-16T05:45:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.895+0000 7f3dcb7fe640 1 -- 172.21.15.32:0/74445194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25177 (secure 0 0 0) 0x7f3dcc09d2e0 con 0x7f3dd4198660 2024-04-16T05:45:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-16T05:45:59.894 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":100,"fsid":"578203a8-fbb1-11ee-bc8f-c7b262605968","created":"2024-04-16T05:24:37.692531+0000","modified":"2024-04-16T05:45:12.798785+0000","last_up_change":"2024-04-16T05:31:46.141467+0000","last_in_change":"2024-04-16T05:31:26.612308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T05:27:56.907609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T05:32:32.418635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T05:32:33.067994+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T05:32:36.906465+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3646a416-b8ca-4d84-9f20-cb3b2cb2273d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6803","nonce":4225872342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6805","nonce":4225872342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6809","nonce":4225872342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4225872342},{"type":"v1","addr":"172.21.15.32:6807","nonce":4225872342}]},"public_addr":"172.21.15.32:6803/4225872342","cluster_addr":"172.21.15.32:6805/4225872342","heartbeat_back_addr":"172.21.15.32:6809/4225872342","heartbeat_front_addr":"172.21.15.32:6807/4225872342","state":["exists","up"]},{"osd":1,"uuid":"0bc55145-c8d6-445c-aa03-7617bf25e77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6801","nonce":4042955203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6803","nonce":4042955203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6807","nonce":4042955203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":4042955203},{"type":"v1","addr":"172.21.15.82:6805","nonce":4042955203}]},"public_addr":"172.21.15.82:6801/4042955203","cluster_addr":"172.21.15.82:6803/4042955203","heartbeat_back_addr":"172.21.15.82:6807/4042955203","heartbeat_front_addr":"172.21.15.82:6805/4042955203","state":["exists","up"]},{"osd":2,"uuid":"b6916f0e-af2f-4e04-a6e1-656f11cc3423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6801","nonce":2834744702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6803","nonce":2834744702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6807","nonce":2834744702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2834744702},{"type":"v1","addr":"172.21.15.134:6805","nonce":2834744702}]},"public_addr":"172.21.15.134:6801/2834744702","cluster_addr":"172.21.15.134:6803/2834744702","heartbeat_back_addr":"172.21.15.134:6807/2834744702","heartbeat_front_addr":"172.21.15.134:6805/2834744702","state":["exists","up"]},{"osd":3,"uuid":"847ede2b-81d0-4993-895f-2e712666ea5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6811","nonce":3929583360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6813","nonce":3929583360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6817","nonce":3929583360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3929583360},{"type":"v1","addr":"172.21.15.32:6815","nonce":3929583360}]},"public_addr":"172.21.15.32:6811/3929583360","cluster_addr":"172.21.15.32:6813/3929583360","heartbeat_back_addr":"172.21.15.32:6817/3929583360","heartbeat_front_addr":"172.21.15.32:6815/3929583360","state":["exists","up"]},{"osd":4,"uuid":"7f816e73-9c2a-4392-80fd-ad98adc90aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6809","nonce":3071889784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6811","nonce":3071889784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6815","nonce":3071889784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3071889784},{"type":"v1","addr":"172.21.15.82:6813","nonce":3071889784}]},"public_addr":"172.21.15.82:6809/3071889784","cluster_addr":"172.21.15.82:6811/3071889784","heartbeat_back_addr":"172.21.15.82:6815/3071889784","heartbeat_front_addr":"172.21.15.82:6813/3071889784","state":["exists","up"]},{"osd":5,"uuid":"49194b43-0fad-47b3-ad59-e6f24c8865c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6809","nonce":1189980878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6811","nonce":1189980878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6815","nonce":1189980878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1189980878},{"type":"v1","addr":"172.21.15.134:6813","nonce":1189980878}]},"public_addr":"172.21.15.134:6809/1189980878","cluster_addr":"172.21.15.134:6811/1189980878","heartbeat_back_addr":"172.21.15.134:6815/1189980878","heartbeat_front_addr":"172.21.15.134:6813/1189980878","state":["exists","up"]},{"osd":6,"uuid":"cc3c0af7-426d-4be6-8d1a-87854c3eb8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6819","nonce":1227886227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6821","nonce":1227886227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6825","nonce":1227886227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1227886227},{"type":"v1","addr":"172.21.15.32:6823","nonce":1227886227}]},"public_addr":"172.21.15.32:6819/1227886227","cluster_addr":"172.21.15.32:6821/1227886227","heartbeat_back_addr":"172.21.15.32:6825/1227886227","heartbeat_front_addr":"172.21.15.32:6823/1227886227","state":["exists","up"]},{"osd":7,"uuid":"5a3fe236-c01c-45f0-896a-f97bee5a4fb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6817","nonce":529517800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6819","nonce":529517800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6823","nonce":529517800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":529517800},{"type":"v1","addr":"172.21.15.82:6821","nonce":529517800}]},"public_addr":"172.21.15.82:6817/529517800","cluster_addr":"172.21.15.82:6819/529517800","heartbeat_back_addr":"172.21.15.82:6823/529517800","heartbeat_front_addr":"172.21.15.82:6821/529517800","state":["exists","up"]},{"osd":8,"uuid":"b3917ee4-9f4e-400e-be3e-0e0558dd8881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6817","nonce":2280284325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6819","nonce":2280284325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6823","nonce":2280284325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2280284325},{"type":"v1","addr":"172.21.15.134:6821","nonce":2280284325}]},"public_addr":"172.21.15.134:6817/2280284325","cluster_addr":"172.21.15.134:6819/2280284325","heartbeat_back_addr":"172.21.15.134:6823/2280284325","heartbeat_front_addr":"172.21.15.134:6821/2280284325","state":["exists","up"]},{"osd":9,"uuid":"981db02c-51a8-4779-bc37-1c111ef20bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6827","nonce":192041066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6829","nonce":192041066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6833","nonce":192041066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":192041066},{"type":"v1","addr":"172.21.15.32:6831","nonce":192041066}]},"public_addr":"172.21.15.32:6827/192041066","cluster_addr":"172.21.15.32:6829/192041066","heartbeat_back_addr":"172.21.15.32:6833/192041066","heartbeat_front_addr":"172.21.15.32:6831/192041066","state":["exists","up"]},{"osd":10,"uuid":"13a54f6a-923a-4ce0-b6fb-f5331706fe76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6825","nonce":4010044592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6827","nonce":4010044592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6831","nonce":4010044592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":4010044592},{"type":"v1","addr":"172.21.15.82:6829","nonce":4010044592}]},"public_addr":"172.21.15.82:6825/4010044592","cluster_addr":"172.21.15.82:6827/4010044592","heartbeat_back_addr":"172.21.15.82:6831/4010044592","heartbeat_front_addr":"172.21.15.82:6829/4010044592","state":["exists","up"]},{"osd":11,"uuid":"67e3e8a5-0d88-445e-b37a-d7cd3db91a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6825","nonce":1766870366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6827","nonce":1766870366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6831","nonce":1766870366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1766870366},{"type":"v1","addr":"172.21.15.134:6829","nonce":1766870366}]},"public_addr":"172.21.15.134:6825/1766870366","cluster_addr":"172.21.15.134:6827/1766870366","heartbeat_back_addr":"172.21.15.134:6831/1766870366","heartbeat_front_addr":"172.21.15.134:6829/1766870366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:07.243431+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:31.078711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:27:54.428761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:19.646497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:28:44.436198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:09.525135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:29:35.476538+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:00.631046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:26.077247+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:30:53.104534+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:18.980356+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T05:31:44.709198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1356969979":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6800/2875882657":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/558122612":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2362160984":"2024-04-17T05:34:56.002667+0000","172.21.15.32:0/2557553342":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6801/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:6800/1590295441":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/2929016634":"2024-04-17T05:25:32.817074+0000","172.21.15.32:0/26695350":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/3665169244":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1729232468":"2024-04-17T05:34:56.002667+0000","172.21.15.32:6801/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1771946956":"2024-04-17T05:25:04.672077+0000","172.21.15.32:6800/3517252554":"2024-04-17T05:25:04.672077+0000","172.21.15.32:0/1006904605":"2024-04-17T05:25:32.817074+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T05:45:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.897+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3dc00a3d20 msgr2=0x7f3dc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3dc00a3d20 0x7f3dc00a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0006010 tx=0x7f3dd0000bc0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4198660 msgr2=0x7f3dd4198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:45:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4198660 0x7f3dd4198ae0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00eb10 tx=0x7f3dcc00c2d0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 shutdown_connections 2024-04-16T05:45:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f3dc00a3d20 0x7f3dc00a61e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd4198660 0x7f3dd4198ae0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dd4106210 0x7f3dd419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 --2- 172.21.15.32:0/74445194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd4101140 0x7f3dd419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:45:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 >> 172.21.15.32:0/74445194 conn(0x7f3dd4077280 msgr2=0x7f3dd40fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:45:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 shutdown_connections 2024-04-16T05:45:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:45:59.898+0000 7f3ddc688640 1 -- 172.21.15.32:0/74445194 wait complete. 2024-04-16T05:45:59.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-16T05:46:00.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 -- 172.21.15.32:0/3286731354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c106210 msgr2=0x7fd32c10dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 --2- 172.21.15.32:0/3286731354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c106210 0x7fd32c10dde0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009ca0 tx=0x7fd31c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 -- 172.21.15.32:0/3286731354 shutdown_connections 2024-04-16T05:46:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 --2- 172.21.15.32:0/3286731354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c106210 0x7fd32c10dde0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 --2- 172.21.15.32:0/3286731354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c101af0 0x7fd32c105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 --2- 172.21.15.32:0/3286731354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c101140 0x7fd32c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 -- 172.21.15.32:0/3286731354 >> 172.21.15.32:0/3286731354 conn(0x7fd32c078000 msgr2=0x7fd32c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 -- 172.21.15.32:0/3286731354 shutdown_connections 2024-04-16T05:46:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 -- 172.21.15.32:0/3286731354 wait complete. 2024-04-16T05:46:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 Processor -- start 2024-04-16T05:46:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.016+0000 7fd333c55640 1 -- start start 2024-04-16T05:46:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd333c55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c101140 0x7fd32c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd333c55640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c106210 0x7fd32c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd333c55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c1986e0 0x7fd32c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd333c55640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1112b0 con 0x7fd32c101140 2024-04-16T05:46:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd333c55640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c111130 con 0x7fd32c106210 2024-04-16T05:46:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd333c55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c111430 con 0x7fd32c1986e0 2024-04-16T05:46:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c101140 0x7fd32c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c101140 0x7fd32c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37452/0 (socket says 172.21.15.32:37452) 2024-04-16T05:46:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 -- 172.21.15.32:0/2122222129 learned_addr learned my addr 172.21.15.32:0/2122222129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:46:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3321f2640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c1986e0 0x7fd32c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 -- 172.21.15.32:0/2122222129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c1986e0 msgr2=0x7fd32c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:00.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c1986e0 0x7fd32c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3311f0640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c106210 0x7fd32c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:00.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 -- 172.21.15.32:0/2122222129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c106210 msgr2=0x7fd32c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:00.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c106210 0x7fd32c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 -- 172.21.15.32:0/2122222129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c1992e0 con 0x7fd32c101140 2024-04-16T05:46:00.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3321f2640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c1986e0 0x7fd32c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T05:46:00.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd3319f1640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c101140 0x7fd32c19e620 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd32400c890 tx=0x7fd32400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:00.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd32affd640 1 -- 172.21.15.32:0/2122222129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324004220 con 0x7fd32c101140 2024-04-16T05:46:00.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd32affd640 1 -- 172.21.15.32:0/2122222129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3240043c0 con 0x7fd32c101140 2024-04-16T05:46:00.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c1a53c0 con 0x7fd32c101140 2024-04-16T05:46:00.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd32affd640 1 -- 172.21.15.32:0/2122222129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3240049e0 con 0x7fd32c101140 2024-04-16T05:46:00.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.017+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c1a5820 con 0x7fd32c101140 2024-04-16T05:46:00.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.018+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c102610 con 0x7fd32c101140 2024-04-16T05:46:00.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.020+0000 7fd32affd640 1 -- 172.21.15.32:0/2122222129 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd3240079b0 con 0x7fd32c101140 2024-04-16T05:46:00.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.020+0000 7fd32affd640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd3140a3da0 0x7fd3140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:00.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.020+0000 7fd3311f0640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd3140a3da0 0x7fd3140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.021+0000 7fd32affd640 1 -- 172.21.15.32:0/2122222129 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9438+0+0 (secure 0 0 0) 0x7fd3240d0c80 con 0x7fd32c101140 2024-04-16T05:46:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.021+0000 7fd3311f0640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd3140a3da0 0x7fd3140a6260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd32c199f30 tx=0x7fd320005be0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.024+0000 7fd32affd640 1 -- 172.21.15.32:0/2122222129 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd324014030 con 0x7fd32c101140 2024-04-16T05:46:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:59 smithi134 ceph-mon[36153]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:46:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:59 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2502378406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:00.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:45:59 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1329247937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:00.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:59 smithi082 ceph-mon[36195]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:46:00.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:59 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2502378406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:00.149 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:45:59 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1329247937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:00.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:59 smithi032 ceph-mon[32351]: pgmap v342: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T05:46:00.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:59 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2502378406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:00.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:45:59 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1329247937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T05:46:00.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.248+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd32c068e20 con 0x7fd32c101140 2024-04-16T05:46:00.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.251+0000 7fd32affd640 1 -- 172.21.15.32:0/2122222129 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7fd32409d010 con 0x7fd32c101140 2024-04-16T05:46:00.249 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-16T05:46:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd3140a3da0 msgr2=0x7fd3140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd3140a3da0 0x7fd3140a6260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd32c199f30 tx=0x7fd320005be0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c101140 msgr2=0x7fd32c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c101140 0x7fd32c19e620 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd32400c890 tx=0x7fd32400cd50 comp rx=0 tx=0).stop 2024-04-16T05:46:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 shutdown_connections 2024-04-16T05:46:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c1986e0 0x7fd32c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd32c106210 0x7fd32c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fd3140a3da0 0x7fd3140a6260 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 --2- 172.21.15.32:0/2122222129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd32c101140 0x7fd32c19e620 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 >> 172.21.15.32:0/2122222129 conn(0x7fd32c078000 msgr2=0x7fd32c0693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 shutdown_connections 2024-04-16T05:46:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.253+0000 7fd333c55640 1 -- 172.21.15.32:0/2122222129 wait complete. 2024-04-16T05:46:00.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-16T05:46:00.295 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b[66419]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:00.295 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h[66113]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:00.296 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e[65792]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.373+0000 7fb87cc50640 1 -- 172.21.15.32:0/547292533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878100650 msgr2=0x7fb878100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.373+0000 7fb87cc50640 1 --2- 172.21.15.32:0/547292533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878100650 0x7fb878100ad0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009930 tx=0x7fb86c02bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.373+0000 7fb87cc50640 1 -- 172.21.15.32:0/547292533 shutdown_connections 2024-04-16T05:46:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.373+0000 7fb87cc50640 1 --2- 172.21.15.32:0/547292533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878101010 0x7fb87810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.373+0000 7fb87cc50640 1 --2- 172.21.15.32:0/547292533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878100650 0x7fb878100ad0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.373+0000 7fb87cc50640 1 --2- 172.21.15.32:0/547292533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb878106660 0x7fb878106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.373+0000 7fb87cc50640 1 -- 172.21.15.32:0/547292533 >> 172.21.15.32:0/547292533 conn(0x7fb878077490 msgr2=0x7fb8780fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:00.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.373+0000 7fb87cc50640 1 -- 172.21.15.32:0/547292533 shutdown_connections 2024-04-16T05:46:00.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 -- 172.21.15.32:0/547292533 wait complete. 2024-04-16T05:46:00.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 Processor -- start 2024-04-16T05:46:00.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 -- start start 2024-04-16T05:46:00.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb878100650 0x7fb87819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:00.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878101010 0x7fb87819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878106660 0x7fb878197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb878111270 con 0x7fb878106660 2024-04-16T05:46:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781110f0 con 0x7fb878100650 2024-04-16T05:46:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.374+0000 7fb87cc50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781113f0 con 0x7fb878101010 2024-04-16T05:46:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878106660 0x7fb878197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878106660 0x7fb878197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37470/0 (socket says 172.21.15.32:37470) 2024-04-16T05:46:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 -- 172.21.15.32:0/426853023 learned_addr learned my addr 172.21.15.32:0/426853023 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:46:00.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb87659c640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb878100650 0x7fb87819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:00.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb875d9b640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878101010 0x7fb87819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:00.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 -- 172.21.15.32:0/426853023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878101010 msgr2=0x7fb87819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:00.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878101010 0x7fb87819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 -- 172.21.15.32:0/426853023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb878100650 msgr2=0x7fb87819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:00.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb878100650 0x7fb87819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:00.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 -- 172.21.15.32:0/426853023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb878197ef0 con 0x7fb878106660 2024-04-16T05:46:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb875d9b640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878101010 0x7fb87819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:46:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb876d9d640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878106660 0x7fb878197630 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb87000d950 tx=0x7fb87000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb8677fe640 1 -- 172.21.15.32:0/426853023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870017030 con 0x7fb878106660 2024-04-16T05:46:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb8677fe640 1 -- 172.21.15.32:0/426853023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb870004020 con 0x7fb878106660 2024-04-16T05:46:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb8677fe640 1 -- 172.21.15.32:0/426853023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870003c00 con 0x7fb878106660 2024-04-16T05:46:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8781981e0 con 0x7fb878106660 2024-04-16T05:46:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.375+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8781a44f0 con 0x7fb878106660 2024-04-16T05:46:00.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.376+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb878101d90 con 0x7fb878106660 2024-04-16T05:46:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.377+0000 7fb8677fe640 1 -- 172.21.15.32:0/426853023 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb87001a070 con 0x7fb878106660 2024-04-16T05:46:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.378+0000 7fb8677fe640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb8600a3d20 0x7fb8600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.378+0000 7fb87659c640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb8600a3d20 0x7fb8600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.378+0000 7fb8677fe640 1 -- 172.21.15.32:0/426853023 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9608+0+0 (secure 0 0 0) 0x7fb8700d0860 con 0x7fb878106660 2024-04-16T05:46:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.378+0000 7fb87659c640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb8600a3d20 0x7fb8600a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb868005fd0 tx=0x7fb868009680 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.381+0000 7fb8677fe640 1 -- 172.21.15.32:0/426853023 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb870021070 con 0x7fb878106660 2024-04-16T05:46:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:00.610+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb878069270 con 0x7fb878106660 2024-04-16T05:46:00.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f[67278]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:00.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f[67278]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:00.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mds[67282]: -- 172.21.15.134:0/3934480633 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f07742da40 con 0x55f077401000 2024-04-16T05:46:00.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f[67278]: starting mds.f at 2024-04-16T05:46:00.635 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i[67899]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:00.635 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i[67899]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:00.635 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mds[67903]: -- 172.21.15.134:0/2286786436 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56364d51da40 con 0x56364d4f0c00 2024-04-16T05:46:00.635 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i[67899]: starting mds.i at 2024-04-16T05:46:00.635 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-c[67587]: ignoring --setuser ceph since I am not root 2024-04-16T05:46:00.636 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-c[67587]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:00.636 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mds[67591]: -- 172.21.15.134:0/66733563 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5568d056ba40 con 0x5568d053e800 2024-04-16T05:46:00.636 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-c[67587]: starting mds.c at 2024-04-16T05:46:00.649 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b[66419]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:00.649 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mds[66423]: -- 172.21.15.82:0/3539861555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d4cfafa40 con 0x560d4cf83000 2024-04-16T05:46:00.650 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b[66419]: starting mds.b at 2024-04-16T05:46:00.650 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e[65792]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:00.650 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mds[65796]: -- 172.21.15.82:0/3147182777 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c73fa87a40 con 0x55c73fa5ac00 2024-04-16T05:46:00.650 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e[65792]: starting mds.e at 2024-04-16T05:46:00.650 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h[66113]: ignoring --setgroup ceph since I am not root 2024-04-16T05:46:00.651 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mds[66117]: -- 172.21.15.82:0/3874197478 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55adf0cd9a40 con 0x55adf0cac800 2024-04-16T05:46:00.651 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h[66113]: starting mds.h at 2024-04-16T05:46:00.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a[32347]: 2024-04-16T05:46:00.249+0000 7fdd76660700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T05:46:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/74445194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:46:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2122222129' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T05:46:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T05:46:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T05:46:01.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/2122222129' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T05:46:01.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T05:46:01.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T05:46:01.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:00 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/426853023' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:46:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/74445194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:46:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2122222129' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T05:46:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T05:46:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T05:46:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/2122222129' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T05:46:01.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T05:46:01.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T05:46:01.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:00 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/426853023' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:46:01.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.252+0000 7fb8677fe640 1 -- 172.21.15.32:0/426853023 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fb87009ca60 con 0x7fb878106660 2024-04-16T05:46:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb8600a3d20 msgr2=0x7fb8600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb8600a3d20 0x7fb8600a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb868005fd0 tx=0x7fb868009680 comp rx=0 tx=0).stop 2024-04-16T05:46:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878106660 msgr2=0x7fb878197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878106660 0x7fb878197630 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb87000d950 tx=0x7fb87000de10 comp rx=0 tx=0).stop 2024-04-16T05:46:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 shutdown_connections 2024-04-16T05:46:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fb8600a3d20 0x7fb8600a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb878106660 0x7fb878197630 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878101010 0x7fb87819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 --2- 172.21.15.32:0/426853023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb878100650 0x7fb87819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 >> 172.21.15.32:0/426853023 conn(0x7fb878077490 msgr2=0x7fb878103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.255+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 shutdown_connections 2024-04-16T05:46:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.256+0000 7fb87cc50640 1 -- 172.21.15.32:0/426853023 wait complete. 2024-04-16T05:46:01.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-16T05:46:01.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/74445194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T05:46:01.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: pgmap v343: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T05:46:01.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2122222129' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T05:46:01.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T05:46:01.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T05:46:01.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/2122222129' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T05:46:01.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T05:46:01.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T05:46:01.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:00 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/426853023' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T05:46:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 -- 172.21.15.32:0/807174419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103280 msgr2=0x7fa468103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/807174419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103280 0x7fa468103660 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa460009900 tx=0x7fa46002bd60 comp rx=0 tx=0).stop 2024-04-16T05:46:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 -- 172.21.15.32:0/807174419 shutdown_connections 2024-04-16T05:46:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/807174419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa46810c5c0 0x7fa46810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/807174419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa468103ba0 0x7fa46810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/807174419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103280 0x7fa468103660 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 -- 172.21.15.32:0/807174419 >> 172.21.15.32:0/807174419 conn(0x7fa468078000 msgr2=0x7fa468078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 -- 172.21.15.32:0/807174419 shutdown_connections 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 -- 172.21.15.32:0/807174419 wait complete. 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.387+0000 7fa46ccc5640 1 Processor -- start 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa46ccc5640 1 -- start start 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa46ccc5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa468103280 0x7fa46819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa46ccc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103ba0 0x7fa46819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa46ccc5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46810c5c0 0x7fa4681986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa46ccc5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681123d0 con 0x7fa468103ba0 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa46ccc5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa468112250 con 0x7fa468103280 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa46ccc5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa468112550 con 0x7fa46810c5c0 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103ba0 0x7fa46819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103ba0 0x7fa46819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37488/0 (socket says 172.21.15.32:37488) 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 -- 172.21.15.32:0/1389092280 learned_addr learned my addr 172.21.15.32:0/1389092280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa46659c640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa468103280 0x7fa46819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:01.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa466d9d640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46810c5c0 0x7fa4681986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 -- 172.21.15.32:0/1389092280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46810c5c0 msgr2=0x7fa4681986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46810c5c0 0x7fa4681986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 -- 172.21.15.32:0/1389092280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa468103280 msgr2=0x7fa46819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa468103280 0x7fa46819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 -- 172.21.15.32:0/1389092280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa468198fb0 con 0x7fa468103ba0 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.388+0000 7fa465d9b640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103ba0 0x7fa46819eb70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa45000c890 tx=0x7fa45000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.389+0000 7fa45f7fe640 1 -- 172.21.15.32:0/1389092280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450004220 con 0x7fa468103ba0 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.389+0000 7fa45f7fe640 1 -- 172.21.15.32:0/1389092280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4500043c0 con 0x7fa468103ba0 2024-04-16T05:46:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.389+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4681992a0 con 0x7fa468103ba0 2024-04-16T05:46:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.389+0000 7fa45f7fe640 1 -- 172.21.15.32:0/1389092280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4500049e0 con 0x7fa468103ba0 2024-04-16T05:46:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.389+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4681a55b0 con 0x7fa468103ba0 2024-04-16T05:46:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.390+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa468069990 con 0x7fa468103ba0 2024-04-16T05:46:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.391+0000 7fa45f7fe640 1 -- 172.21.15.32:0/1389092280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4500079b0 con 0x7fa468103ba0 2024-04-16T05:46:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.391+0000 7fa45f7fe640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4540a3d20 0x7fa4540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.391+0000 7fa46659c640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4540a3d20 0x7fa4540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.391+0000 7fa45f7fe640 1 -- 172.21.15.32:0/1389092280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9608+0+0 (secure 0 0 0) 0x7fa4500d0bc0 con 0x7fa468103ba0 2024-04-16T05:46:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.392+0000 7fa46659c640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4540a3d20 0x7fa4540a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa460000c00 tx=0x7fa4600058d0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:01.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.396+0000 7fa45f7fe640 1 -- 172.21.15.32:0/1389092280 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa450014030 con 0x7fa468103ba0 2024-04-16T05:46:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:01.583+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa468107740 con 0x7fa468103ba0 2024-04-16T05:46:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T05:46:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T05:46:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: Cluster is now healthy 2024-04-16T05:46:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/426853023' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:46:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6836/3315953743,v1:172.21.15.82:6839/3315953743] up:boot 2024-04-16T05:46:02.150 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.134:6837/3326384234,v1:172.21.15.134:6839/3326384234] up:boot 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6835/331744441,v1:172.21.15.82:6838/331744441] up:boot 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.82:6834/1651769357,v1:172.21.15.82:6837/1651769357] up:boot 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.134:6834/3987603629,v1:172.21.15.134:6835/3987603629] up:boot 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: mds.? [v2:172.21.15.134:6836/2856088097,v1:172.21.15.134:6838/2856088097] up:boot 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: fsmap 12 up:standby 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:46:02.151 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:46:02.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:46:02.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:46:02.152 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:01 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1389092280' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:02.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T05:46:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T05:46:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: Cluster is now healthy 2024-04-16T05:46:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/426853023' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:46:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6836/3315953743,v1:172.21.15.82:6839/3315953743] up:boot 2024-04-16T05:46:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.134:6837/3326384234,v1:172.21.15.134:6839/3326384234] up:boot 2024-04-16T05:46:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6835/331744441,v1:172.21.15.82:6838/331744441] up:boot 2024-04-16T05:46:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.82:6834/1651769357,v1:172.21.15.82:6837/1651769357] up:boot 2024-04-16T05:46:02.233 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.134:6834/3987603629,v1:172.21.15.134:6835/3987603629] up:boot 2024-04-16T05:46:02.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: mds.? [v2:172.21.15.134:6836/2856088097,v1:172.21.15.134:6838/2856088097] up:boot 2024-04-16T05:46:02.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: fsmap 12 up:standby 2024-04-16T05:46:02.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:46:02.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:46:02.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:46:02.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:46:02.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:46:02.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:46:02.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:01 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1389092280' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:02.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.254+0000 7fa45f7fe640 1 -- 172.21.15.32:0/1389092280 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7fa45009c7d0 con 0x7fa468103ba0 2024-04-16T05:46:02.252 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-16T05:46:02.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.257+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4540a3d20 msgr2=0x7fa4540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:02.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4540a3d20 0x7fa4540a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa460000c00 tx=0x7fa4600058d0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103ba0 msgr2=0x7fa46819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:02.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103ba0 0x7fa46819eb70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa45000c890 tx=0x7fa45000cd50 comp rx=0 tx=0).stop 2024-04-16T05:46:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 shutdown_connections 2024-04-16T05:46:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46810c5c0 0x7fa4681986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fa4540a3d20 0x7fa4540a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa468103ba0 0x7fa46819eb70 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 --2- 172.21.15.32:0/1389092280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa468103280 0x7fa46819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 >> 172.21.15.32:0/1389092280 conn(0x7fa468078000 msgr2=0x7fa4680fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 shutdown_connections 2024-04-16T05:46:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.258+0000 7fa46ccc5640 1 -- 172.21.15.32:0/1389092280 wait complete. 2024-04-16T05:46:02.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-16T05:46:02.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T05:46:02.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T05:46:02.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: Cluster is now healthy 2024-04-16T05:46:02.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/426853023' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T05:46:02.384 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6836/3315953743,v1:172.21.15.82:6839/3315953743] up:boot 2024-04-16T05:46:02.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.134:6837/3326384234,v1:172.21.15.134:6839/3326384234] up:boot 2024-04-16T05:46:02.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6835/331744441,v1:172.21.15.82:6838/331744441] up:boot 2024-04-16T05:46:02.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.82:6834/1651769357,v1:172.21.15.82:6837/1651769357] up:boot 2024-04-16T05:46:02.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.134:6834/3987603629,v1:172.21.15.134:6835/3987603629] up:boot 2024-04-16T05:46:02.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: mds.? [v2:172.21.15.134:6836/2856088097,v1:172.21.15.134:6838/2856088097] up:boot 2024-04-16T05:46:02.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: fsmap 12 up:standby 2024-04-16T05:46:02.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T05:46:02.386 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T05:46:02.386 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T05:46:02.386 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T05:46:02.386 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T05:46:02.386 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: from='mgr.15429 172.21.15.32:0/907790217' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T05:46:02.386 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:01 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1389092280' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 -- 172.21.15.32:0/2522514056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110102250 msgr2=0x7fc110102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 --2- 172.21.15.32:0/2522514056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110102250 0x7fc110102630 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc100009900 tx=0x7fc10002bc30 comp rx=0 tx=0).stop 2024-04-16T05:46:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 -- 172.21.15.32:0/2522514056 shutdown_connections 2024-04-16T05:46:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 --2- 172.21.15.32:0/2522514056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc110107320 0x7fc11010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 --2- 172.21.15.32:0/2522514056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110102c00 0x7fc110106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 --2- 172.21.15.32:0/2522514056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110102250 0x7fc110102630 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 -- 172.21.15.32:0/2522514056 >> 172.21.15.32:0/2522514056 conn(0x7fc110079400 msgr2=0x7fc110079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 -- 172.21.15.32:0/2522514056 shutdown_connections 2024-04-16T05:46:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 -- 172.21.15.32:0/2522514056 wait complete. 2024-04-16T05:46:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 Processor -- start 2024-04-16T05:46:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.393+0000 7fc116922640 1 -- start start 2024-04-16T05:46:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc116922640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc110102250 0x7fc11019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc116922640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110107320 0x7fc11019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc116922640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110198740 0x7fc110198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc116922640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101123e0 con 0x7fc110107320 2024-04-16T05:46:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc116922640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc110112260 con 0x7fc110198740 2024-04-16T05:46:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc116922640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc110112560 con 0x7fc110102250 2024-04-16T05:46:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110107320 0x7fc11019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110107320 0x7fc11019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37504/0 (socket says 172.21.15.32:37504) 2024-04-16T05:46:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 -- 172.21.15.32:0/3155355434 learned_addr learned my addr 172.21.15.32:0/3155355434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:46:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10ffff640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc110102250 0x7fc11019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc114ebf640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110198740 0x7fc110198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 -- 172.21.15.32:0/3155355434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc110102250 msgr2=0x7fc11019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc110102250 0x7fc11019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 -- 172.21.15.32:0/3155355434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110198740 msgr2=0x7fc110198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110198740 0x7fc110198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 -- 172.21.15.32:0/3155355434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc110199340 con 0x7fc110107320 2024-04-16T05:46:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc114ebf640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110198740 0x7fc110198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T05:46:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10f7fe640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110107320 0x7fc11019ebc0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc104005fd0 tx=0x7fc10400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.394+0000 7fc10d7fa640 1 -- 172.21.15.32:0/3155355434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104017070 con 0x7fc110107320 2024-04-16T05:46:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.395+0000 7fc10d7fa640 1 -- 172.21.15.32:0/3155355434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc104009c30 con 0x7fc110107320 2024-04-16T05:46:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.395+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1101a5420 con 0x7fc110107320 2024-04-16T05:46:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.395+0000 7fc10d7fa640 1 -- 172.21.15.32:0/3155355434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104010460 con 0x7fc110107320 2024-04-16T05:46:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.395+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1101a5820 con 0x7fc110107320 2024-04-16T05:46:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.395+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc110103720 con 0x7fc110107320 2024-04-16T05:46:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.397+0000 7fc10d7fa640 1 -- 172.21.15.32:0/3155355434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc104003e70 con 0x7fc110107320 2024-04-16T05:46:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.397+0000 7fc10d7fa640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc0f80a3d20 0x7fc0f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.397+0000 7fc10ffff640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc0f80a3d20 0x7fc0f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.397+0000 7fc10d7fa640 1 -- 172.21.15.32:0/3155355434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9112+0+0 (secure 0 0 0) 0x7fc1040d06e0 con 0x7fc110107320 2024-04-16T05:46:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.398+0000 7fc10ffff640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc0f80a3d20 0x7fc0f80a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc100006010 tx=0x7fc100005dd0 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.402+0000 7fc10d7fa640 1 -- 172.21.15.32:0/3155355434 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc104020030 con 0x7fc110107320 2024-04-16T05:46:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:02.590+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc11010a550 con 0x7fc110107320 2024-04-16T05:46:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.256+0000 7fc10d7fa640 1 -- 172.21.15.32:0/3155355434 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7fc10409cb30 con 0x7fc110107320 2024-04-16T05:46:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-16T05:46:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.260+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc0f80a3d20 msgr2=0x7fc0f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.260+0000 7fc116922640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc0f80a3d20 0x7fc0f80a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc100006010 tx=0x7fc100005dd0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.260+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110107320 msgr2=0x7fc11019ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.260+0000 7fc116922640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110107320 0x7fc11019ebc0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc104005fd0 tx=0x7fc10400dde0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.261+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 shutdown_connections 2024-04-16T05:46:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.261+0000 7fc116922640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc110198740 0x7fc110198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.261+0000 7fc116922640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7fc0f80a3d20 0x7fc0f80a61e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.261+0000 7fc116922640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110107320 0x7fc11019ebc0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.261+0000 7fc116922640 1 --2- 172.21.15.32:0/3155355434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc110102250 0x7fc11019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.261+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 >> 172.21.15.32:0/3155355434 conn(0x7fc110079400 msgr2=0x7fc110100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.261+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 shutdown_connections 2024-04-16T05:46:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.261+0000 7fc116922640 1 -- 172.21.15.32:0/3155355434 wait complete. 2024-04-16T05:46:03.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-16T05:46:03.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 -- 172.21.15.32:0/2633712625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988410e9b0 msgr2=0x7f9884110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:03.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 --2- 172.21.15.32:0/2633712625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988410e9b0 0x7f9884110da0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f987c009ca0 tx=0x7f987c02c0f0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 -- 172.21.15.32:0/2633712625 shutdown_connections 2024-04-16T05:46:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 --2- 172.21.15.32:0/2633712625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988410e9b0 0x7f9884110da0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 --2- 172.21.15.32:0/2633712625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9884101720 0x7f988410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 --2- 172.21.15.32:0/2633712625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9884100e00 0x7f98841011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 -- 172.21.15.32:0/2633712625 >> 172.21.15.32:0/2633712625 conn(0x7f9884078000 msgr2=0x7f9884078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 -- 172.21.15.32:0/2633712625 shutdown_connections 2024-04-16T05:46:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 -- 172.21.15.32:0/2633712625 wait complete. 2024-04-16T05:46:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 Processor -- start 2024-04-16T05:46:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.394+0000 7f988a9ca640 1 -- start start 2024-04-16T05:46:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f988a9ca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9884100e00 0x7f98841a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f988a9ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884101720 0x7f98841a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f988a9ca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988410e9b0 0x7f988419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f988a9ca640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9884114620 con 0x7f9884101720 2024-04-16T05:46:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f988a9ca640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98841144a0 con 0x7f988410e9b0 2024-04-16T05:46:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f988a9ca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98841147a0 con 0x7f9884100e00 2024-04-16T05:46:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884101720 0x7f98841a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884101720 0x7f98841a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37532/0 (socket says 172.21.15.32:37532) 2024-04-16T05:46:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 -- 172.21.15.32:0/56260866 learned_addr learned my addr 172.21.15.32:0/56260866 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-16T05:46:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f9888f67640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988410e9b0 0x7f988419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f9883fff640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9884100e00 0x7f98841a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 -- 172.21.15.32:0/56260866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9884100e00 msgr2=0x7f98841a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9884100e00 0x7f98841a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 -- 172.21.15.32:0/56260866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988410e9b0 msgr2=0x7f988419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988410e9b0 0x7f988419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 -- 172.21.15.32:0/56260866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988419b0a0 con 0x7f9884101720 2024-04-16T05:46:03.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f987bfff640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884101720 0x7f98841a0cb0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f986c00b4a0 tx=0x7f986c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:03.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.395+0000 7f9881ffb640 1 -- 172.21.15.32:0/56260866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c014070 con 0x7f9884101720 2024-04-16T05:46:03.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.396+0000 7f9881ffb640 1 -- 172.21.15.32:0/56260866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986c010af0 con 0x7f9884101720 2024-04-16T05:46:03.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.396+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988419b390 con 0x7f9884101720 2024-04-16T05:46:03.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.396+0000 7f9881ffb640 1 -- 172.21.15.32:0/56260866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c017d10 con 0x7f9884101720 2024-04-16T05:46:03.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.396+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98841a7630 con 0x7f9884101720 2024-04-16T05:46:03.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.396+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9884068e20 con 0x7f9884101720 2024-04-16T05:46:03.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.398+0000 7f9881ffb640 1 -- 172.21.15.32:0/56260866 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f986c017070 con 0x7f9884101720 2024-04-16T05:46:03.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.398+0000 7f9881ffb640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98700a3d20 0x7f98700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T05:46:03.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.398+0000 7f9881ffb640 1 -- 172.21.15.32:0/56260866 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8582+0+0 (secure 0 0 0) 0x7f986c0cfef0 con 0x7f9884101720 2024-04-16T05:46:03.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.398+0000 7f9883fff640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98700a3d20 0x7f98700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T05:46:03.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.399+0000 7f9883fff640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98700a3d20 0x7f98700a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9874005fd0 tx=0x7f9874005e40 comp rx=0 tx=0).ready entity=mgr.15429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T05:46:03.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.403+0000 7f9881ffb640 1 -- 172.21.15.32:0/56260866 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f986c09c660 con 0x7f9884101720 2024-04-16T05:46:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:03.589+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f988419be50 con 0x7f9884101720 2024-04-16T05:46:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:03 smithi134 ceph-mon[36153]: pgmap v345: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:46:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:03 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/1389092280' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:03 smithi134 ceph-mon[36153]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T05:46:03.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:03 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3155355434' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:03.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:03 smithi082 ceph-mon[36195]: pgmap v345: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:46:03.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:03 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/1389092280' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:03.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:03 smithi082 ceph-mon[36195]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T05:46:03.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:03 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3155355434' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:03.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:03 smithi032 ceph-mon[32351]: pgmap v345: 129 pgs: 129 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T05:46:03.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:03 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/1389092280' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:03.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:03 smithi032 ceph-mon[32351]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T05:46:03.733 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:03 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3155355434' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.269+0000 7f9881ffb640 1 -- 172.21.15.32:0/56260866 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) ==== 178+0+0 (secure 0 0 0) 0x7f988419be50 con 0x7f9884101720 2024-04-16T05:46:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data_ec' removed 2024-04-16T05:46:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98700a3d20 msgr2=0x7f98700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98700a3d20 0x7f98700a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9874005fd0 tx=0x7f9874005e40 comp rx=0 tx=0).stop 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884101720 msgr2=0x7f98841a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884101720 0x7f98841a0cb0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f986c00b4a0 tx=0x7f986c00b960 comp rx=0 tx=0).stop 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 shutdown_connections 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988410e9b0 0x7f988419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:6800/3496546430,v1:172.21.15.32:6801/3496546430] conn(0x7f98700a3d20 0x7f98700a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884101720 0x7f98841a0cb0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 --2- 172.21.15.32:0/56260866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9884100e00 0x7f98841a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 >> 172.21.15.32:0/56260866 conn(0x7f9884078000 msgr2=0x7f988410ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T05:46:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 shutdown_connections 2024-04-16T05:46:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-16T05:46:04.274+0000 7f988a9ca640 1 -- 172.21.15.32:0/56260866 wait complete. 2024-04-16T05:46:04.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:46:04.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:46:04.356 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:46:04.390 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-16T05:46:04.391 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T05:46:04.407 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T05:46:04.424 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T05:46:04.447 INFO:tasks.cephadm:Stopping all daemons... 2024-04-16T05:46:04.448 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-16T05:46:04.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.a 2024-04-16T05:46:04.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:04 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/3155355434' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:04.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:04 smithi134 ceph-mon[36153]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T05:46:04.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:04 smithi134 ceph-mon[36153]: from='client.? 172.21.15.32:0/56260866' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:04.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:04 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/3155355434' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:04.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:04 smithi082 ceph-mon[36195]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T05:46:04.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:04 smithi082 ceph-mon[36195]: from='client.? 172.21.15.32:0/56260866' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:04.732 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:04 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/3155355434' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T05:46:04.741 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:04 smithi032 ceph-mon[32351]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T05:46:04.742 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:04 smithi032 ceph-mon[32351]: from='client.? 172.21.15.32:0/56260866' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T05:46:04.742 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:04 smithi032 systemd[1]: Stopping Ceph mon.a for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:05.191 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a[32347]: 2024-04-16T05:46:04.766+0000 7fdd7be6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:05.192 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a[32347]: 2024-04-16T05:46:04.766+0000 7fdd7be6b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-16T05:46:05.192 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:04 smithi032 podman[102139]: 2024-04-16 05:46:04.79777788 +0000 UTC m=+0.146491400 container died 38b0b9542547dfa25b8186721c89cec5e63569a5ce8b972c57b9ed3f0f5db857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-16T05:46:05.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:05 smithi032 podman[102139]: 2024-04-16 05:46:05.284480159 +0000 UTC m=+0.633193674 container cleanup 38b0b9542547dfa25b8186721c89cec5e63569a5ce8b972c57b9ed3f0f5db857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-16T05:46:05.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Apr 16 05:46:05 smithi032 bash[102139]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-a 2024-04-16T05:46:05.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.a.service' 2024-04-16T05:46:05.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:05.710 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-16T05:46:05.711 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-16T05:46:05.711 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.b 2024-04-16T05:46:06.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:05 smithi082 systemd[1]: Stopping Ceph mon.b for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:06.367 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-b[36191]: 2024-04-16T05:46:06.008+0000 7fd78956e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:06.367 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-b[36191]: 2024-04-16T05:46:06.008+0000 7fd78956e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-16T05:46:06.367 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:06 smithi082 podman[71850]: 2024-04-16 05:46:06.080690304 +0000 UTC m=+0.186624138 container died 2e947f49936b2bbae14c5d572b62cc9fe1ead8440f03bb3a88de90833275cc25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-16T05:46:06.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:06 smithi082 podman[71850]: 2024-04-16 05:46:06.459418396 +0000 UTC m=+0.565352221 container cleanup 2e947f49936b2bbae14c5d572b62cc9fe1ead8440f03bb3a88de90833275cc25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-b, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:46:06.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Apr 16 05:46:06 smithi082 bash[71850]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-b 2024-04-16T05:46:06.916 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.b.service' 2024-04-16T05:46:06.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:06.966 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-16T05:46:06.966 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-16T05:46:06.966 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.c 2024-04-16T05:46:07.286 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:07 smithi134 systemd[1]: Stopping Ceph mon.c for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:07 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-c[36149]: 2024-04-16T05:46:07.286+0000 7f6cfd408700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:07 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-c[36149]: 2024-04-16T05:46:07.286+0000 7f6cfd408700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-16T05:46:07.634 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:07 smithi134 podman[73391]: 2024-04-16 05:46:07.471521782 +0000 UTC m=+0.302026396 container died 758f8f2d850159c8a5ee5eeaf5c7a8e33759c46a9216f50e757c925af3f6dd65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-16T05:46:07.968 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:07 smithi134 podman[73391]: 2024-04-16 05:46:07.864090465 +0000 UTC m=+0.694595077 container cleanup 758f8f2d850159c8a5ee5eeaf5c7a8e33759c46a9216f50e757c925af3f6dd65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:07.968 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:07 smithi134 bash[73391]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-c 2024-04-16T05:46:08.298 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 16 05:46:08 smithi134 podman[73405]: 2024-04-16 05:46:08.151088168 +0000 UTC m=+0.674564025 container remove 758f8f2d850159c8a5ee5eeaf5c7a8e33759c46a9216f50e757c925af3f6dd65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mon-c, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-16T05:46:08.318 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mon.c.service' 2024-04-16T05:46:08.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:08.362 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-16T05:46:08.362 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-16T05:46:08.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.x 2024-04-16T05:46:08.728 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:46:08 smithi032 systemd[1]: Stopping Ceph mgr.x for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:08.982 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:46:08 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x[32582]: 2024-04-16T05:46:08.775+0000 7f97017aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:08.982 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:46:08 smithi032 podman[102347]: 2024-04-16 05:46:08.807019153 +0000 UTC m=+0.246257576 container died 9e101b4f648f53e6765819ed0ac782505c8b1877e86d5324ab9d23df1ac84572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=) 2024-04-16T05:46:09.360 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:46:09 smithi032 podman[102347]: 2024-04-16 05:46:09.126830292 +0000 UTC m=+0.566068703 container cleanup 9e101b4f648f53e6765819ed0ac782505c8b1877e86d5324ab9d23df1ac84572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a87fcff6340baeedd43af126e793f3241b6400ea, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-16T05:46:09.360 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Apr 16 05:46:09 smithi032 bash[102347]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-x 2024-04-16T05:46:09.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.x.service' 2024-04-16T05:46:09.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:09.509 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-16T05:46:09.509 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-16T05:46:09.509 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.y 2024-04-16T05:46:09.833 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:46:09 smithi082 systemd[1]: Stopping Ceph mgr.y for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:10.112 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:46:09 smithi082 podman[71975]: 2024-04-16 05:46:09.835163677 +0000 UTC m=+0.139789448 container died d58fb8e1390f6ebf13c0c7544e86290b52b5b454048c459f3ea775a6dbcaace8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:10.399 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:46:10 smithi082 podman[71975]: 2024-04-16 05:46:10.214680443 +0000 UTC m=+0.519306211 container cleanup d58fb8e1390f6ebf13c0c7544e86290b52b5b454048c459f3ea775a6dbcaace8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:10.400 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Apr 16 05:46:10 smithi082 bash[71975]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-y 2024-04-16T05:46:10.635 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.y.service' 2024-04-16T05:46:10.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:10.670 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-16T05:46:10.671 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-16T05:46:10.671 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.z 2024-04-16T05:46:11.023 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:46:10 smithi134 systemd[1]: Stopping Ceph mgr.z for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:11.287 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:46:11 smithi134 podman[73522]: 2024-04-16 05:46:11.025620397 +0000 UTC m=+0.158538911 container died cea3fb9e5da54b24b661f7a6e960dd290f72e8128343e40595d42cdfa7de1913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:11.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:46:11 smithi134 podman[73522]: 2024-04-16 05:46:11.389357609 +0000 UTC m=+0.522276122 container cleanup cea3fb9e5da54b24b661f7a6e960dd290f72e8128343e40595d42cdfa7de1913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-16T05:46:11.634 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 16 05:46:11 smithi134 bash[73522]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mgr-z 2024-04-16T05:46:11.850 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mgr.z.service' 2024-04-16T05:46:11.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:11.887 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-16T05:46:11.887 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-16T05:46:11.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.0 2024-04-16T05:46:12.221 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:11 smithi032 systemd[1]: Stopping Ceph osd.0 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:12.483 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:12 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0[47110]: 2024-04-16T05:46:12.222+0000 7f4ab71db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:12.483 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:12 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0[47110]: 2024-04-16T05:46:12.222+0000 7f4ab71db700 -1 osd.0 104 *** Got signal Terminated *** 2024-04-16T05:46:12.483 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:12 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0[47110]: 2024-04-16T05:46:12.222+0000 7f4ab71db700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:46:17.889 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:17 smithi032 podman[102468]: 2024-04-16 05:46:17.598187649 +0000 UTC m=+5.503169384 container died 71aa7cdf3208b05878c76b6664aca61d9ca684a871e7b138d0978084bc3acdfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:46:18.232 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:18 smithi032 podman[102468]: 2024-04-16 05:46:18.007602021 +0000 UTC m=+5.912583752 container cleanup 71aa7cdf3208b05878c76b6664aca61d9ca684a871e7b138d0978084bc3acdfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:46:18.232 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:18 smithi032 bash[102468]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0 2024-04-16T05:46:18.482 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:18 smithi032 podman[102485]: 2024-04-16 05:46:18.234339714 +0000 UTC m=+0.638656814 container remove 71aa7cdf3208b05878c76b6664aca61d9ca684a871e7b138d0978084bc3acdfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-16T05:46:18.969 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:18 smithi032 podman[102575]: 2024-04-16 05:46:18.588931835 +0000 UTC m=+0.017999537 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:46:18.970 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:18 smithi032 podman[102575]: 2024-04-16 05:46:18.71043839 +0000 UTC m=+0.139506119 container create 6efc888e652435155e6350b1649acab361d11d3fef4a94584e677bc4230dabb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:19.232 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:18 smithi032 podman[102575]: 2024-04-16 05:46:18.971615742 +0000 UTC m=+0.400683429 container init 6efc888e652435155e6350b1649acab361d11d3fef4a94584e677bc4230dabb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD) 2024-04-16T05:46:19.233 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:18 smithi032 podman[102575]: 2024-04-16 05:46:18.976119483 +0000 UTC m=+0.405187173 container start 6efc888e652435155e6350b1649acab361d11d3fef4a94584e677bc4230dabb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:19.233 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:19 smithi032 podman[102575]: 2024-04-16 05:46:19.080272743 +0000 UTC m=+0.509340453 container attach 6efc888e652435155e6350b1649acab361d11d3fef4a94584e677bc4230dabb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:19.565 INFO:journalctl@ceph.osd.0.smithi032.stdout:Apr 16 05:46:19 smithi032 podman[102575]: 2024-04-16 05:46:19.277608652 +0000 UTC m=+0.706676347 container died 6efc888e652435155e6350b1649acab361d11d3fef4a94584e677bc4230dabb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:46:19.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.0.service' 2024-04-16T05:46:20.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:20.006 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-16T05:46:20.006 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-16T05:46:20.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.1 2024-04-16T05:46:20.293 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:20 smithi082 systemd[1]: Stopping Ceph osd.1 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:20.650 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1[40796]: 2024-04-16T05:46:20.336+0000 7fc62994d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:20.650 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1[40796]: 2024-04-16T05:46:20.336+0000 7fc62994d700 -1 osd.1 104 *** Got signal Terminated *** 2024-04-16T05:46:20.650 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1[40796]: 2024-04-16T05:46:20.336+0000 7fc62994d700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:46:25.975 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:25 smithi082 podman[72102]: 2024-04-16 05:46:25.69061764 +0000 UTC m=+5.484086907 container died 80dfc408f9f09beeecbdbbba5983139df80e98182ff374d3e7d92db57867569b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=) 2024-04-16T05:46:26.367 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:26 smithi082 podman[72102]: 2024-04-16 05:46:26.067983268 +0000 UTC m=+5.861452527 container cleanup 80dfc408f9f09beeecbdbbba5983139df80e98182ff374d3e7d92db57867569b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-16T05:46:26.367 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:26 smithi082 bash[72102]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1 2024-04-16T05:46:26.649 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:26 smithi082 podman[72130]: 2024-04-16 05:46:26.369755582 +0000 UTC m=+0.676410553 container remove 80dfc408f9f09beeecbdbbba5983139df80e98182ff374d3e7d92db57867569b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-04-16T05:46:26.937 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:26 smithi082 podman[72224]: 2024-04-16 05:46:26.591404472 +0000 UTC m=+0.018006631 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:46:26.937 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:26 smithi082 podman[72224]: 2024-04-16 05:46:26.712340977 +0000 UTC m=+0.138943116 container create 7f5ad10eadb32234fd1dc1c7f50e029b09c9189c4ed9af1528a91d7db9710403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-16T05:46:26.937 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:26 smithi082 podman[72224]: 2024-04-16 05:46:26.939915723 +0000 UTC m=+0.366517874 container init 7f5ad10eadb32234fd1dc1c7f50e029b09c9189c4ed9af1528a91d7db9710403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:27.230 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:26 smithi082 podman[72224]: 2024-04-16 05:46:26.943451025 +0000 UTC m=+0.370053173 container start 7f5ad10eadb32234fd1dc1c7f50e029b09c9189c4ed9af1528a91d7db9710403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:27.230 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:27 smithi082 podman[72224]: 2024-04-16 05:46:27.039995922 +0000 UTC m=+0.466598075 container attach 7f5ad10eadb32234fd1dc1c7f50e029b09c9189c4ed9af1528a91d7db9710403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-16T05:46:27.582 INFO:journalctl@ceph.osd.1.smithi082.stdout:Apr 16 05:46:27 smithi082 podman[72224]: 2024-04-16 05:46:27.232794278 +0000 UTC m=+0.659396442 container died 7f5ad10eadb32234fd1dc1c7f50e029b09c9189c4ed9af1528a91d7db9710403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-1-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:46:27.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.1.service' 2024-04-16T05:46:27.960 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:27.960 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-16T05:46:27.960 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-16T05:46:27.961 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.2 2024-04-16T05:46:28.258 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:28 smithi134 systemd[1]: Stopping Ceph osd.2 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:28.634 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2[41312]: 2024-04-16T05:46:28.259+0000 7fd0067e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:28.634 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2[41312]: 2024-04-16T05:46:28.259+0000 7fd0067e3700 -1 osd.2 104 *** Got signal Terminated *** 2024-04-16T05:46:28.634 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2[41312]: 2024-04-16T05:46:28.259+0000 7fd0067e3700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:46:34.066 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:33 smithi134 podman[73648]: 2024-04-16 05:46:33.748773178 +0000 UTC m=+5.599384131 container died 5b4986669f3e46444725921876554bd121cdea8392c0875768fa2d56a337d264 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2, ceph=True, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:46:34.384 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:34 smithi134 podman[73648]: 2024-04-16 05:46:34.068412594 +0000 UTC m=+5.919023569 container cleanup 5b4986669f3e46444725921876554bd121cdea8392c0875768fa2d56a337d264 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5) 2024-04-16T05:46:34.384 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:34 smithi134 bash[73648]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2 2024-04-16T05:46:34.384 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:34 smithi134 podman[73665]: 2024-04-16 05:46:34.261378491 +0000 UTC m=+0.513626979 container remove 5b4986669f3e46444725921876554bd121cdea8392c0875768fa2d56a337d264 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:34.998 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:34 smithi134 podman[73761]: 2024-04-16 05:46:34.575874039 +0000 UTC m=+0.017628491 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:46:34.998 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:34 smithi134 podman[73761]: 2024-04-16 05:46:34.679664844 +0000 UTC m=+0.121419280 container create 25f80ad2db797b67bdd0cd0e7bbf51cace45fe3f884f883713ae77d06bdf7205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-16T05:46:34.998 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:34 smithi134 podman[73761]: 2024-04-16 05:46:34.908663536 +0000 UTC m=+0.350417984 container init 25f80ad2db797b67bdd0cd0e7bbf51cace45fe3f884f883713ae77d06bdf7205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-16T05:46:34.998 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:34 smithi134 podman[73761]: 2024-04-16 05:46:34.912150967 +0000 UTC m=+0.353905406 container start 25f80ad2db797b67bdd0cd0e7bbf51cace45fe3f884f883713ae77d06bdf7205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-04-16T05:46:34.998 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:35 smithi134 podman[73761]: 2024-04-16 05:46:35.000209572 +0000 UTC m=+0.441964016 container attach 25f80ad2db797b67bdd0cd0e7bbf51cace45fe3f884f883713ae77d06bdf7205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-16T05:46:35.384 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 16 05:46:35 smithi134 podman[73761]: 2024-04-16 05:46:35.202301037 +0000 UTC m=+0.644055475 container died 25f80ad2db797b67bdd0cd0e7bbf51cace45fe3f884f883713ae77d06bdf7205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-04-16T05:46:35.797 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.2.service' 2024-04-16T05:46:35.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:35.838 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-16T05:46:35.838 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-16T05:46:35.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.3 2024-04-16T05:46:36.183 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:35 smithi032 systemd[1]: Stopping Ceph osd.3 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:36.482 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:36 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3[54062]: 2024-04-16T05:46:36.185+0000 7f9057ab8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:36.483 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:36 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3[54062]: 2024-04-16T05:46:36.185+0000 7f9057ab8700 -1 osd.3 104 *** Got signal Terminated *** 2024-04-16T05:46:36.483 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:36 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3[54062]: 2024-04-16T05:46:36.185+0000 7f9057ab8700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:46:39.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:38 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:38.809+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:39.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:38 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:38.838+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:39.732 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:39 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3[54062]: 2024-04-16T05:46:39.473+0000 7f90540ac700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.098732+0000 front 2024-04-16T05:46:14.098576+0000 (oldest deadline 2024-04-16T05:46:39.398605+0000) 2024-04-16T05:46:40.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:39 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:39.811+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:40.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:40.045+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:40.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:39 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:39.793+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:40.732 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:40 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3[54062]: 2024-04-16T05:46:40.426+0000 7f90540ac700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.098732+0000 front 2024-04-16T05:46:14.098576+0000 (oldest deadline 2024-04-16T05:46:39.398605+0000) 2024-04-16T05:46:41.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:41.033+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:41.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:40.847+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:41.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:40 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:40.753+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:41.862 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:41 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:41.780+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:41.862 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:41 smithi032 podman[102693]: 2024-04-16 05:46:41.571526432 +0000 UTC m=+5.532718746 container died deab1c5a359be965f9206165f7a5352c83f35cfc711fb5aa302c9017de3c971c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409) 2024-04-16T05:46:41.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:41 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:41.644+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:42.121 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:41 smithi032 podman[102693]: 2024-04-16 05:46:41.946956083 +0000 UTC m=+5.908148395 container cleanup deab1c5a359be965f9206165f7a5352c83f35cfc711fb5aa302c9017de3c971c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:42.121 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:41 smithi032 bash[102693]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3 2024-04-16T05:46:42.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:41.890+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:42.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:42.048+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:42.482 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:42 smithi032 podman[102710]: 2024-04-16 05:46:42.123390153 +0000 UTC m=+0.546111957 container remove deab1c5a359be965f9206165f7a5352c83f35cfc711fb5aa302c9017de3c971c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:42.858 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:42 smithi032 podman[102800]: 2024-04-16 05:46:42.475474315 +0000 UTC m=+0.017707517 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:46:42.858 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:42 smithi032 podman[102800]: 2024-04-16 05:46:42.599409372 +0000 UTC m=+0.141642550 container create 22894e200fa0e1b0a78e3f25b3618c0ca7341661c66a34b62f4d1c3868222959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:46:42.859 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:42 smithi032 podman[102800]: 2024-04-16 05:46:42.860357855 +0000 UTC m=+0.402591025 container init 22894e200fa0e1b0a78e3f25b3618c0ca7341661c66a34b62f4d1c3868222959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T05:46:42.859 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:42 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:42.757+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:42.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:42 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:42.623+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:43.158 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:42 smithi032 podman[102800]: 2024-04-16 05:46:42.863854263 +0000 UTC m=+0.406087436 container start 22894e200fa0e1b0a78e3f25b3618c0ca7341661c66a34b62f4d1c3868222959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-16T05:46:43.158 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:42 smithi032 podman[102800]: 2024-04-16 05:46:42.969167348 +0000 UTC m=+0.511400520 container attach 22894e200fa0e1b0a78e3f25b3618c0ca7341661c66a34b62f4d1c3868222959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, ceph=True) 2024-04-16T05:46:43.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:42.916+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:43.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:43.094+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:43.447 INFO:journalctl@ceph.osd.3.smithi032.stdout:Apr 16 05:46:43 smithi032 podman[102800]: 2024-04-16 05:46:43.160566765 +0000 UTC m=+0.702799952 container died 22894e200fa0e1b0a78e3f25b3618c0ca7341661c66a34b62f4d1c3868222959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-16T05:46:43.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.3.service' 2024-04-16T05:46:43.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:43.891 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-16T05:46:43.892 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-16T05:46:43.892 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.4 2024-04-16T05:46:43.921 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:43 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:43.664+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:44.210 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:43 smithi082 systemd[1]: Stopping Ceph osd.4 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:44.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:43 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:43.758+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:44.383 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:43.900+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:44.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:44 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:44.065+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:44.649 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:44 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4[47644]: 2024-04-16T05:46:44.211+0000 7f55e481e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:44.649 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:44 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4[47644]: 2024-04-16T05:46:44.211+0000 7f55e481e700 -1 osd.4 104 *** Got signal Terminated *** 2024-04-16T05:46:44.650 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:44 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4[47644]: 2024-04-16T05:46:44.211+0000 7f55e481e700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:46:45.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:44 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:44.857+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:45.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:45.036+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:45.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:44 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:44.712+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:45.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:44 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:44.797+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:46.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:46 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:46.061+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:46.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:45 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:45.877+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:46.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:45 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:45.707+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:46.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:45 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:45.831+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:46.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:46 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:46.210+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:46.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:46.405+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:47.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:46 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:46.835+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:47.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:47.073+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:47.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:47.073+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:47.149 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:46 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4[47644]: 2024-04-16T05:46:46.907+0000 7f55e0e12700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.864214+0000 front 2024-04-16T05:46:23.864194+0000 (oldest deadline 2024-04-16T05:46:46.164348+0000) 2024-04-16T05:46:47.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:46 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:46.681+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:47.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:46 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:46.879+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:47.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:47.173+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:47.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:47 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:47.430+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:48.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:47 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:47.828+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:48.135 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:48.043+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:48.135 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:48.043+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:48.149 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:47 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4[47644]: 2024-04-16T05:46:47.870+0000 7f55e0e12700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.864214+0000 front 2024-04-16T05:46:23.864194+0000 (oldest deadline 2024-04-16T05:46:46.164348+0000) 2024-04-16T05:46:48.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:47 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:47.715+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:48.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:47 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:47.715+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:48.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:47 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:47.921+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:48.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:48.141+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:48.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:48.398+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:49.051 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4[47644]: 2024-04-16T05:46:48.903+0000 7f55e0e12700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.864214+0000 front 2024-04-16T05:46:23.864194+0000 (oldest deadline 2024-04-16T05:46:46.164348+0000) 2024-04-16T05:46:49.051 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:48.726+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:49.051 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:48 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:48.726+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:49.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:48 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:48.796+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:49.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:49 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:49.015+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:49.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:49 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:49.015+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:49.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:48.942+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:49.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:48 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:48.942+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:49.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:49 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:49.191+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:49.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:49.375+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:50.026 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:49 smithi082 podman[72332]: 2024-04-16 05:46:49.729985633 +0000 UTC m=+5.640693271 container died f2c135dfe30ba523668d37cf6e26d0878922c37122dafdbc43aece636a20a41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T05:46:50.026 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:49 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:49.730+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:50.026 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:49 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:49.730+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:50.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:49 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:49.830+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:50.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:50.034+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:50.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:50.034+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:50.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:49.900+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:50.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:49 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:49.900+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:50.285 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:50 smithi082 podman[72332]: 2024-04-16 05:46:50.027883318 +0000 UTC m=+5.938590933 container cleanup f2c135dfe30ba523668d37cf6e26d0878922c37122dafdbc43aece636a20a41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T05:46:50.285 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:50 smithi082 bash[72332]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4 2024-04-16T05:46:50.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:50.237+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:50.649 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:50 smithi082 podman[72349]: 2024-04-16 05:46:50.287575041 +0000 UTC m=+0.549319627 container remove f2c135dfe30ba523668d37cf6e26d0878922c37122dafdbc43aece636a20a41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-16T05:46:50.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:50.367+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:50.938 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:50.691+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:50.938 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:50.691+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:50.939 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:50 smithi082 podman[72448]: 2024-04-16 05:46:50.56026978 +0000 UTC m=+0.018330388 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:46:50.939 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:50 smithi082 podman[72448]: 2024-04-16 05:46:50.680115662 +0000 UTC m=+0.138176267 container create e9b33a9f162b7da2f22b75684cf5fb203b6680fc9528343c1ed51098c328af64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:50.939 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:50 smithi082 podman[72448]: 2024-04-16 05:46:50.94115739 +0000 UTC m=+0.399218018 container init e9b33a9f162b7da2f22b75684cf5fb203b6680fc9528343c1ed51098c328af64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T05:46:51.081 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:50 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:50.797+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:51.191 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:50 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:50.989+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:51.191 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:50 smithi082 podman[72448]: 2024-04-16 05:46:50.944464874 +0000 UTC m=+0.402525483 container start e9b33a9f162b7da2f22b75684cf5fb203b6680fc9528343c1ed51098c328af64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:46:51.191 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:51 smithi082 podman[72448]: 2024-04-16 05:46:51.024684911 +0000 UTC m=+0.482745538 container attach e9b33a9f162b7da2f22b75684cf5fb203b6680fc9528343c1ed51098c328af64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T05:46:51.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:50.878+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:51.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:50 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:50.878+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:51.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:51 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:51.283+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:51.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:51 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:51.083+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:51.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:51 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:51.083+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:51.558 INFO:journalctl@ceph.osd.4.smithi082.stdout:Apr 16 05:46:51 smithi082 podman[72448]: 2024-04-16 05:46:51.253481095 +0000 UTC m=+0.711541715 container died e9b33a9f162b7da2f22b75684cf5fb203b6680fc9528343c1ed51098c328af64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:51.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:51.407+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:51.859 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:51 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:51.724+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:51.859 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:51 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:51.724+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:51.890 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.4.service' 2024-04-16T05:46:51.972 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:51.973 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-16T05:46:51.973 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-16T05:46:51.973 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.5 2024-04-16T05:46:52.050 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:51 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:51.802+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:52.051 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:52 smithi134 systemd[1]: Stopping Ceph osd.5 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:46:52.051 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:52.051+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:52.051 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:52.051+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:52.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:51 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:51.991+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:52.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:51.926+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:52.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:51 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:51.926+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:52.323 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:52.249+0000 7f819155e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:46:52.323 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:52.249+0000 7f819155e700 -1 osd.5 104 *** Got signal Terminated *** 2024-04-16T05:46:52.323 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:52.249+0000 7f819155e700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:46:52.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:52.324+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:52.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:52.435+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:52.975 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:52.687+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:52.975 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:52.687+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:53.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:52 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:52.841+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:53.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:53.074+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:53.135 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:53.074+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:53.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:52.878+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:53.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:52.878+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:53.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:52 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:52.976+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:53.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:53.291+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:53.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:53.387+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:53.975 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:53.712+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:53.975 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:53.712+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:54.121 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:53 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:53.851+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:54.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:53.839+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:54.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:53 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:53.839+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:54.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:54 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:54.247+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:54.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:54 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:54.123+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:54.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:54 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:54.123+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:54.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:53 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:53.976+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:54.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:54 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:54.401+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:54.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:54 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:54.401+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:46:55.018 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:54 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:54.744+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:55.018 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:54 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:54.744+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:55.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:54 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:54.818+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:55.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:54 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:54.818+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:46:55.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:54 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:54.799+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:55.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:54 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:54.799+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:55.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:55 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:55.020+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:55.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:55.292+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:55.634 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:55.136+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:55.634 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:55.136+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:55.634 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:55.136+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.487137+0000 front 2024-04-16T05:46:31.487304+0000 (oldest deadline 2024-04-16T05:46:54.387190+0000) 2024-04-16T05:46:55.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:55 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:55.367+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:55.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:55 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:55.367+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:46:56.052 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:55 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:55.768+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:56.052 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:55 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:55.768+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:56.133 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:55.808+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:56.133 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:55 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:55.808+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:46:56.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:55 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:55.754+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:56.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:55 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:55.754+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:56.383 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:56 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:56.317+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:56.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:56 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:56.317+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:46:56.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:56 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:56.135+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:56.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:56 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:56.135+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:56.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:56 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:56.135+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.487137+0000 front 2024-04-16T05:46:31.487304+0000 (oldest deadline 2024-04-16T05:46:54.387190+0000) 2024-04-16T05:46:56.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:56 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:56.054+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:56.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:56.387+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:56.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:56.387+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:46:57.110 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:56 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:56.846+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:57.111 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:56 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:56.846+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:46:57.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:57 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:57.024+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:57.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:56 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:56.788+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:57.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:56 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:56.788+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:57.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:56 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:56.788+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:46:57.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:56.803+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:57.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:56 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:56.803+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:57.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:57 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:57.310+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:57.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:57 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:57.310+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:46:57.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:57 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:57.111+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:15.084930+0000 front 2024-04-16T05:46:15.085049+0000 (oldest deadline 2024-04-16T05:46:39.185016+0000) 2024-04-16T05:46:57.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:57 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:57.112+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.286278+0000 front 2024-04-16T05:46:23.286037+0000 (oldest deadline 2024-04-16T05:46:46.786033+0000) 2024-04-16T05:46:57.385 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:57 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5[48303]: 2024-04-16T05:46:57.112+0000 7f8189b4a700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.487137+0000 front 2024-04-16T05:46:31.487304+0000 (oldest deadline 2024-04-16T05:46:54.387190+0000) 2024-04-16T05:46:57.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:57 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:57.382+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:57.733 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:57 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:57.382+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:46:57.878 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:57 smithi134 podman[73879]: 2024-04-16 05:46:57.592802913 +0000 UTC m=+5.423863187 container died c25ad392f21cf2f52d656f08b16917058aa658a5ab9fa4a41e2d857816f2ccb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:46:58.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:57 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:57.879+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:58.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:57 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:57.879+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:46:58.135 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:57 smithi134 podman[73879]: 2024-04-16 05:46:57.983499367 +0000 UTC m=+5.814559587 container cleanup c25ad392f21cf2f52d656f08b16917058aa658a5ab9fa4a41e2d857816f2ccb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5, ceph=True, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409) 2024-04-16T05:46:58.135 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:57 smithi134 bash[73879]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5 2024-04-16T05:46:58.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:58 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:58.006+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:58.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:57 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:57.813+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:58.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:57 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:57.813+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:58.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:57 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:57.813+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:46:58.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:57 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:57.835+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:58.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:57 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:57.835+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:58.594 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:58 smithi134 podman[73897]: 2024-04-16 05:46:58.26820461 +0000 UTC m=+0.671896447 container remove c25ad392f21cf2f52d656f08b16917058aa658a5ab9fa4a41e2d857816f2ccb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-16T05:46:58.594 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:58 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:58.285+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:58.594 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:58 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:58.285+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:46:58.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:58 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:58.353+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:58.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:58 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:58.354+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:46:58.845 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:58 smithi134 podman[73989]: 2024-04-16 05:46:58.499073088 +0000 UTC m=+0.017834328 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:46:58.845 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:58 smithi134 podman[73989]: 2024-04-16 05:46:58.610951782 +0000 UTC m=+0.129713012 container create 23314decced3de60de2d6a1647cce7f97ae43529855929e15d8db3c2262e636d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-16T05:46:58.845 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:58 smithi134 podman[73989]: 2024-04-16 05:46:58.847151368 +0000 UTC m=+0.365912604 container init 23314decced3de60de2d6a1647cce7f97ae43529855929e15d8db3c2262e636d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-16T05:46:59.133 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:58 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:58.882+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:46:59.133 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:58 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:58.882+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:46:59.133 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:58 smithi134 podman[73989]: 2024-04-16 05:46:58.850351702 +0000 UTC m=+0.369112939 container start 23314decced3de60de2d6a1647cce7f97ae43529855929e15d8db3c2262e636d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-16T05:46:59.134 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:58 smithi134 podman[73989]: 2024-04-16 05:46:58.930590465 +0000 UTC m=+0.449351711 container attach 23314decced3de60de2d6a1647cce7f97ae43529855929e15d8db3c2262e636d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-16T05:46:59.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:59 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:59.048+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:46:59.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:46:59 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:46:59.048+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:46:59.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:58 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:58.836+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:46:59.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:58 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:58.836+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:46:59.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:58 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:58.836+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:46:59.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:58 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:58.788+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:46:59.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:58 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:58.788+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:46:59.383 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:59 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:59.262+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:46:59.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:46:59 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:46:59.262+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:46:59.384 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 16 05:46:59 smithi134 podman[73989]: 2024-04-16 05:46:59.136733762 +0000 UTC m=+0.655494998 container died 23314decced3de60de2d6a1647cce7f97ae43529855929e15d8db3c2262e636d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T05:46:59.711 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.5.service' 2024-04-16T05:46:59.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:59.381+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:46:59.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:46:59.381+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:46:59.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:46:59.752 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-16T05:46:59.752 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-16T05:46:59.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.6 2024-04-16T05:47:00.066 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:46:59 smithi032 systemd[1]: Stopping Ceph osd.6 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:00.066 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:00.067+0000 7f4b4327c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:00.067 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:59.761+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:00.067 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:59.761+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:00.067 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:46:59 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:46:59.761+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:00.088 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:59 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:59.808+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:00.088 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:59 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:59.808+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:00.088 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:46:59 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:46:59.808+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:00.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:59 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:59.904+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:00.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:46:59 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:46:59.904+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:00.372 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:00.067+0000 7f4b4327c700 -1 osd.6 104 *** Got signal Terminated *** 2024-04-16T05:47:00.373 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:00.067+0000 7f4b4327c700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:00.373 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:00.374+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:47:00.373 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:00.374+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:47:00.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:00.090+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:00.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:00.090+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:00.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:00.247+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:00.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:00.247+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:01.104 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:00.816+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:01.105 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:00.816+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:01.105 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:00 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:00.816+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:01.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:00.879+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:01.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:00 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:00.879+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:01.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:00.769+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:01.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:00.769+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:01.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:00 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:00.769+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:01.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:01 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:01.106+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:01.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:01 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:01.106+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:01.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:01 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:01.278+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:01.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:01 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:01.278+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:01.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:01.362+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:47:01.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:01.362+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:47:02.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:02 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:02.071+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:02.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:02 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:02.071+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:02.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:01 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:01.824+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:02.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:01 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:01.824+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:02.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:01 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:01.824+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:02.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:01.772+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:02.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:01.772+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:02.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:01.772+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:02.269 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:01 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:01.901+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:02.269 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:01 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:01.901+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:02.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:02 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:02.271+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:02.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:02 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:02.271+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:02.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:02.394+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:47:02.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:02.394+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:47:03.102 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:02 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:02.850+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:03.102 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:02 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:02.850+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:03.102 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:02 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:02.850+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:03.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:02.773+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:03.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:02.773+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:03.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:02 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:02.773+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:03.307 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:02 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:02.930+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:03.307 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:02 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:02.930+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:03.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:03 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:03.104+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:03.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:03 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:03.104+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:03.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:03 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:03.309+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:03.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:03 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:03.309+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:03.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:03 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:03.309+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:03.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:03 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:03.387+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:47:03.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:03 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:03.387+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:47:03.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:03 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:03.387+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.677799+0000 front 2024-04-16T05:46:38.677830+0000 (oldest deadline 2024-04-16T05:47:03.377939+0000) 2024-04-16T05:47:04.057 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:03 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:03.800+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:04.058 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:03 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:03.800+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:04.058 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:03 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:03.800+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:04.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:03 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:03.816+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:04.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:03 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:03.816+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:04.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:03 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:03.816+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:04.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:03 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:03.816+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:04.277 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:03 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:03.953+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:04.277 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:03 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:03.953+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:04.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:04 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:04.059+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:04.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:04 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:04.059+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:04.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:04 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:04.279+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:04.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:04 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:04.279+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:04.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:04 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:04.279+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:04.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:04.417+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:22.974457+0000 front 2024-04-16T05:46:22.974629+0000 (oldest deadline 2024-04-16T05:46:45.874622+0000) 2024-04-16T05:47:04.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:04.417+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.276420+0000 front 2024-04-16T05:46:32.276482+0000 (oldest deadline 2024-04-16T05:46:53.976608+0000) 2024-04-16T05:47:04.732 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6[60632]: 2024-04-16T05:47:04.417+0000 7f4b3b868700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.677799+0000 front 2024-04-16T05:46:38.677830+0000 (oldest deadline 2024-04-16T05:47:03.377939+0000) 2024-04-16T05:47:05.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:05 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:05.034+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:05.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:05 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:05.034+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:05.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:05 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:05.034+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.788288+0000 front 2024-04-16T05:46:38.788446+0000 (oldest deadline 2024-04-16T05:47:04.688119+0000) 2024-04-16T05:47:05.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:04 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:04.786+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:05.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:04 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:04.786+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:05.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:04 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:04.786+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:05.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:04.782+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:05.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:04.782+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:05.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:04.782+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:05.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:04 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:04.782+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:05.316 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:04 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:04.944+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:05.316 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:04 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:04.944+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:05.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:05 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:05.318+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:05.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:05 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:05.318+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:05.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:05 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:05.318+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:05.694 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:05 smithi032 podman[102917]: 2024-04-16 05:47:05.41642712 +0000 UTC m=+5.463387422 container died a8610e884ed89c2af6d98fcebd875f43a5af1108cc29e031fbc64b58c05130ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-16T05:47:05.982 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:05 smithi032 podman[102917]: 2024-04-16 05:47:05.784924475 +0000 UTC m=+5.831884777 container cleanup a8610e884ed89c2af6d98fcebd875f43a5af1108cc29e031fbc64b58c05130ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:05.982 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:05 smithi032 bash[102917]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6 2024-04-16T05:47:05.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:05 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:05.758+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:05.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:05 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:05.758+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:05.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:05 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:05.758+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:05.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:05 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:05.758+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:06.067 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:05 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:05.756+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:06.067 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:05 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:05.756+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:06.067 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:05 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:05.756+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:06.067 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:05 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:05.756+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:06.311 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:05 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:05.909+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:06.311 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:05 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:05.909+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:06.361 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:06 smithi032 podman[102933]: 2024-04-16 05:47:06.044862732 +0000 UTC m=+0.624324377 container remove a8610e884ed89c2af6d98fcebd875f43a5af1108cc29e031fbc64b58c05130ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:06.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:06.068+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:06.400 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:06.068+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:06.400 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:06.068+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.788288+0000 front 2024-04-16T05:46:38.788446+0000 (oldest deadline 2024-04-16T05:47:04.688119+0000) 2024-04-16T05:47:06.612 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:06 smithi032 podman[103026]: 2024-04-16 05:47:06.267384308 +0000 UTC m=+0.017823366 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:06.612 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:06 smithi032 podman[103026]: 2024-04-16 05:47:06.395714548 +0000 UTC m=+0.146153642 container create bb39c5083d4e7351484dad2bc2d536a52c87752052a505c3f1cf0cb8f935f709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-16T05:47:06.612 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:06 smithi032 podman[103026]: 2024-04-16 05:47:06.614954351 +0000 UTC m=+0.365393407 container init bb39c5083d4e7351484dad2bc2d536a52c87752052a505c3f1cf0cb8f935f709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-16T05:47:06.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:06 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:06.313+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:06.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:06 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:06.313+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:06.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:06 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:06.313+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:06.915 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:06 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:06.729+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:06.915 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:06 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:06.729+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:06.915 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:06 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:06.729+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:06.915 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:06 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:06.729+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:06.916 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:06 smithi032 podman[103026]: 2024-04-16 05:47:06.618426457 +0000 UTC m=+0.368865513 container start bb39c5083d4e7351484dad2bc2d536a52c87752052a505c3f1cf0cb8f935f709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-16T05:47:06.916 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:06 smithi032 podman[103026]: 2024-04-16 05:47:06.698175817 +0000 UTC m=+0.448614878 container attach bb39c5083d4e7351484dad2bc2d536a52c87752052a505c3f1cf0cb8f935f709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-16T05:47:06.916 INFO:journalctl@ceph.osd.6.smithi032.stdout:Apr 16 05:47:06 smithi032 podman[103026]: 2024-04-16 05:47:06.917871794 +0000 UTC m=+0.668310858 container died bb39c5083d4e7351484dad2bc2d536a52c87752052a505c3f1cf0cb8f935f709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-16T05:47:07.035 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:06.747+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:07.036 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:06.747+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:07.036 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:06.747+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:07.036 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:06.747+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:07.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:06 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:06.946+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:07.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:06 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:06.946+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:07.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:06 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:06.946+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:07.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:07.037+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:07.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:07.037+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:07.400 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:07.037+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.788288+0000 front 2024-04-16T05:46:38.788446+0000 (oldest deadline 2024-04-16T05:47:04.688119+0000) 2024-04-16T05:47:07.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.6.service' 2024-04-16T05:47:07.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:07.504 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-16T05:47:07.504 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-16T05:47:07.504 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.7 2024-04-16T05:47:07.633 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:07 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:07.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:07.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:07 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:07.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:07.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:07 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:07.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:07.810 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:07 smithi082 systemd[1]: Stopping Ceph osd.7 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:07.810 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:07.742+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:07.810 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:07.742+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:07.811 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:07.742+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:07.811 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:07.742+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:08.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:07.811+0000 7f78b2845700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:08.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:07.811+0000 7f78b2845700 -1 osd.7 104 *** Got signal Terminated *** 2024-04-16T05:47:08.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:07 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:07.811+0000 7f78b2845700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:08.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:08.011+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:08.151 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:08.011+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:08.151 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:08.011+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.788288+0000 front 2024-04-16T05:46:38.788446+0000 (oldest deadline 2024-04-16T05:47:04.688119+0000) 2024-04-16T05:47:08.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:07 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:07.749+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:08.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:07 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:07.749+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:08.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:07 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:07.749+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:08.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:07 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:07.749+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:08.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:07 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:07.939+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:08.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:07 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:07.939+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:08.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:07 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:07.939+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:08.884 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:08 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:08.387+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:08.884 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:08 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:08.387+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:08.884 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:08 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:08.387+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:08.986 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:08.731+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:08.987 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:08.731+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:08.987 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:08.731+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:08.987 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:08.731+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:09.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:08 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:08.750+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:09.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:08 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:08.750+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:09.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:08 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:08.750+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:09.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:08 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:08.750+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:09.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:08 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:08.975+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:09.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:08 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:08.975+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:09.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:08 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:08.975+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:09.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:08.988+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:09.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:08.988+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:09.400 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:08 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:08.988+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.788288+0000 front 2024-04-16T05:46:38.788446+0000 (oldest deadline 2024-04-16T05:47:04.688119+0000) 2024-04-16T05:47:09.884 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:09 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:09.403+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:09.884 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:09 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:09.403+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:09.884 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:09 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:09.403+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:09.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:09.708+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:09.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:09.708+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:09.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:09.708+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:09.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:09 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:09.708+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:10.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:09.768+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:10.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:09.768+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:10.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:09.768+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:10.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:09.768+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:10.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:09.962+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:10.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:09.963+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:10.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:09.963+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.788288+0000 front 2024-04-16T05:46:38.788446+0000 (oldest deadline 2024-04-16T05:47:04.688119+0000) 2024-04-16T05:47:10.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:10.023+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:10.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:10.023+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:10.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:10.023+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:10.382 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:10.023+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:10.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:10.383+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:10.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:10.383+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:10.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:10 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:10.383+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:10.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:10 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:10.668+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:10.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:10 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:10.668+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:10.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:10 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:10.668+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:10.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:10 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:10.668+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:11.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:10 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:10.937+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:11.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:10 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:10.937+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:11.150 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:10 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:10.937+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.788288+0000 front 2024-04-16T05:46:38.788446+0000 (oldest deadline 2024-04-16T05:47:04.688119+0000) 2024-04-16T05:47:11.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:10 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:10.728+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:11.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:10 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:10.728+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:11.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:10 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:10.728+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:11.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:10 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:10.728+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:11.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:11 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:11.011+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:11.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:11 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:11.011+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:11.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:11 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:11.011+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:11.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:11 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:11.011+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:11.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:11 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:11.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:11.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:11 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:11.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:11.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:11 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:11.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:11.968 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:11 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:11.684+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:11.968 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:11 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:11.684+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:11.968 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:11 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:11.684+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:11.968 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:11 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:11.684+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:11.968 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:11 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:11.684+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:11.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:11 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:11.625+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:11.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:11 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:11.625+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:11.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:11 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:11.625+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:11.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:11 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:11.625+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:12.383 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:12 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:12.046+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:12.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:12 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:12.046+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:12.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:12 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:12.046+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:12.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:12 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:12.046+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:12.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:11 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:11.969+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:25.286594+0000 front 2024-04-16T05:46:25.286689+0000 (oldest deadline 2024-04-16T05:46:50.586731+0000) 2024-04-16T05:47:12.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:11 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:11.969+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.887730+0000 front 2024-04-16T05:46:32.887857+0000 (oldest deadline 2024-04-16T05:46:58.787739+0000) 2024-04-16T05:47:12.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:11 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7[54216]: 2024-04-16T05:47:11.969+0000 7f78aee39700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:38.788288+0000 front 2024-04-16T05:46:38.788446+0000 (oldest deadline 2024-04-16T05:47:04.688119+0000) 2024-04-16T05:47:12.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:12 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:12.385+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:12.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:12 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:12.385+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:12.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:12 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:12.385+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:12.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:12 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:12.643+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:12.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:12 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:12.643+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:12.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:12 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:12.643+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:12.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:12 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:12.643+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:13.114 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:12 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:12.683+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:13.115 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:12 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:12.683+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:13.115 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:12 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:12.683+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:13.115 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:12 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:12.683+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:13.115 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:12 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:12.683+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:13.360 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:13.059+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:13.360 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:13.059+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:13.360 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:13.059+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:13.361 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:13.059+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:13.399 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:13 smithi082 podman[72556]: 2024-04-16 05:47:13.117268583 +0000 UTC m=+5.422931067 container died 702efa35451f0ccce59b12c3733a01438561c26e8e9e8713017ae0e7df955c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:13.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:13.362+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:13.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:13.362+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:13.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:13.363+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:13.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:13 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:13.363+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:47.171449+0000 front 2024-04-16T05:46:47.171422+0000 (oldest deadline 2024-04-16T05:47:13.071363+0000) 2024-04-16T05:47:13.877 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:13 smithi082 podman[72556]: 2024-04-16 05:47:13.603664914 +0000 UTC m=+5.909327423 container cleanup 702efa35451f0ccce59b12c3733a01438561c26e8e9e8713017ae0e7df955c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:13.877 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:13 smithi082 bash[72556]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7 2024-04-16T05:47:13.878 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:13 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:13.725+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:13.878 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:13 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:13.725+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:13.878 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:13 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:13.725+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:13.878 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:13 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:13.725+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:13.878 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:13 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:13.725+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:13.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:13 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:13.607+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:13.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:13 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:13.607+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:13.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:13 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:13.607+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:13.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:13 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:13.607+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:14.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:13 smithi082 podman[72573]: 2024-04-16 05:47:13.8799514 +0000 UTC m=+0.758760716 container remove 702efa35451f0ccce59b12c3733a01438561c26e8e9e8713017ae0e7df955c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-16T05:47:14.347 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:14 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:14.027+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:14.347 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:14 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:14.027+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:14.347 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:14 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:14.027+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:14.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:14 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:14.027+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:14.488 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:14 smithi082 podman[72670]: 2024-04-16 05:47:14.127465743 +0000 UTC m=+0.017849418 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:14.489 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:14 smithi082 podman[72670]: 2024-04-16 05:47:14.239051263 +0000 UTC m=+0.129434921 container create 32a88b9eff1b0dc585c2b79060919ce01a24a7e284f9b803c0226718588d3c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-16T05:47:14.489 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:14 smithi082 podman[72670]: 2024-04-16 05:47:14.491512375 +0000 UTC m=+0.381896040 container init 32a88b9eff1b0dc585c2b79060919ce01a24a7e284f9b803c0226718588d3c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-16T05:47:14.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:14 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:14.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:14.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:14 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:14.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:14.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:14 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:14.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:14.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:14 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:14.349+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:47.171449+0000 front 2024-04-16T05:46:47.171422+0000 (oldest deadline 2024-04-16T05:47:13.071363+0000) 2024-04-16T05:47:14.762 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:14 smithi082 podman[72670]: 2024-04-16 05:47:14.494991612 +0000 UTC m=+0.385375277 container start 32a88b9eff1b0dc585c2b79060919ce01a24a7e284f9b803c0226718588d3c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:14.762 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:14 smithi082 podman[72670]: 2024-04-16 05:47:14.566254145 +0000 UTC m=+0.456637816 container attach 32a88b9eff1b0dc585c2b79060919ce01a24a7e284f9b803c0226718588d3c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:14.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:14 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:14.648+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:14.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:14 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:14.648+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:14.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:14 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:14.648+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:14.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:14 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:14.648+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:15.075 INFO:journalctl@ceph.osd.7.smithi082.stdout:Apr 16 05:47:14 smithi082 podman[72670]: 2024-04-16 05:47:14.786395303 +0000 UTC m=+0.676778981 container died 32a88b9eff1b0dc585c2b79060919ce01a24a7e284f9b803c0226718588d3c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-7-deactivate, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-16T05:47:15.075 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:14 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:14.763+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:15.075 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:14 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:14.763+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:15.075 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:14 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:14.763+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:15.075 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:14 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:14.764+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:15.075 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:14 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:14.764+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:15.357 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:15.054+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:15.357 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:15.054+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:15.357 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:15.054+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:15.357 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:15.054+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:15.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.7.service' 2024-04-16T05:47:15.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:15.454 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-16T05:47:15.454 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-16T05:47:15.454 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.8 2024-04-16T05:47:15.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:15.359+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:15.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:15.359+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:15.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:15.359+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:15.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:15.359+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:47.171449+0000 front 2024-04-16T05:46:47.171422+0000 (oldest deadline 2024-04-16T05:47:13.071363+0000) 2024-04-16T05:47:15.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:15 smithi134 systemd[1]: Stopping Ceph osd.8 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:15.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:15 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:15.649+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:15.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:15 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:15.649+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:15.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:15 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:15.649+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:15.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:15 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:15.649+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:15.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:15 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:15.649+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:16.134 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:15.778+0000 7f3721346700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:16.134 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:15.778+0000 7f3721346700 -1 osd.8 104 *** Got signal Terminated *** 2024-04-16T05:47:16.134 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:15 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:15.778+0000 7f3721346700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:16.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:16 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:16.022+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:16.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:16 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:16.022+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:16.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:16 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:16.022+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:16.135 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:16 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:16.022+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:16.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:15 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:15.720+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:16.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:15 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:15.720+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:16.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:15 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:15.720+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:16.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:15 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:15.720+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:16.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:15 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:15.720+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:16.633 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:16 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:16.325+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:16.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:16 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:16.325+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:16.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:16 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:16.325+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:16.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:16 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:16.325+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:47.171449+0000 front 2024-04-16T05:46:47.171422+0000 (oldest deadline 2024-04-16T05:47:13.071363+0000) 2024-04-16T05:47:16.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:16 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:16.677+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:16.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:16 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:16.677+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:16.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:16 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:16.677+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:16.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:16 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:16.677+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:16.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:16 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:16.677+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:16.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:16 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:16.675+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:16.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:16 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:16.675+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:16.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:16 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:16.675+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:16.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:16 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:16.675+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:16.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:16 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:16.675+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:17.313 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:17 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:17.039+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:17.314 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:17 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:17.039+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:17.314 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:17 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:17.039+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:17.314 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:17 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:17.039+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:17.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:17 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:17.315+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:17.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:17 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:17.315+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:17.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:17 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:17.315+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:17.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:17 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:17.315+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:47.171449+0000 front 2024-04-16T05:46:47.171422+0000 (oldest deadline 2024-04-16T05:47:13.071363+0000) 2024-04-16T05:47:17.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:17 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:17.700+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:17.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:17 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:17.700+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:17.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:17 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:17.700+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:17.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:17 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:17.700+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:17.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:17 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:17.700+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:18.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:17 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:17.682+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:18.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:17 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:17.682+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:18.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:17 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:17.682+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:18.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:17 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:17.682+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:18.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:17 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:17.682+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:18.331 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:18.010+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:18.332 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:18.010+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:18.332 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:18.010+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:18.332 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:18.010+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:18.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:18.333+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:18.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:18.333+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:18.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:18.333+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:18.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:18.333+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:47.171449+0000 front 2024-04-16T05:46:47.171422+0000 (oldest deadline 2024-04-16T05:47:13.071363+0000) 2024-04-16T05:47:18.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:18 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:18.636+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:18.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:18 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:18.636+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:18.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:18 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:18.636+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:18.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:18 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:18.636+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:18.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:18 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:18.636+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:19.135 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:18 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:18.745+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:19.136 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:18 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:18.745+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:19.136 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:18 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:18.745+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:19.136 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:18 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:18.745+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:19.136 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:18 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:18.745+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:19.290 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:18.989+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:19.290 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:18.989+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:19.290 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:18.989+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:19.290 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:18 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:18.989+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:19.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:19 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:19.291+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:19.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:19 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:19.291+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:19.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:19 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:19.291+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:19.634 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:19 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:19.291+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:47.171449+0000 front 2024-04-16T05:46:47.171422+0000 (oldest deadline 2024-04-16T05:47:13.071363+0000) 2024-04-16T05:47:20.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:19 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:19.653+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:20.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:19 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:19.653+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:20.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:19 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:19.653+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:20.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:19 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:19.653+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:20.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:19 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:19.653+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:20.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:19 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:19.653+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:20.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:19 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:19.786+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:20.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:19 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:19.786+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:20.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:19 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:19.786+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:20.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:19 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:19.786+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:20.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:19 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:19.786+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:20.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:20.256+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:23.565606+0000 front 2024-04-16T05:46:23.565515+0000 (oldest deadline 2024-04-16T05:46:45.865563+0000) 2024-04-16T05:47:20.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:20.256+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.369351+0000 front 2024-04-16T05:46:31.369333+0000 (oldest deadline 2024-04-16T05:46:55.469369+0000) 2024-04-16T05:47:20.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:20.256+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.770167+0000 front 2024-04-16T05:46:40.770210+0000 (oldest deadline 2024-04-16T05:47:03.070083+0000) 2024-04-16T05:47:20.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:20.256+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:47.171449+0000 front 2024-04-16T05:46:47.171422+0000 (oldest deadline 2024-04-16T05:47:13.071363+0000) 2024-04-16T05:47:20.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8[55329]: 2024-04-16T05:47:20.256+0000 7f3719932700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:54.772224+0000 front 2024-04-16T05:46:54.772265+0000 (oldest deadline 2024-04-16T05:47:19.472354+0000) 2024-04-16T05:47:20.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.030+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:20.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.030+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:20.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.030+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:20.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.030+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:20.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.030+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:20.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:20.610+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:20.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:20.610+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:20.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:20.610+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:20.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:20.610+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:20.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:20.610+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:20.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:20 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:20.610+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:21.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:20 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:20.762+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:21.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:20 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:20.762+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:21.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:20 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:20.762+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:21.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:20 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:20.762+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:21.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:20 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:20.762+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:21.384 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:21 smithi134 podman[74105]: 2024-04-16 05:47:21.205422273 +0000 UTC m=+5.545866096 container died 2fe94b80bd48114df7ef727c95d89d84cf095ddc10947186950474918fd729ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-16T05:47:21.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:21.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:21.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:21.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:21.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:20 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:20.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:21.751 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:21 smithi134 podman[74105]: 2024-04-16 05:47:21.544937492 +0000 UTC m=+5.885381342 container cleanup 2fe94b80bd48114df7ef727c95d89d84cf095ddc10947186950474918fd729ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:47:21.752 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:21 smithi134 bash[74105]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8 2024-04-16T05:47:21.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:21 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:21.580+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:21.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:21 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:21.580+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:21.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:21 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:21.580+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:21.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:21 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:21.580+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:21.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:21 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:21.580+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:21.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:21 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:21.580+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:22.134 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:21 smithi134 podman[74122]: 2024-04-16 05:47:21.754523436 +0000 UTC m=+0.545284134 container remove 2fe94b80bd48114df7ef727c95d89d84cf095ddc10947186950474918fd729ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:22.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:21 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:21.992+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:22.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:21 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:21.992+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:22.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:21 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:21.992+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:22.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:21 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:21.992+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:22.134 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:21 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:21.992+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:22.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:21 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:21.752+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:22.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:21 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:21.752+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:22.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:21 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:21.752+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:22.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:21 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:21.752+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:22.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:21 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:21.752+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:22.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:21 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:21.752+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.295069+0000 front 2024-04-16T05:46:55.295145+0000 (oldest deadline 2024-04-16T05:47:21.195043+0000) 2024-04-16T05:47:22.497 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:22 smithi134 podman[74220]: 2024-04-16 05:47:22.060058861 +0000 UTC m=+0.018014849 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:22.497 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:22 smithi134 podman[74220]: 2024-04-16 05:47:22.1637528 +0000 UTC m=+0.121708830 container create 1e764c0ce5a948146a9a1a85d1d3b50e47099912fc489a8f1d567b362c3fdff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409) 2024-04-16T05:47:22.497 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:22 smithi134 podman[74220]: 2024-04-16 05:47:22.399473989 +0000 UTC m=+0.357429977 container init 1e764c0ce5a948146a9a1a85d1d3b50e47099912fc489a8f1d567b362c3fdff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-16T05:47:22.497 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:22 smithi134 podman[74220]: 2024-04-16 05:47:22.402606538 +0000 UTC m=+0.360562526 container start 1e764c0ce5a948146a9a1a85d1d3b50e47099912fc489a8f1d567b362c3fdff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T05:47:22.497 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:22 smithi134 podman[74220]: 2024-04-16 05:47:22.499466825 +0000 UTC m=+0.457422824 container attach 1e764c0ce5a948146a9a1a85d1d3b50e47099912fc489a8f1d567b362c3fdff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-16T05:47:22.884 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 16 05:47:22 smithi134 podman[74220]: 2024-04-16 05:47:22.6957922 +0000 UTC m=+0.653748189 container died 1e764c0ce5a948146a9a1a85d1d3b50e47099912fc489a8f1d567b362c3fdff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-16T05:47:22.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:22 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:22.581+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:22.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:22 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:22.581+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:22.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:22 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:22.581+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:22.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:22 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:22.581+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:22.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:22 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:22.581+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:22.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:22 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:22.581+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:23.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:22 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:22.779+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:23.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:22 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:22.779+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:23.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:22 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:22.779+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:23.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:22 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:22.779+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:23.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:22 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:22.779+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:23.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:22 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:22.779+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.295069+0000 front 2024-04-16T05:46:55.295145+0000 (oldest deadline 2024-04-16T05:47:21.195043+0000) 2024-04-16T05:47:23.292 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:22 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:22.965+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:23.293 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:22 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:22.965+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:23.293 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:22 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:22.965+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:23.293 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:22 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:22.965+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:23.293 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:22 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:22.965+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:23.315 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.8.service' 2024-04-16T05:47:23.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:23.356 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-16T05:47:23.356 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-16T05:47:23.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.9 2024-04-16T05:47:23.685 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 systemd[1]: Stopping Ceph osd.9 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:23.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:23 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:23.619+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:23.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:23 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:23.619+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:23.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:23 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:23.619+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:23.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:23 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:23.619+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:23.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:23 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:23.619+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:23.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:23 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:23.619+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:23.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.687+0000 7f23a1dde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:23.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.687+0000 7f23a1dde700 -1 osd.9 104 *** Got signal Terminated *** 2024-04-16T05:47:23.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.687+0000 7f23a1dde700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:23.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.814+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:23.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.814+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:23.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.814+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:23.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.814+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:23.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.814+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:23.983 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:23 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:23.814+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.295069+0000 front 2024-04-16T05:46:55.295145+0000 (oldest deadline 2024-04-16T05:47:21.195043+0000) 2024-04-16T05:47:24.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:23 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:23.973+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:24.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:23 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:23.973+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:24.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:23 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:23.973+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:24.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:23 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:23.973+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:24.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:23 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:23.973+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:25.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:24 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:24.665+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:25.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:24 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:24.666+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:25.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:24 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:24.666+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:25.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:24 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:24.666+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:25.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:24 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:24.666+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:25.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:24 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:24.666+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:25.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:24 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:24.843+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:25.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:24 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:24.843+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:25.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:24 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:24.843+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:25.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:24 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:24.843+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:25.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:24 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:24.843+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:25.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:24 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:24.843+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.295069+0000 front 2024-04-16T05:46:55.295145+0000 (oldest deadline 2024-04-16T05:47:21.195043+0000) 2024-04-16T05:47:25.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:24 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:24.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:25.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:24 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:24.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:25.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:24 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:24.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:25.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:24 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:24.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:25.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:24 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:24.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:25.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:25 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:25.634+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:25.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:25 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:25.634+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:25.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:25 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:25.634+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:25.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:25 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:25.634+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:25.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:25 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:25.634+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:25.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:25 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:25.634+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:26.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:25 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:25.865+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:26.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:25 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:25.865+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:26.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:25 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:25.865+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:26.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:25 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:25.865+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:26.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:25 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:25.865+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:26.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:25 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:25.865+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.295069+0000 front 2024-04-16T05:46:55.295145+0000 (oldest deadline 2024-04-16T05:47:21.195043+0000) 2024-04-16T05:47:26.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:26 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:25.999+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:26.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:26 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:25.999+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:26.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:26 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:25.999+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:26.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:26 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:25.999+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:26.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:26 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:25.999+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:26.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:26 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:25.999+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:26.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:26 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:26.618+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:26.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:26 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:26.618+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:26.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:26 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:26.618+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:26.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:26 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:26.618+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:26.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:26 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:26.618+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:26.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:26 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:26.618+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:27.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:26 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:26.894+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:27.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:26 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:26.894+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:27.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:26 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:26.894+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:27.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:26 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:26.894+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:27.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:26 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:26.894+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:27.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:26 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:26.894+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.295069+0000 front 2024-04-16T05:46:55.295145+0000 (oldest deadline 2024-04-16T05:47:21.195043+0000) 2024-04-16T05:47:27.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.005+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:27.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.005+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:27.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.005+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:27.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.005+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:27.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.005+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:27.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.005+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:28.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:27 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:27.654+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:28.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:27 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:27.654+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:28.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:27 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:27.654+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:28.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:27 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:27.654+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:28.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:27 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:27.654+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:28.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:27 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:27.654+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:28.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:27 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:27.654+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:28.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:27.877+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.488621+0000 front 2024-04-16T05:46:14.488758+0000 (oldest deadline 2024-04-16T05:46:37.988776+0000) 2024-04-16T05:47:28.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:27.877+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:24.891108+0000 front 2024-04-16T05:46:24.891167+0000 (oldest deadline 2024-04-16T05:46:48.390997+0000) 2024-04-16T05:47:28.232 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:27.877+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:32.492279+0000 front 2024-04-16T05:46:32.492516+0000 (oldest deadline 2024-04-16T05:46:59.492604+0000) 2024-04-16T05:47:28.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:27.877+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:39.493083+0000 front 2024-04-16T05:46:39.493114+0000 (oldest deadline 2024-04-16T05:47:02.993158+0000) 2024-04-16T05:47:28.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:27.877+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.494418+0000 front 2024-04-16T05:46:46.494431+0000 (oldest deadline 2024-04-16T05:47:15.294582+0000) 2024-04-16T05:47:28.233 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:27 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9[67684]: 2024-04-16T05:47:27.877+0000 7f239e3d2700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.295069+0000 front 2024-04-16T05:46:55.295145+0000 (oldest deadline 2024-04-16T05:47:21.195043+0000) 2024-04-16T05:47:28.383 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.977+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:28.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.977+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:28.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.977+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:28.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.977+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:28.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.977+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:28.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:27 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:27.977+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:28.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:28 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:28.640+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:28.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:28 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:28.640+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:28.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:28 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:28.640+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:28.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:28 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:28.640+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:28.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:28 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:28.640+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:28.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:28 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:28.640+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:28.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:28 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:28.640+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:29.360 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:29 smithi032 podman[103144]: 2024-04-16 05:47:29.089148098 +0000 UTC m=+5.526166867 container died c149ca5a2661fad51ab175e7dafba2d3a095686777ea2327d219c0f68aa90f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:47:29.383 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:28.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:29.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:28.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:29.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:28.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:29.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:28.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:29.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:28.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:29.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:28 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:28.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:29.711 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:29 smithi032 podman[103144]: 2024-04-16 05:47:29.462555591 +0000 UTC m=+5.899574352 container cleanup c149ca5a2661fad51ab175e7dafba2d3a095686777ea2327d219c0f68aa90f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:29.711 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:29 smithi032 bash[103144]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9 2024-04-16T05:47:29.888 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:29 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:29.638+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:29.888 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:29 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:29.638+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:29.888 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:29 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:29.638+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:29.888 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:29 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:29.638+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:29.888 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:29 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:29.638+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:29.888 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:29 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:29.638+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:29.888 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:29 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:29.638+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:29.982 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:29 smithi032 podman[103161]: 2024-04-16 05:47:29.714051777 +0000 UTC m=+0.622179234 container remove c149ca5a2661fad51ab175e7dafba2d3a095686777ea2327d219c0f68aa90f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:47:30.356 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:30 smithi032 podman[103253]: 2024-04-16 05:47:30.031447313 +0000 UTC m=+0.112793710 container create 0ff51c3e12209a39199c1e39640ab52209ed7b01983a1254f2dd9dc5e76ebc87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:30.356 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:30 smithi032 podman[103253]: 2024-04-16 05:47:29.93656558 +0000 UTC m=+0.017911989 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:30.356 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:30 smithi032 podman[103253]: 2024-04-16 05:47:30.275440668 +0000 UTC m=+0.356787067 container init 0ff51c3e12209a39199c1e39640ab52209ed7b01983a1254f2dd9dc5e76ebc87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:30.356 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:30 smithi032 podman[103253]: 2024-04-16 05:47:30.278604197 +0000 UTC m=+0.359950596 container start 0ff51c3e12209a39199c1e39640ab52209ed7b01983a1254f2dd9dc5e76ebc87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:47:30.356 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:30 smithi032 podman[103253]: 2024-04-16 05:47:30.358738877 +0000 UTC m=+0.440085302 container attach 0ff51c3e12209a39199c1e39640ab52209ed7b01983a1254f2dd9dc5e76ebc87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-16T05:47:30.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:29 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:29.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:30.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:29 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:29.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:30.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:29 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:29.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:30.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:29 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:29.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:30.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:29 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:29.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:30.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:29 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:29.937+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:30.732 INFO:journalctl@ceph.osd.9.smithi032.stdout:Apr 16 05:47:30 smithi032 podman[103253]: 2024-04-16 05:47:30.579210112 +0000 UTC m=+0.660556509 container died 0ff51c3e12209a39199c1e39640ab52209ed7b01983a1254f2dd9dc5e76ebc87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-16T05:47:31.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:30 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:30.675+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:31.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:30 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:30.675+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:31.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:30 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:30.675+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:31.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:30 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:30.675+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:31.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:30 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:30.675+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:31.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:30 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:30.675+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:31.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:30 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:30.675+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:31.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.9.service' 2024-04-16T05:47:31.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:31.323 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-16T05:47:31.323 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-16T05:47:31.323 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.10 2024-04-16T05:47:31.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:30 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:30.953+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:31.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:30 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:30.953+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:31.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:30 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:30.953+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:31.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:30 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:30.953+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:31.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:30 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:30.953+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:31.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:30 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:30.953+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:31.610 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 systemd[1]: Stopping Ceph osd.10 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:31.899 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.612+0000 7fc76db9f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:31.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.612+0000 7fc76db9f700 -1 osd.10 104 *** Got signal Terminated *** 2024-04-16T05:47:31.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.612+0000 7fc76db9f700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:31.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.690+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:31.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.690+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:31.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.690+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:31.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.690+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:31.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.690+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:31.901 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.690+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:31.901 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:31 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:31.690+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:32.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:31 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:31.928+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:32.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:31 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:31.928+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:32.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:31 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:31.928+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:32.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:31 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:31.928+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:32.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:31 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:31.928+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:32.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:31 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:31.928+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:33.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:32.699+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:33.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:32.699+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:33.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:32.699+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:33.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:32.699+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:33.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:32.699+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:33.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:32.699+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:33.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:32 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:32.699+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:33.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:32 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:32.976+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:33.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:32 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:32.976+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:33.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:32 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:32.976+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:33.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:32 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:32.976+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:33.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:32 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:32.976+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:33.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:32 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:32.976+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:34.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:33 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:33.656+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:34.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:33 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:33.656+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:34.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:33 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:33.656+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:34.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:33 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:33.656+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:34.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:33 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:33.656+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:34.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:33 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:33.656+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:34.151 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:33 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:33.656+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:34.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:33 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:33.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:34.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:33 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:33.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:34.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:33 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:33.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:34.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:33 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:33.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:34.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:33 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:33.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:34.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:33 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:33.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:35.149 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:34 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:34.669+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:35.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:34 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:34.669+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:35.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:34 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:34.669+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:35.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:34 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:34.669+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:35.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:34 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:34.669+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:35.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:34 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:34.669+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:35.150 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:34 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:34.669+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:35.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:35 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:35.043+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:35.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:35 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:35.043+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:35.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:35 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:35.043+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:35.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:35 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:35.043+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:35.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:35 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:35.043+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:35.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:35 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:35.043+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:35.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:35 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:35.043+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:35.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:35 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:35.629+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:16.181018+0000 front 2024-04-16T05:46:16.180966+0000 (oldest deadline 2024-04-16T05:46:41.480701+0000) 2024-04-16T05:47:35.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:35 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:35.629+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-04-16T05:46:21.481053+0000 front 2024-04-16T05:46:21.481101+0000 (oldest deadline 2024-04-16T05:46:47.381236+0000) 2024-04-16T05:47:35.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:35 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:35.629+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:31.482157+0000 front 2024-04-16T05:46:31.482241+0000 (oldest deadline 2024-04-16T05:46:56.782199+0000) 2024-04-16T05:47:35.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:35 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:35.629+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.784188+0000 front 2024-04-16T05:46:40.784037+0000 (oldest deadline 2024-04-16T05:47:04.883912+0000) 2024-04-16T05:47:35.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:35 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:35.629+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:44.884236+0000 front 2024-04-16T05:46:44.884259+0000 (oldest deadline 2024-04-16T05:47:10.784341+0000) 2024-04-16T05:47:35.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:35 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:35.629+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:55.385820+0000 front 2024-04-16T05:46:55.385853+0000 (oldest deadline 2024-04-16T05:47:18.885608+0000) 2024-04-16T05:47:35.900 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:35 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10[60854]: 2024-04-16T05:47:35.629+0000 7fc76a193700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:02.986897+0000 front 2024-04-16T05:47:02.986870+0000 (oldest deadline 2024-04-16T05:47:27.086735+0000) 2024-04-16T05:47:36.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.012+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:36.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.012+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:36.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.012+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:36.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.012+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:36.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.012+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:36.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.012+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:36.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.012+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:37.358 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:37 smithi082 podman[72788]: 2024-04-16 05:47:37.047663148 +0000 UTC m=+5.529570398 container died 78dfe623d3aaf1d9265fbf810f8587fa172265a93e6bbd9434a2460f4773f50d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-16T05:47:37.359 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:37 smithi082 podman[72788]: 2024-04-16 05:47:37.361046322 +0000 UTC m=+5.842953598 container cleanup 78dfe623d3aaf1d9265fbf810f8587fa172265a93e6bbd9434a2460f4773f50d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True) 2024-04-16T05:47:37.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:37.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:37.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:37.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:37.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:37.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:37.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:36 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:36.988+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:37.649 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:37 smithi082 bash[72788]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10 2024-04-16T05:47:37.649 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:37 smithi082 podman[72805]: 2024-04-16 05:47:37.570425415 +0000 UTC m=+0.515993973 container remove 78dfe623d3aaf1d9265fbf810f8587fa172265a93e6bbd9434a2460f4773f50d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:47:38.238 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:37 smithi082 podman[72898]: 2024-04-16 05:47:37.880616173 +0000 UTC m=+0.017951289 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:38.238 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:37 smithi082 podman[72898]: 2024-04-16 05:47:37.979716058 +0000 UTC m=+0.117051207 container create 368bec452dc12c17c381eeb748794ab39b8436eed518cc8ee0239541e633b3c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:38.238 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:38 smithi082 podman[72898]: 2024-04-16 05:47:38.157178599 +0000 UTC m=+0.294513712 container init 368bec452dc12c17c381eeb748794ab39b8436eed518cc8ee0239541e633b3c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:38.238 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:38 smithi082 podman[72898]: 2024-04-16 05:47:38.160661303 +0000 UTC m=+0.297996409 container start 368bec452dc12c17c381eeb748794ab39b8436eed518cc8ee0239541e633b3c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409) 2024-04-16T05:47:38.238 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:38 smithi082 podman[72898]: 2024-04-16 05:47:38.240078446 +0000 UTC m=+0.377413578 container attach 368bec452dc12c17c381eeb748794ab39b8436eed518cc8ee0239541e633b3c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-16T05:47:38.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:37 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:37.942+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:38.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:37 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:37.942+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:38.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:37 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:37.942+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:38.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:37 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:37.942+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:38.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:37 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:37.942+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:38.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:37 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:37.942+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:38.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:37 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:37.942+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:38.649 INFO:journalctl@ceph.osd.10.smithi082.stdout:Apr 16 05:47:38 smithi082 podman[72898]: 2024-04-16 05:47:38.451717731 +0000 UTC m=+0.589052854 container died 368bec452dc12c17c381eeb748794ab39b8436eed518cc8ee0239541e633b3c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-16T05:47:39.026 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.10.service' 2024-04-16T05:47:39.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:39.070 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-16T05:47:39.070 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-16T05:47:39.070 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.11 2024-04-16T05:47:39.284 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:38 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:38.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:39.285 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:38 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:38.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:39.285 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:38 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:38.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:39.285 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:38 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:38.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:39.285 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:38 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:38.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:39.285 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:38 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:38.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:39.285 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:38 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:38.960+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:39.285 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:39 smithi134 systemd[1]: Stopping Ceph osd.11 for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:39.634 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:39 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:39.396+0000 7f5211939700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:39.634 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:39 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:39.396+0000 7f5211939700 -1 osd.11 104 *** Got signal Terminated *** 2024-04-16T05:47:39.634 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:39 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:39.396+0000 7f5211939700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T05:47:40.301 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.004+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:40.301 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.004+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:40.301 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.004+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:40.301 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.004+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:40.302 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.004+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:40.302 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.004+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:40.302 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.004+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:41.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:41.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:41.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:41.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:41.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:41.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:41.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:40 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:40.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:42.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:41.968+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:42.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:41.968+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:42.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:41.968+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:42.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:41.968+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:42.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:41.968+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:42.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:41.968+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:42.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:41 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:41.968+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:43.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:42.987+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:43.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:42.987+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:43.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:42.987+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:43.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:42.987+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:43.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:42.987+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:43.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:42.987+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:43.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:42.987+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:43.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:42 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:42.987+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-04-16T05:47:20.603901+0000 front 2024-04-16T05:47:20.603917+0000 (oldest deadline 2024-04-16T05:47:42.303882+0000) 2024-04-16T05:47:44.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:43.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-04-16T05:46:14.992946+0000 front 2024-04-16T05:46:14.993044+0000 (oldest deadline 2024-04-16T05:46:38.492895+0000) 2024-04-16T05:47:44.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:43.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-16T05:46:29.495815+0000 front 2024-04-16T05:46:29.495576+0000 (oldest deadline 2024-04-16T05:46:54.795633+0000) 2024-04-16T05:47:44.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:43.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-04-16T05:46:40.597170+0000 front 2024-04-16T05:46:40.597290+0000 (oldest deadline 2024-04-16T05:47:06.497172+0000) 2024-04-16T05:47:44.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:43.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-04-16T05:46:46.998396+0000 front 2024-04-16T05:46:46.998420+0000 (oldest deadline 2024-04-16T05:47:09.898237+0000) 2024-04-16T05:47:44.384 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:43.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-16T05:46:56.299640+0000 front 2024-04-16T05:46:56.299599+0000 (oldest deadline 2024-04-16T05:47:19.100172+0000) 2024-04-16T05:47:44.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:43.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-04-16T05:47:04.901812+0000 front 2024-04-16T05:47:04.901765+0000 (oldest deadline 2024-04-16T05:47:25.902214+0000) 2024-04-16T05:47:44.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:43.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-04-16T05:47:08.803217+0000 front 2024-04-16T05:47:08.803255+0000 (oldest deadline 2024-04-16T05:47:34.703159+0000) 2024-04-16T05:47:44.385 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:43 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11[62397]: 2024-04-16T05:47:43.995+0000 7f520df2d700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-04-16T05:47:20.603901+0000 front 2024-04-16T05:47:20.603917+0000 (oldest deadline 2024-04-16T05:47:42.303882+0000) 2024-04-16T05:47:45.195 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:44 smithi134 podman[74338]: 2024-04-16 05:47:44.920521146 +0000 UTC m=+5.651762185 container died 778bc6433199d6beb10846c9dcd08739b1232fc05642b92f666d2f15c350d269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:45.528 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:45 smithi134 podman[74338]: 2024-04-16 05:47:45.27995903 +0000 UTC m=+6.011200059 container cleanup 778bc6433199d6beb10846c9dcd08739b1232fc05642b92f666d2f15c350d269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-16T05:47:45.528 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:45 smithi134 bash[74338]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11 2024-04-16T05:47:45.839 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:45 smithi134 podman[74368]: 2024-04-16 05:47:45.53093542 +0000 UTC m=+0.609451113 container remove 778bc6433199d6beb10846c9dcd08739b1232fc05642b92f666d2f15c350d269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:46.135 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:45 smithi134 podman[74464]: 2024-04-16 05:47:45.745136889 +0000 UTC m=+0.017204119 image pull 3e786b67158fa845c69272819d3a605b7fbe7c1e1355d9bbf14518829dbb9317 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1 2024-04-16T05:47:46.136 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:45 smithi134 podman[74464]: 2024-04-16 05:47:45.940401003 +0000 UTC m=+0.212468232 container create 6c6d41b493d30c5fd9a61583afbbfdebc71a97ded9bd3b6fd2c047917ed7f9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:47:46.443 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:46 smithi134 podman[74464]: 2024-04-16 05:47:46.155288328 +0000 UTC m=+0.427355562 container init 6c6d41b493d30c5fd9a61583afbbfdebc71a97ded9bd3b6fd2c047917ed7f9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409) 2024-04-16T05:47:46.443 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:46 smithi134 podman[74464]: 2024-04-16 05:47:46.158430122 +0000 UTC m=+0.430497361 container start 6c6d41b493d30c5fd9a61583afbbfdebc71a97ded9bd3b6fd2c047917ed7f9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-16T05:47:46.443 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:46 smithi134 podman[74464]: 2024-04-16 05:47:46.242745181 +0000 UTC m=+0.514812426 container attach 6c6d41b493d30c5fd9a61583afbbfdebc71a97ded9bd3b6fd2c047917ed7f9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0) 2024-04-16T05:47:46.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 16 05:47:46 smithi134 podman[74464]: 2024-04-16 05:47:46.445800894 +0000 UTC m=+0.717868138 container died 6c6d41b493d30c5fd9a61583afbbfdebc71a97ded9bd3b6fd2c047917ed7f9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-16T05:47:46.998 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@osd.11.service' 2024-04-16T05:47:47.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:47.038 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-16T05:47:47.038 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-16T05:47:47.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.a 2024-04-16T05:47:47.462 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:47:47 smithi032 systemd[1]: Stopping Ceph mds.a for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:47.732 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:47:47 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a[74508]: 2024-04-16T05:47:47.464+0000 7fc923683700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:47.732 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:47:47 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a[74508]: 2024-04-16T05:47:47.464+0000 7fc923683700 -1 mds.a *** got signal Terminated *** 2024-04-16T05:47:51.732 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:47:51 smithi032 podman[103358]: 2024-04-16 05:47:51.47280494 +0000 UTC m=+4.233901774 container died de09e00922c0c0bffd10d2a2f6db48b9a75cf836e21e1d50c75ae188037aac89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-16T05:47:52.104 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:47:51 smithi032 podman[103358]: 2024-04-16 05:47:51.839003652 +0000 UTC m=+4.600100495 container cleanup de09e00922c0c0bffd10d2a2f6db48b9a75cf836e21e1d50c75ae188037aac89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a, org.label-schema.vendor=CentOS, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-16T05:47:52.104 INFO:journalctl@ceph.mds.a.smithi032.stdout:Apr 16 05:47:51 smithi032 bash[103358]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-a 2024-04-16T05:47:52.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.a.service' 2024-04-16T05:47:52.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:52.298 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-16T05:47:52.298 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-16T05:47:52.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.d 2024-04-16T05:47:52.616 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:52 smithi032 systemd[1]: Stopping Ceph mds.d for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:52.982 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d[73551]: 2024-04-16T05:47:52.617+0000 7f41f1593700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:52.982 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:52 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d[73551]: 2024-04-16T05:47:52.617+0000 7f41f1593700 -1 mds.d *** got signal Terminated *** 2024-04-16T05:47:56.157 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:55 smithi032 podman[103491]: 2024-04-16 05:47:55.869236722 +0000 UTC m=+3.350188583 container stop d1d1f5c2e00536caf8f2f51aad12e9cc5e2cfa6d0730d963fe20337e9fa4db1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-04-16T05:47:56.157 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:55 smithi032 podman[103491]: 2024-04-16 05:47:55.894878831 +0000 UTC m=+3.375830711 container died d1d1f5c2e00536caf8f2f51aad12e9cc5e2cfa6d0730d963fe20337e9fa4db1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:56.482 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:56 smithi032 podman[103491]: 2024-04-16 05:47:56.376064425 +0000 UTC m=+3.857016285 container cleanup d1d1f5c2e00536caf8f2f51aad12e9cc5e2cfa6d0730d963fe20337e9fa4db1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-16T05:47:56.483 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:56 smithi032 bash[103491]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d 2024-04-16T05:47:56.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.d.service' 2024-04-16T05:47:56.822 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:56 smithi032 podman[103505]: 2024-04-16 05:47:56.568902779 +0000 UTC m=+0.692899196 container remove d1d1f5c2e00536caf8f2f51aad12e9cc5e2cfa6d0730d963fe20337e9fa4db1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff) 2024-04-16T05:47:56.822 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:56 smithi032 systemd[1]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-16T05:47:56.823 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:56 smithi032 systemd[1]: Stopped Ceph mds.d for 578203a8-fbb1-11ee-bc8f-c7b262605968. 2024-04-16T05:47:56.823 INFO:journalctl@ceph.mds.d.smithi032.stdout:Apr 16 05:47:56 smithi032 systemd[1]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.d.service: Consumed 1.099s CPU time. 2024-04-16T05:47:56.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:47:56.834 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-16T05:47:56.834 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-16T05:47:56.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.g 2024-04-16T05:47:57.153 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:47:56 smithi032 systemd[1]: Stopping Ceph mds.g for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:47:57.482 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:47:57 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g[74149]: 2024-04-16T05:47:57.154+0000 7f29c1c0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:47:57.482 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:47:57 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g[74149]: 2024-04-16T05:47:57.154+0000 7f29c1c0c700 -1 mds.g *** got signal Terminated *** 2024-04-16T05:48:01.061 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:48:00 smithi032 podman[103611]: 2024-04-16 05:48:00.801111893 +0000 UTC m=+3.745748079 container died 694d2cbe0a4abf6174198d162c08b3800361a6d5f3c760d9ad9397358876857a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-16T05:48:01.412 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:48:01 smithi032 podman[103611]: 2024-04-16 05:48:01.202595651 +0000 UTC m=+4.147231824 container cleanup 694d2cbe0a4abf6174198d162c08b3800361a6d5f3c760d9ad9397358876857a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True) 2024-04-16T05:48:01.412 INFO:journalctl@ceph.mds.g.smithi032.stdout:Apr 16 05:48:01 smithi032 bash[103611]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-g 2024-04-16T05:48:01.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.g.service' 2024-04-16T05:48:01.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:01.568 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-16T05:48:01.568 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-16T05:48:01.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.j 2024-04-16T05:48:01.732 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:48:01 smithi032 systemd[1]: Stopping Ceph mds.j for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:02.232 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:48:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j[73196]: 2024-04-16T05:48:01.890+0000 7f291bd13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:02.233 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:48:01 smithi032 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j[73196]: 2024-04-16T05:48:01.890+0000 7f291bd13700 -1 mds.j *** got signal Terminated *** 2024-04-16T05:48:05.198 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:48:04 smithi032 podman[103731]: 2024-04-16 05:48:04.944816555 +0000 UTC m=+3.154184905 container died ad68f6820de12141c28d739206c67d66ad6be6e8aa9b111fc4b4d841c6219ca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True) 2024-04-16T05:48:05.482 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:48:05 smithi032 podman[103731]: 2024-04-16 05:48:05.292088019 +0000 UTC m=+3.501456368 container cleanup ad68f6820de12141c28d739206c67d66ad6be6e8aa9b111fc4b4d841c6219ca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-16T05:48:05.483 INFO:journalctl@ceph.mds.j.smithi032.stdout:Apr 16 05:48:05 smithi032 bash[103731]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-j 2024-04-16T05:48:05.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.j.service' 2024-04-16T05:48:05.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:05.749 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-16T05:48:05.749 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-16T05:48:05.750 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.b 2024-04-16T05:48:06.055 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:48:05 smithi082 systemd[1]: Stopping Ceph mds.b for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:06.399 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:48:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b[66419]: 2024-04-16T05:48:06.056+0000 7fe999032700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:06.400 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:48:06 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b[66419]: 2024-04-16T05:48:06.056+0000 7fe999032700 -1 mds.b *** got signal Terminated *** 2024-04-16T05:48:08.613 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:48:08 smithi082 podman[73003]: 2024-04-16 05:48:08.311015803 +0000 UTC m=+2.371795174 container died e7a0a4ffdc18b322d69e6764a443a27c60a0d1cca02cd156090276d16c56679b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b, ceph=True, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS) 2024-04-16T05:48:08.897 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:48:08 smithi082 podman[73003]: 2024-04-16 05:48:08.690319644 +0000 UTC m=+2.751099006 container cleanup e7a0a4ffdc18b322d69e6764a443a27c60a0d1cca02cd156090276d16c56679b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-16T05:48:08.898 INFO:journalctl@ceph.mds.b.smithi082.stdout:Apr 16 05:48:08 smithi082 bash[73003]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-b 2024-04-16T05:48:09.026 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.b.service' 2024-04-16T05:48:09.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:09.064 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-16T05:48:09.064 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-16T05:48:09.064 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.e 2024-04-16T05:48:09.149 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:48:09 smithi082 systemd[1]: Stopping Ceph mds.e for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:09.650 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:48:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e[65792]: 2024-04-16T05:48:09.368+0000 7f92a245a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:09.650 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:48:09 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e[65792]: 2024-04-16T05:48:09.368+0000 7f92a245a700 -1 mds.e *** got signal Terminated *** 2024-04-16T05:48:12.649 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:48:12 smithi082 podman[73136]: 2024-04-16 05:48:12.320751902 +0000 UTC m=+3.040001445 container died 263d0625fb4237602cf43c56faf5847b8bf69589e8dc7f8323b4de07c6c3bd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0) 2024-04-16T05:48:13.142 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:48:12 smithi082 podman[73136]: 2024-04-16 05:48:12.91045314 +0000 UTC m=+3.629702719 container cleanup 263d0625fb4237602cf43c56faf5847b8bf69589e8dc7f8323b4de07c6c3bd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-16T05:48:13.142 INFO:journalctl@ceph.mds.e.smithi082.stdout:Apr 16 05:48:12 smithi082 bash[73136]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-e 2024-04-16T05:48:13.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.e.service' 2024-04-16T05:48:13.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:13.318 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-16T05:48:13.318 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-16T05:48:13.319 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.h 2024-04-16T05:48:13.649 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:48:13 smithi082 systemd[1]: Stopping Ceph mds.h for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:13.650 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:48:13 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h[66113]: 2024-04-16T05:48:13.638+0000 7efcc4794700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:13.650 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:48:13 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h[66113]: 2024-04-16T05:48:13.638+0000 7efcc4794700 -1 mds.h *** got signal Terminated *** 2024-04-16T05:48:16.649 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:48:16 smithi082 podman[73258]: 2024-04-16 05:48:16.312615563 +0000 UTC m=+2.778094575 container died d9882b97cc8a0900336ae7d45fa7ab602c75e6b83a070b67f7b720309bd9155b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-16T05:48:16.922 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:48:16 smithi082 podman[73258]: 2024-04-16 05:48:16.731421372 +0000 UTC m=+3.196900360 container cleanup d9882b97cc8a0900336ae7d45fa7ab602c75e6b83a070b67f7b720309bd9155b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-16T05:48:16.923 INFO:journalctl@ceph.mds.h.smithi082.stdout:Apr 16 05:48:16 smithi082 bash[73258]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-h 2024-04-16T05:48:17.053 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.h.service' 2024-04-16T05:48:17.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:17.088 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-16T05:48:17.088 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-16T05:48:17.088 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.k 2024-04-16T05:48:17.392 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:48:17 smithi082 systemd[1]: Stopping Ceph mds.k for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:17.649 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:48:17 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k[65472]: 2024-04-16T05:48:17.394+0000 7f29e4ace700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:17.650 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:48:17 smithi082 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k[65472]: 2024-04-16T05:48:17.394+0000 7f29e4ace700 -1 mds.k *** got signal Terminated *** 2024-04-16T05:48:21.659 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:48:21 smithi082 podman[73381]: 2024-04-16 05:48:21.409115999 +0000 UTC m=+4.106127439 container died c7b28af0361065d24b50eefe8efba2efc64275b2ed0d9d8d892125555ff78d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T05:48:21.936 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:48:21 smithi082 podman[73381]: 2024-04-16 05:48:21.760580561 +0000 UTC m=+4.457591994 container cleanup c7b28af0361065d24b50eefe8efba2efc64275b2ed0d9d8d892125555ff78d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T05:48:21.936 INFO:journalctl@ceph.mds.k.smithi082.stdout:Apr 16 05:48:21 smithi082 bash[73381]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-k 2024-04-16T05:48:22.161 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.k.service' 2024-04-16T05:48:22.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:22.195 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-16T05:48:22.195 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-16T05:48:22.195 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.c 2024-04-16T05:48:22.516 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:48:22 smithi134 systemd[1]: Stopping Ceph mds.c for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:23.134 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:48:22 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-c[67587]: 2024-04-16T05:48:22.770+0000 7f57fe3ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:23.134 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:48:22 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-c[67587]: 2024-04-16T05:48:22.770+0000 7f57fe3ea700 -1 mds.c *** got signal Terminated *** 2024-04-16T05:48:24.602 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:48:24 smithi134 podman[74569]: 2024-04-16 05:48:24.33124542 +0000 UTC m=+1.941000960 container died 428061409e05037214ce619a8125f3e63f3ffe8d1038f1b0d10257c6b67955cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-16T05:48:24.884 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:48:24 smithi134 podman[74569]: 2024-04-16 05:48:24.703561751 +0000 UTC m=+2.313317278 container cleanup 428061409e05037214ce619a8125f3e63f3ffe8d1038f1b0d10257c6b67955cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T05:48:24.884 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 16 05:48:24 smithi134 bash[74569]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-c 2024-04-16T05:48:25.075 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.c.service' 2024-04-16T05:48:25.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:25.114 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-16T05:48:25.114 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-16T05:48:25.115 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.f 2024-04-16T05:48:25.634 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:48:25 smithi134 systemd[1]: Stopping Ceph mds.f for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:25.635 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:48:25 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f[67278]: 2024-04-16T05:48:25.431+0000 7f52c6061700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:25.635 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:48:25 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f[67278]: 2024-04-16T05:48:25.431+0000 7f52c6061700 -1 mds.f *** got signal Terminated *** 2024-04-16T05:48:28.598 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:48:28 smithi134 podman[74704]: 2024-04-16 05:48:28.318672496 +0000 UTC m=+2.991221382 container died 3ad03c21da9a9a42a898515f24413752a7e1a24b79551f2d8fc24640c296a470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T05:48:28.884 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:48:28 smithi134 podman[74704]: 2024-04-16 05:48:28.707667754 +0000 UTC m=+3.380216615 container cleanup 3ad03c21da9a9a42a898515f24413752a7e1a24b79551f2d8fc24640c296a470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-16T05:48:28.884 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 16 05:48:28 smithi134 bash[74704]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-f 2024-04-16T05:48:29.082 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.f.service' 2024-04-16T05:48:29.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:29.116 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-16T05:48:29.117 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-16T05:48:29.117 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.i 2024-04-16T05:48:29.634 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:48:29 smithi134 systemd[1]: Stopping Ceph mds.i for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:29.634 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:48:29 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i[67899]: 2024-04-16T05:48:29.436+0000 7fd53367f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:29.634 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:48:29 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i[67899]: 2024-04-16T05:48:29.436+0000 7fd53367f700 -1 mds.i *** got signal Terminated *** 2024-04-16T05:48:32.634 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:48:32 smithi134 podman[74824]: 2024-04-16 05:48:32.326955067 +0000 UTC m=+2.989616656 container died b310c02cc59f188c4fbd5ab0d607a05df2d9284ff84f7ffa2fcfe40711748d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-16T05:48:32.934 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:48:32 smithi134 podman[74824]: 2024-04-16 05:48:32.719512321 +0000 UTC m=+3.382173891 container cleanup b310c02cc59f188c4fbd5ab0d607a05df2d9284ff84f7ffa2fcfe40711748d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T05:48:32.935 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 16 05:48:32 smithi134 bash[74824]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-i 2024-04-16T05:48:33.058 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.i.service' 2024-04-16T05:48:33.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:33.092 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-16T05:48:33.092 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-16T05:48:33.093 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.l 2024-04-16T05:48:33.202 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:48:33 smithi134 systemd[1]: Stopping Ceph mds.l for 578203a8-fbb1-11ee-bc8f-c7b262605968... 2024-04-16T05:48:33.634 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:48:33 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l[66969]: 2024-04-16T05:48:33.414+0000 7f961514d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T05:48:33.634 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:48:33 smithi134 ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l[66969]: 2024-04-16T05:48:33.414+0000 7f961514d700 -1 mds.l *** got signal Terminated *** 2024-04-16T05:48:35.538 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:48:35 smithi134 podman[74943]: 2024-04-16 05:48:35.207857827 +0000 UTC m=+1.889925226 container died d50418adbc417477dd78663270c4dffa447b40733b56478412773b46e06bb912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-16T05:48:35.884 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:48:35 smithi134 podman[74943]: 2024-04-16 05:48:35.615083217 +0000 UTC m=+2.297150606 container cleanup d50418adbc417477dd78663270c4dffa447b40733b56478412773b46e06bb912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T05:48:35.884 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:48:35 smithi134 bash[74943]: ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l 2024-04-16T05:48:35.884 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 16 05:48:35 smithi134 podman[74957]: 2024-04-16 05:48:35.776603041 +0000 UTC m=+0.564300753 container remove d50418adbc417477dd78663270c4dffa447b40733b56478412773b46e06bb912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5d703950a7674b29e5ad087e2e8d1540b6426e851a1568bf8a762af9e30ac2a1, name=ceph-578203a8-fbb1-11ee-bc8f-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=qa-debug-resource-temporarily-unavailable-issue-a87fcff, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T05:48:35.992 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-578203a8-fbb1-11ee-bc8f-c7b262605968@mds.l.service' 2024-04-16T05:48:36.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T05:48:36.026 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-16T05:48:36.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 --force --keep-logs 2024-04-16T05:48:36.226 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:48:38.679 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 --force --keep-logs 2024-04-16T05:48:38.871 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:48:41.355 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 --force --keep-logs 2024-04-16T05:48:41.546 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:48:43.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:48:43.959 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:48:43.993 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T05:48:44.022 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-16T05:48:44.025 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi032/crash 2024-04-16T05:48:44.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash -- . 2024-04-16T05:48:44.070 INFO:teuthology.orchestra.run.smithi032.stderr:tar: /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T05:48:44.070 INFO:teuthology.orchestra.run.smithi032.stderr:tar: Error is not recoverable: exiting now 2024-04-16T05:48:44.072 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi082/crash 2024-04-16T05:48:44.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash -- . 2024-04-16T05:48:44.213 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T05:48:44.213 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2024-04-16T05:48:44.215 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi134/crash 2024-04-16T05:48:44.216 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash -- . 2024-04-16T05:48:44.251 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T05:48:44.251 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-04-16T05:48:44.252 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-16T05:48:44.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-16T05:48:44.295 INFO:tasks.cephadm:Compressing logs... 2024-04-16T05:48:44.295 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T05:48:44.338 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T05:48:44.341 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T05:48:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T05:48:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:: No such file or directory 2024-04-16T05:48:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-16T05:48:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.6.log 2024-04-16T05:48:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.a.log 2024-04-16T05:48:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log 2024-04-16T05:48:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.g.log 2024-04-16T05:48:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.a.log: /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.j.log 2024-04-16T05:48:44.370 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T05:48:44.370 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2024-04-16T05:48:44.370 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.l.log 2024-04-16T05:48:44.370 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log 2024-04-16T05:48:44.371 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.i.log 2024-04-16T05:48:44.371 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.11.log 2024-04-16T05:48:44.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.3.log 2024-04-16T05:48:44.385 INFO:teuthology.orchestra.run.smithi032.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-16T05:48:44.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.x.log 2024-04-16T05:48:44.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.0.log 2024-04-16T05:48:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.x.log: 90.2% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log.gz 2024-04-16T05:48:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log 2024-04-16T05:48:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.g.log.gz 2024-04-16T05:48:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log 2024-04-16T05:48:44.386 INFO:teuthology.orchestra.run.smithi032.stderr: 94.7% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.j.log.gz/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log: 2024-04-16T05:48:44.386 INFO:teuthology.orchestra.run.smithi032.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T05:48:44.386 INFO:teuthology.orchestra.run.smithi032.stderr: 84.3% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log.gz 2024-04-16T05:48:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log 2024-04-16T05:48:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.9.log 2024-04-16T05:48:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.d.log 2024-04-16T05:48:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.9.log: 87.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log.gz 2024-04-16T05:48:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.a.log 2024-04-16T05:48:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.d.log.gz 2024-04-16T05:48:44.391 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.a.log.gz 2024-04-16T05:48:44.391 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log.gz 2024-04-16T05:48:44.393 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T05:48:44.393 INFO:teuthology.orchestra.run.smithi082.stderr:: No such file or directory 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.10.log 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.b.log 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.4.log 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.1.log 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.k.log 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.h.log 2024-04-16T05:48:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.1.log: /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.7.log 2024-04-16T05:48:44.395 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.e.log 2024-04-16T05:48:44.395 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.7.log: 90.3% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log.gz 2024-04-16T05:48:44.395 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.y.log 2024-04-16T05:48:44.395 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.k.log.gz 2024-04-16T05:48:44.395 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log 2024-04-16T05:48:44.395 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.y.log: 91.8% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.y.log.gz 2024-04-16T05:48:44.395 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log 2024-04-16T05:48:44.396 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log.gz 2024-04-16T05:48:44.396 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log 2024-04-16T05:48:44.396 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log: 87.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log.gz 2024-04-16T05:48:44.396 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.b.log 2024-04-16T05:48:44.397 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.z.log 2024-04-16T05:48:44.397 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.i.log: /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.8.log 2024-04-16T05:48:44.397 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T05:48:44.397 INFO:teuthology.orchestra.run.smithi134.stderr: -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.5.log 2024-04-16T05:48:44.397 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.f.log 2024-04-16T05:48:44.397 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log 2024-04-16T05:48:44.398 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.audit.log.gz 2024-04-16T05:48:44.398 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log 2024-04-16T05:48:44.398 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.cephadm.log.gz 2024-04-16T05:48:44.398 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.2.log 2024-04-16T05:48:44.398 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.l.log.gz 2024-04-16T05:48:44.398 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.c.log 2024-04-16T05:48:44.398 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.2.log: 87.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph.log.gz 2024-04-16T05:48:44.398 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log 2024-04-16T05:48:44.399 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.c.log: 91.9% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.z.log.gz 2024-04-16T05:48:44.399 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.c.log 2024-04-16T05:48:44.399 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log: /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log.gz 2024-04-16T05:48:44.407 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log: /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-volume.log.gz 2024-04-16T05:48:44.432 INFO:teuthology.orchestra.run.smithi134.stderr: 92.7% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.i.log.gz 2024-04-16T05:48:44.434 INFO:teuthology.orchestra.run.smithi082.stderr: 92.8% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.h.log.gz 2024-04-16T05:48:44.439 INFO:teuthology.orchestra.run.smithi134.stderr: 94.4% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.f.log.gz 2024-04-16T05:48:44.441 INFO:teuthology.orchestra.run.smithi082.stderr: 94.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.e.log.gz 2024-04-16T05:48:44.447 INFO:teuthology.orchestra.run.smithi134.stderr: 94.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.c.log.gz 2024-04-16T05:48:44.569 INFO:teuthology.orchestra.run.smithi082.stderr: 92.8% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mds.b.log.gz 2024-04-16T05:48:44.622 INFO:teuthology.orchestra.run.smithi032.stderr: 90.2% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mgr.x.log.gz 2024-04-16T05:48:44.672 INFO:teuthology.orchestra.run.smithi134.stderr: 92.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.c.log.gz 2024-04-16T05:48:44.734 INFO:teuthology.orchestra.run.smithi082.stderr: 92.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.b.log.gz 2024-04-16T05:48:44.941 INFO:teuthology.orchestra.run.smithi134.stderr: 92.9% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.11.log.gz 2024-04-16T05:48:45.076 INFO:teuthology.orchestra.run.smithi082.stderr: 92.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.7.log.gz 2024-04-16T05:48:45.208 INFO:teuthology.orchestra.run.smithi032.stderr: 92.5% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.9.log.gz 2024-04-16T05:48:45.224 INFO:teuthology.orchestra.run.smithi082.stderr: 92.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.4.log.gz 2024-04-16T05:48:45.333 INFO:teuthology.orchestra.run.smithi032.stderr: 92.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.0.log.gz 2024-04-16T05:48:45.335 INFO:teuthology.orchestra.run.smithi032.stderr: 90.6% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-mon.a.log.gz 2024-04-16T05:48:45.343 INFO:teuthology.orchestra.run.smithi082.stderr: 92.2% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.10.log.gz 2024-04-16T05:48:45.498 INFO:teuthology.orchestra.run.smithi032.stderr: 92.5% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.3.log.gz 2024-04-16T05:48:45.546 INFO:teuthology.orchestra.run.smithi134.stderr: 93.2% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.8.log.gz 2024-04-16T05:48:45.622 INFO:teuthology.orchestra.run.smithi134.stderr: 93.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.2.log.gz 2024-04-16T05:48:45.630 INFO:teuthology.orchestra.run.smithi032.stderr: 93.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.6.log.gz 2024-04-16T05:48:45.632 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-16T05:48:45.632 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m1.280s 2024-04-16T05:48:45.632 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m5.298s 2024-04-16T05:48:45.632 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m0.189s 2024-04-16T05:48:45.641 INFO:teuthology.orchestra.run.smithi082.stderr: 93.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.1.log.gz 2024-04-16T05:48:45.643 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-16T05:48:45.643 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m1.288s 2024-04-16T05:48:45.643 INFO:teuthology.orchestra.run.smithi082.stderr:user 0m4.356s 2024-04-16T05:48:45.643 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m0.169s 2024-04-16T05:48:45.758 INFO:teuthology.orchestra.run.smithi134.stderr: 93.0% -- replaced with /var/log/ceph/578203a8-fbb1-11ee-bc8f-c7b262605968/ceph-osd.5.log.gz 2024-04-16T05:48:45.759 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-16T05:48:45.759 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m1.403s 2024-04-16T05:48:45.759 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m4.749s 2024-04-16T05:48:45.759 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.181s 2024-04-16T05:48:45.759 INFO:tasks.cephadm:Archiving logs... 2024-04-16T05:48:45.760 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi032/log 2024-04-16T05:48:45.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T05:48:46.501 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi082/log 2024-04-16T05:48:46.502 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T05:48:47.144 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi134/log 2024-04-16T05:48:47.145 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T05:48:47.848 INFO:tasks.cephadm:Removing cluster... 2024-04-16T05:48:47.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 --force 2024-04-16T05:48:48.037 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:48:48.298 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 --force 2024-04-16T05:48:48.490 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:48:48.781 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 578203a8-fbb1-11ee-bc8f-c7b262605968 --force 2024-04-16T05:48:48.972 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 578203a8-fbb1-11ee-bc8f-c7b262605968 2024-04-16T05:48:49.252 INFO:tasks.cephadm:Removing cephadm ... 2024-04-16T05:48:49.253 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T05:48:49.271 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T05:48:49.287 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T05:48:49.302 INFO:tasks.cephadm:Teardown complete 2024-04-16T05:48:49.302 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-16T05:48:49.314 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-16T05:48:49.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T05:48:49.318 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T05:48:49.329 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T05:48:49.440 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-16T05:48:49.450 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-16T05:48:49.451 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:48:49.453 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:48:49.456 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T05:48:49.467 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:48:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:48:49.469 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-16T05:48:49.471 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:48:49.472 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-04-16T05:48:49.472 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +1615us[+1621us] +/- 42ms 2024-04-16T05:48:49.472 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +312us[ +318us] +/- 71ms 2024-04-16T05:48:49.472 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -3035us[-3029us] +/- 70ms 2024-04-16T05:48:49.472 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:48:49.473 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:48:49.473 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-04-16T05:48:49.474 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +1629us[+1632us] +/- 42ms 2024-04-16T05:48:49.474 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 7 +327us[ +327us] +/- 71ms 2024-04-16T05:48:49.474 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -3012us[-3012us] +/- 70ms 2024-04-16T05:48:49.474 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:48:49.475 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T05:48:49.476 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-16T05:48:49.476 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1606us[+1598us] +/- 42ms 2024-04-16T05:48:49.476 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 +302us[ +302us] +/- 71ms 2024-04-16T05:48:49.476 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 -3033us[-3033us] +/- 70ms 2024-04-16T05:48:49.476 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T05:48:49.476 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-16T05:48:49.487 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-16T05:48:49.487 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-16T05:48:49.511 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T05:48:49.609 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T05:48:49.681 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T05:48:49.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:48:49.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:48:49.786 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:48:49.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:48:49.816 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T05:48:49.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:48:49.846 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-16T05:48:49.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-16T05:48:49.894 INFO:teuthology.task.internal:Duration was 2089.475671 seconds 2024-04-16T05:48:49.895 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-16T05:48:49.928 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-16T05:48:49.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T05:48:49.930 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T05:48:49.933 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T05:48:49.973 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:48:49.976 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:48:49.976 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T05:48:50.415 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-16T05:48:50.415 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-04-16T05:48:50.415 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T05:48:50.437 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-04-16T05:48:50.437 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T05:48:50.456 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-04-16T05:48:50.457 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T05:48:50.479 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-16T05:48:50.479 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T05:48:50.481 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T05:48:50.499 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T05:48:50.547 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-16T05:48:50.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T05:48:50.552 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T05:48:50.567 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T05:48:50.944 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-16T05:48:50.955 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-16T05:48:50.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T05:48:50.983 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T05:48:51.010 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T05:48:51.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-16T05:48:51.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:48:51.049 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:48:51.052 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:48:51.074 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-04-16T05:48:51.075 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-04-16T05:48:51.102 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-04-16T05:48:51.126 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:48:51.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:48:51.145 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:48:51.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:48:51.159 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T05:48:51.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T05:48:51.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-16T05:48:51.191 INFO:teuthology.task.internal:Transferring archived files... 2024-04-16T05:48:51.192 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi032 2024-04-16T05:48:51.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T05:48:51.237 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi082 2024-04-16T05:48:51.237 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T05:48:51.274 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900/remote/smithi134 2024-04-16T05:48:51.274 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T05:48:51.312 INFO:teuthology.task.internal:Removing archive directory... 2024-04-16T05:48:51.312 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T05:48:51.314 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T05:48:51.317 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T05:48:51.359 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-16T05:48:51.373 INFO:teuthology.task.internal:Not uploading archives. 2024-04-16T05:48:51.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-16T05:48:51.397 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-16T05:48:51.397 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T05:48:51.399 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T05:48:51.401 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T05:48:51.411 INFO:teuthology.orchestra.run.smithi032.stdout: 265575 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 05:48 /home/ubuntu/cephtest 2024-04-16T05:48:51.413 INFO:teuthology.orchestra.run.smithi082.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 05:48 /home/ubuntu/cephtest 2024-04-16T05:48:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 05:48 /home/ubuntu/cephtest 2024-04-16T05:48:51.416 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-16T05:48:51.427 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-16T05:48:51.496 INFO:teuthology.nuke:Checking targets against current locks 2024-04-16T05:48:51.522 DEBUG:teuthology.nuke:shortname: smithi032 2024-04-16T05:48:51.523 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T05:48:51.547 DEBUG:teuthology.nuke:shortname: smithi082 2024-04-16T05:48:51.547 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T05:48:51.553 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:47.818500', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOUw9861A2kAugXtARfjhVPkJuj32IRx3km2XaBKw+HCrIj1EzImIZKfMFJPwyrDHls5zdNClsKAgEEZtZSxE84='} 2024-04-16T05:48:51.571 DEBUG:teuthology.nuke:shortname: smithi134 2024-04-16T05:48:51.571 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T05:48:51.581 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:47.817236', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDlv4FQ2yvg1V1SYekmUQEmzpivEfefYfw5pWw7UWEcNFS/JhgGGDT5BI0ZGTbX94+miYPd7Ex76z8iOmnMvk/Y='} 2024-04-16T05:48:51.592 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_05:02:59-fs-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7657900', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 05:04:47.815343', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/CC4ob0jVLDKDhUZ6Lm6BL6F+8cQ5JQP35kE2KKVTnpj+yO2ef8tzSCEWSuSMU1EfE/szdGmXlqW4Db7haK5s='} 2024-04-16T05:48:51.602 INFO:teuthology.orchestra.console.smithi032:Power off 2024-04-16T05:48:51.602 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T05:48:51.631 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2024-04-16T05:48:51.631 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:51.646 INFO:teuthology.orchestra.console.smithi082:Power off 2024-04-16T05:48:51.646 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T05:48:51.658 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-04-16T05:48:51.670 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-04-16T05:48:51.670 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:51.689 INFO:teuthology.orchestra.console.smithi134:Power off 2024-04-16T05:48:51.690 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T05:48:51.707 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-16T05:48:51.711 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2024-04-16T05:48:51.711 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:51.733 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-16T05:48:55.660 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:55.708 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:55.734 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:55.771 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-04-16T05:48:55.817 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-16T05:48:55.843 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-16T05:48:59.776 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:59.818 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:59.845 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T05:48:59.886 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2024-04-16T05:48:59.928 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-04-16T05:48:59.955 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-04-16T05:48:59.987 INFO:teuthology.orchestra.console.smithi032:Power off completed 2024-04-16T05:49:00.029 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-04-16T05:49:00.056 INFO:teuthology.orchestra.console.smithi134:Power off completed 2024-04-16T05:49:00.162 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 2089.4756712913513 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a87fcff6340baeedd43af126e793f3241b6400ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=dc6265b9adf940ae882a92366928e7bb status: fail success: false 2024-04-16T05:49:00.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T05:49:00.234 INFO:teuthology.run:FAIL